Defect detection method based on anomaly detection algorithm

Abstract

The invention relates to a defect detection method based on an anomaly detection algorithm, and the method comprises the following steps: S1, building an offline feature library, and collecting a plurality of defect-free sample pictures; extracting feature vectors of the sample pictures, and segmenting the feature vectors into a plurality of local features; establishing a feature library, registering the local features into the feature library, and when the number of the features of the feature library exceeds a preset threshold value, replacing the features of the feature library in a dynamic updating manner, so that the number of the features of the feature library does not exceed the preset threshold value; s2, carrying out online defect detection, and collecting a to-be-detected product picture; extracting a feature vector of a to-be-detected product picture, and segmenting the feature vector into a plurality of local features Pm; and judging whether each local feature Pm of the product picture is matched with the features in the feature library or not, and judging whether the product picture to be detected has defects and defect positions or not. The size of the feature library is manually set to control the final algorithm recognition speed, and the algorithm performance is improved by improving the effectiveness of the features in the feature library.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection algorithms, and in particular to a defect detection method based on anomaly detection algorithm. Background Art

[0002] In the industrial field, some materials such as cloth, injection molded parts, and metals have smooth surfaces. However, due to the uncertainty of the manufacturing process of such materials, there may be slight defects such as bumps and oil stains on their surfaces. The use of image defect detection is a key link in quality control in such industries.

[0003] In recent years, how to use the latest artificial intelligence algorithms to replace manual work to achieve fast and accurate defect detection has continued to heat up in the industry. Object detection and object segmentation are the most commonly used artificial intelligence algorithms in defect detection. These algorithms can accurately and efficiently identify defects by learning from a large number of labeled defect samples. However, in actual scenarios, the defect categories are diverse, the number of defects has a long-tail distribution (a small number of types have a large number of samples, and most defect samples have a small number of samples), and even some serious defect samples are missing, making it difficult for the above algorithms to be implemented in the industry and replicated on the production line.

[0004] At present, when building background model and core library, a greedy strategy is used to sample feature vectors and construct a core set (Coreset) to reduce memory usage and improve computational efficiency. The feature vectors in the core set can represent the main features of the entire data set. Nearest neighbor search: In the feature space, find the nearest K neighbors (K-NN) for the newly input image patch, and judge whether the input patch is normal by comparing the feature differences of these neighbors. Abnormal score calculation: Distance calculation: Calculate the distance between the feature vector of the input patch and its K nearest neighbors as the basis for the abnormal score. The larger the distance, the greater the difference between the input patch and the normal sample, and the higher the possibility of abnormality. Threshold judgment: Set a threshold. When the abnormal score of the input patch exceeds the threshold, it is judged as abnormal; otherwise, it is judged as normal. In the process of background modeling, in order to obtain a core library with more complete information, a large amount of defect-free sample data is required for training. However, as the number of training samples increases, the core library will continue to expand, resulting in an expansion of the scope of retrieving K nearest neighbors in the abnormal score calculation step, causing the algorithm reasoning time to become longer, and ultimately reducing the detection speed. Summary of the invention

[0005] Based on this, it is necessary to provide a defect detection method based on anomaly detection algorithm to address the above problems.

[0006] A defect detection method based on anomaly detection algorithm comprises the following steps: S1. Offline feature library establishment, including: S1-1. Collecting a number of defect-free sample images; S1-2. Extracting the feature vectors of the sample images and dividing them into several local features; S1-3. Establishing a feature library, registering the local features into the feature library. When the number of features in the feature library exceeds a preset threshold, the features in the feature library are replaced by means of dynamic update, so that the number of features in the feature library does not exceed the preset threshold; S2. Online defect detection, including: S2-1. Collecting product images to be detected; S2-2. Extracting the feature vectors of the product images to be measured and dividing them into several local features Pm; S2-3. By judging whether each local feature Pm of the product image matches the features in the feature library, online judging whether there are defects and the defect positions in the product image to be measured.

[0007] Preferably, in S1-2, a pre-trained convolutional neural network model is used to extract the feature vectors of the sample images, and the feature vectors corresponding to the sample images are divided into several local features at the feature level.

[0008] Preferably, S2-3 includes the following steps: a. Calculating the nearest distance from each local feature Pm of the product image to the feature Pn in the feature library through the Euclidean distance, and defining the nearest distance as d, and defining the threshold of the feature distance as A1. If d < A1, it is considered that the local feature Pm matches the feature Pn in the feature library; if d ≥ A1, it is considered that the local feature Pm does not match the feature Pn in the feature library, and then the local feature Pm is updated to the feature library; b. Establishing a counter for each feature Pn in the feature library. When the number of local features Pm that are matched in step a is n, the counter of the feature library is incremented by n; c. Defining the current number of features in the feature library as N, manually setting the maximum number of features as N1, and the maximum number of features updated each time as N2. First, obtain the number of features that need to be updated to the feature library from step a, denoted as n; if n > N2, sort them in descending order by distance and take the first N2 features; if N + n < N1, there is no need to delete features from the feature library; if N + n > N1, then (N + n) - N1 features need to be taken out of the feature library, so as to ensure that the features in the database remain N1; the extraction rule is to sort the features in the feature library in descending order according to the validity in step b and remove the last (N + n) - N1 features; d. If there is no update in step c in this loop, or the number of iteration rounds exceeds the manually set round N3, stop, otherwise return to S1.

[0009] The benefit of the present invention lies in that: based on the patch-core algorithm, the present invention proposes a feature library management mechanism, which scores and ranks the effectiveness of features in the feature library before each update of the feature library. When the features in the feature library exceed the manually set size, new features are introduced into the feature library while deleting an equal number of features at the bottom of the ranking. By manually setting the size of the feature library, the control of the final algorithm recognition speed is achieved. In feature libraries of the same size, the effectiveness of the features in the feature library is improved, thereby improving the performance of the algorithm and ensuring that the accuracy of defect detection meets the standards. DETAILED DESCRIPTION

[0010] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0011] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only and do not represent the only implementation method.

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0013] A defect detection method based on anomaly detection algorithm comprises the following steps: S1, offline feature library establishment, including S1-1, collecting a number of defect-free sample images; S1-2, extracting the feature vectors of the sample images and dividing them into a number of local features; S1-3, establishing a feature library, registering the local features into the feature library, and when the number of features in the feature library exceeds a preset threshold, replacing the features of the feature library by dynamic updating, so that the number of features in the feature library does not exceed the preset threshold.

[0014] Specifically, this solution is to use a camera to shoot samples and obtain defect-free sample images. The number of sample images required is not limited. The number of sample images obtained in this embodiment is 1,000. When obtaining sample images, different time periods are generally selected for collection to ensure data diversity. The sample images are then stored in a computing device for subsequent analysis and processing to obtain local features to establish a feature library for comparing product defects.

[0015] Specifically, the feature vector corresponding to the input sample image is divided into several local features at the feature level, and each local feature information is extracted. The feature level specifically refers to the result of the feature layer before the fully connected layer of the sample image is extracted. The feature vector will retain certain position information before passing through the fully connected layer. For example, a sample image of 256*256*3 (length*width*RGB3 channels) becomes 32*32*1024 (length*width*number of feature channels) after extracting local features. It can be seen that the features extracted by the convolutional neural network model can be regarded as a condensed small image (32*32 is called a feature map), and the number of channels corresponding to each point has increased (1024-dimensional features). Each point in the 32*32 feature map corresponds to a sub-area (local area) of the original image of 8*8 size. The principle of judging whether this local area is abnormal is to see whether the feature (1024 dimensions) corresponding to this local area in the feature map can be found in the feature library (if so, it means it is a normal area).

[0016] After the feature library is established using the local features of a batch of defect-free sample images, a preset threshold is manually set for the feature library to limit the number of features in the feature library. When the number of features in the feature library exceeds the preset threshold, the local features Pm of the product image to be tested are used to dynamically update and replace the features in the feature library to form a new feature library. The number of times a feature in the feature library is matched is used to represent the effectiveness of the feature. By eliminating features with low effectiveness each iteration, the most effective features can be retained in the feature library of the same size, thereby controlling the algorithm time consumption and improving the performance of the algorithm.

[0017] S2 online defect detection includes S2-1, collecting the product picture to be tested; S2-2, extracting the feature vector of the product picture to be tested and dividing it into a number of local features Pm; S2-3, judging online whether the product picture to be tested has defects and the defect location by judging whether each local feature Pm of the product picture matches the feature in the feature library.

[0018] Specifically, in this embodiment, the S1-2 uses a pre-trained convolutional neural network model to extract the feature vector of the sample image, and divides the feature vector corresponding to the sample image into several local features at the feature level. The feature vector of the sample image is collected and extracted using a pre-trained convolutional neural network model, and the feature vector corresponding to the sample image is divided into several local features at the feature level, and the local features are extracted and filled into the feature library. The convolutional neural network model is one of the models of deep learning, which is mainly used in the field of computer vision, such as image classification, target detection, image segmentation, medical image analysis, automatic driving, and video analysis. Through the convolutional neural network model, the feature vector of the sample image is obtained. The feature vector is a high-dimensional vector used to represent the sample image data. It captures the key information in the image so that the sample image can be understood and processed by the computer algorithm. These feature vectors are extracted from the original sample image data through the pre-trained convolutional neural network model, and are used for subsequent image recognition, classification, retrieval and other tasks. The feature vector contains information such as the edge, texture, color, etc. of the sample image, which can provide basic data for subsequent abnormality detection.

[0019] Specifically, in this embodiment, S2-3 includes the following steps: a. Calculate the closest distance from each local feature Pm of the product image to the feature Pn in the feature library by Euclidean distance, and define the closest distance as d, define the threshold of feature distance as A1, if d<A1, it is considered that the local feature Pm matches the feature Pn in the feature library; if d≥A1, it is considered that the local feature Pm does not match the feature Pn in the feature library, and update the local feature Pm to the feature library. Among them, Euclidean distance is a method for measuring the "straight line" distance between two points in a multidimensional space, which is a prior art. Calculate the closest distance from the local feature Pm of the product image to the feature Pn in the feature library by Euclidean distance, that is, the straight line distance. Among them, the feature library contains all the features that may be involved in the defect-free sample image, so that any local feature Pm at any position of any defect-free product image can find a feature Pn with a feature distance less than a certain threshold A1 in the feature library corresponding to it.

[0020] It should be noted that the greater the distance between the local feature Pm and the feature Pn in the feature library, the more the local feature Pm needs to be updated to the feature library. If the distance between the local feature Pm and the feature Pn in the feature library is less than the threshold A1, it means that the local area of ​​the current product image has no defects, and there is no need to update the local feature Pm into the feature library. Dynamically updating and adjusting the feature Pn in the feature library based on the feature distance, rather than mechanically adding features to the feature library, effectively improves the overall performance of the algorithm.

[0021] b. Establish a counter for each feature Pn in the feature library. When the number of locally matched features Pm in step a is n, the counter of the feature library is incremented by n.

[0022] Specifically, the value of this counter represents the validity of the feature. In the initial state, the number of features fn in the feature library is N. When the number of locally matched features Pm is n, the counter is counted as N + n.

[0023] Here, the local feature Pm is the local feature of the input product image, and Pn is the feature of the feature library. When the local feature Pm of the input product image is not matched by the feature Pn in the feature library, it means that such a feature is missing in the feature library, so it needs to be updated to the feature library. If the feature Pn in the feature library has not been matched with the local feature Pm of the product image for a long time, at this time, the counter value is low, indicating that this feature Pn in the feature library is a useless feature. The specific reason is that there is probably a more representative feature in the feature library nearby, and the input local feature Pm is all matched to this better feature. Therefore, this feature Pn can be removed from the feature library.

[0024] c. Define the current number of features in the feature library as N, manually set the maximum number of features as N1, and the maximum number of features updated each time as N2. First, obtain the number of features that need to be updated to the feature library from step a, denoted as n. If n > N2, sort them in descending order by distance and take the first N2 features. If N + n < N1, there is no need to delete features from the feature library. If N + n > N1, then (N + n) - N1 features need to be taken out of the feature library to ensure that the number of features in the database remains N1. The rule for taking out is to sort the features in the feature library in descending order according to the validity in step b and remove the last (N + n) - N1 features.

[0025] Specifically, this step is mainly used for evaluating the validity of features in the feature library. When n > N2, that is, the number of features that can be updated at one time is greater than the maximum number of updates per time, take the first N2 features with the largest distance. The purpose is to ensure that the maximum number of features updated at one time is N2, but there is also a limit of N1 (the maximum number of features in the feature library). So if it exceeds this number after the update, (N1 - (N + n)) features need to be taken out. If n < N2, all should be updated into the feature library. If N + n < N1, it means that the features after the update will not exceed the manually set maximum number of features.

[0026] d. If there is no update in step c in this loop, or the number of iteration rounds exceeds the manually set round N3, then stop; otherwise, return to S1. Specifically, this step is used for judging the iteration stop.

[0027] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A defect detection method based on anomaly detection algorithm, characterized in that: It includes the following steps: S1. Offline feature library establishment, including S1-1. Collecting a number of defect-free sample pictures; S1-2. Extracting the feature vectors of the sample pictures and dividing them into several local features; S1-3. Establishing a feature library, registering the local features into the feature library. When the number of features in the feature library exceeds a preset threshold, the features in the feature library are replaced by a dynamic update method so that the number of features in the feature library does not exceed the preset threshold. S2. Online defect detection, including S2-1. Collecting product pictures to be detected; S2-2. Extracting the feature vectors of the product pictures to be measured and dividing them into several local features Pm; S2-3. Online judging whether there are defects and the defect positions in the product pictures to be measured by judging whether each local feature Pm of the product pictures matches the features in the feature library.

2. A defect detection method based on anomaly detection algorithm as claimed in claim 1, characterized in that: In S1-2, a pre-trained convolutional neural network model is used to extract the feature vectors of the sample pictures, and the feature vectors corresponding to the sample pictures are divided into several local features at the feature level.

3. The defect detection method based on anomaly detection algorithm according to claim 1, characterized in that: S2-3 includes the following steps: a. Calculating the nearest distance from each local feature Pm of the product pictures to the feature Pn in the feature library through the Euclidean distance, and defining the nearest distance as d, and defining the threshold of the feature distance as A1. If d < A1, it is considered that the local feature Pm matches the feature Pn in the feature library; if d ≥ A1, it is considered that the local feature Pm does not match the feature Pn in the feature library, and then the local feature Pm is updated to the feature library. b. Establishing a counter for each feature Pn in the feature library. When the number of local features Pm that are matched in step a is n, the counter of the feature library is incremented by n. c. Defining the current number of features in the feature library as N, manually setting the maximum number of features as N1, and the maximum number of features updated each time as N2. First, obtaining the number of features that need to be updated to the feature library from step a, denoted as n; if n > N2, arranging them in descending order according to the distance and taking the first N2 features. If N + n < N1, there is no need to delete features from the feature library; if N + n > N1, it is necessary to take out (N + n) - N1 features from the feature library to ensure that the number of features in the database remains N1; the rule for taking out is to arrange the features in the feature library in descending order according to the validity in step b and remove the last (N + n) - N1 features. d. If there is no update in step c in this loop, or the number of iteration rounds exceeds the manually set round N3, stop; otherwise, return to S1.