Defect detection method based on local vector retrieval

Abstract

The invention provides a defect detection method based on local vector retrieval, and the method comprises the following steps: S1, collecting and marking industrial defect pictures, and building an industrial detection data set; s2, constructing a feature extraction model Backbone, and training the model by using an industrial detection data set; s3, after the model is trained by utilizing the comparison loss, generating a normal feature library and a defect feature library in an off-line manner; s4, detecting the test image by using the normal feature library and the defect feature library; and S5, generating a detection result, and ending. According to the defect detection method based on local vector retrieval, defects are detected in a vector retrieval mode, and the defects can be compared with positive sample features, so that unknown defects are detected; the distance between the features is used as a detection value, the interpretability is high, the detection effect is stable, and the method is not easily influenced by background textures; the method has the advantages of high detection precision and strong robustness, and for false detection and missing detection data, retraining is not needed, only the feature library needs to be updated offline, and the usability is stronger.

Description

Technical Field

[0001] The present invention relates to an industrial defect detection method, in particular to a defect detection method based on local vector retrieval. Background Art

[0002] Product quality control is crucial in modern industrial production, especially when it comes to defect detection. As manufacturing processes become increasingly complex, traditional manual inspection methods are becoming increasingly inadequate and susceptible to human error, leading to missed and false detections. Consequently, automated defect detection technology has emerged.

[0003] In recent years, deep learning-based methods have demonstrated significant advantages in industrial defect detection. Leveraging architectures such as convolutional neural networks (CNNs), deep learning can automatically extract image features to identify and classify different types of defects. For example, many studies have employed CNNs for end-to-end defect detection, training models with extensive labeled data to improve detection accuracy and robustness. These methods are effective in addressing challenges such as complex backgrounds, low contrast, and irregular shapes.

[0004] While deep learning methods have achieved some success in defect detection, they still face challenges such as high data annotation costs and significant computational resource requirements. Furthermore, current methods struggle to detect unknown defects, are prone to false positives and missed detections, and have low interpretability. Summary of the Invention

[0005] The present invention provides a defect detection method based on local vector retrieval, which solves the problem of improving the interpretability of the model and detecting unknown defects through vector retrieval, thereby improving the accuracy and robustness of industrial detection. The technical solution is as follows:

[0006] A defect detection method based on local vector retrieval includes the following steps:

[0007] S1: Collect and annotate industrial defect images and build an industrial inspection dataset;

[0008] S2: Build a feature extraction model, Backbone, and train it using an industrial inspection dataset.

[0009] S3: After training the model using contrast loss, generate the normal feature library and defect feature library offline;

[0010] S4: Use the normal feature library and the defect feature library to detect the test image;

[0011] S5: Generate detection results and end.

[0012] Furthermore, in step S2, the model is trained using the industrial inspection dataset, including the following steps:

[0013] S21: Send an industrial image X to the feature extraction model Backbone. After passing through the visual base model and linear layer of the feature extraction model Backbone, the corresponding feature map F is obtained. The size of the feature map F is Where H and W represent the height and width of the industrial image X, respectively, and N is the downsampling multiple of the visual basic model;

[0014] S22: Based on the defect position annotation information of the industrial image X, the defect vector corresponding to each pixel is obtained and merged to obtain the defect feature F pos , the defect feature F pos The dimension is m×512, where m represents the number of defect features;

[0015] The area outside the defect position is the feature vector of the normal sample, which is merged to obtain the normal feature F neg , the normal feature F neg The dimension is n×512, where n represents the number of normal features;

[0016] S23: Calculate the first similarity matrix M between the defect feature and the normal feature through matrix multiplication pn =F pos ×F neg , whose dimension is m×n; then calculate the second similarity matrix M between each defect feature and other defect features in turn p , whose dimension is m×(m-1); the first similarity matrix M pn and the second similarity matrix M p The third similarity matrix M is obtained by concatenating and combining, and its dimension is m×(n+m-1);

[0017] S24: Calculate the contrast loss L based on the third similarity matrix.

[0018]

[0019] L represents contrast loss, i and j represent integers, which are in the summation symbol; y and M are both two-dimensional matrices, y ij and M ij Indicates that the specific value in the third similarity matrix M is obtained according to the index ij, and τ represents the temperature coefficient, which is set to 0.07 by default.

[0020] Furthermore, in step S24, y also represents the label corresponding to the similarity. The label corresponding to the similarity between the defect features and the defect features is 1, and the label corresponding to the similarity between the defect features and the normal features is 0. The contrast loss L increases the similarity between the defect features and reduces the similarity between the defect features and the normal features.

[0021] Furthermore, in step S3, after the model is trained using contrast loss, a normal feature library and a defect feature library are generated offline, including the following steps:

[0022] S31: Generate normal feature library using greedy algorithm;

[0023] S32: Generate a defect feature library using the KNN clustering algorithm.

[0024] Furthermore, in step S31, a greedy algorithm is used to generate a normal feature library, which includes the following steps:

[0025] S311: After the feature extraction model Backbone extracts the features, the normal features are F A , its dimension is n1×512, where n1 represents the number of normal features;

[0026] S312: Randomly select a feature vector F1, calculate its distance to the other n1-1 feature vectors, and select the feature vector F2 with the largest distance to the feature vector F1;

[0027] S313: Then calculate the distance between the eigenvector F2 and the other n1-2 eigenvectors, and select the eigenvector F3 with the largest distance to F2;

[0028] S314: And so on, finally we get n A feature vectors as the normal feature library.

[0029] Furthermore, in step S32, the defect feature library is generated using the KNN clustering algorithm, which includes the following steps:

[0030] S321: After the feature extraction model Backbone extracts features, the features of all defects are spliced ​​and merged together, denoted as F B , whose dimension is n2×512;

[0031] S322: Using KNN clustering algorithm, get n B Cluster centers are used as defect feature libraries.

[0032] Furthermore, in step S4, the normal feature library and the defect feature library are used to detect the test image, including the following steps:

[0033] S41: Send the test image to the feature extraction model Backbone to extract features and obtain the feature map F C , whose dimensions are

[0034] S42: Calculate feature map F C The cosine distance M between the corresponding feature matrix and the defect feature library CB, whose dimensions are Then in n B The smallest distance is taken as the detection value of each pixel, and then the detection value is compared with the set first threshold pixel by pixel. If it is less than the first threshold, it means that the pixel has a defect. The first threshold refers to the similarity threshold with the defect feature.

[0035] S43: Calculate feature map F C The cosine distance M between the corresponding feature matrix and the normal feature library CA , whose dimensions are Then in n A The smallest distance is taken as the detection value of each pixel, and then the detection value is compared with the set second threshold pixel by pixel. If it is greater than the second threshold, it means that the pixel has a defect. The second threshold refers to the similarity threshold with the normal feature.

[0036] Furthermore, in step S42, if the value of the first threshold is set low, more pixels will be detected, and if the value of the first threshold is set high, fewer pixels will be detected.

[0037] Furthermore, in step S43, the test pattern F C The greater the difference between the feature matrix and the normal feature library, the higher the probability that there is a defect in the area.

[0038] The defect detection method based on local vector retrieval has the following advantages:

[0039] (1) Defects are detected by vector retrieval, which can be compared with the features of positive samples to detect unknown defects.

[0040] (2) Using the distance between features as the detection value has strong interpretability, stable detection effect, and is not easily affected by background texture.

[0041] (3) It has the advantages of high detection accuracy and strong robustness. In addition, for false detection and missed detection data, there is no need for retraining, only offline updating of the feature library is required, which makes it easier to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of the defect detection method based on local vector retrieval. DETAILED DESCRIPTION

[0043] like Figure 1 As shown, the defect detection method based on local vector retrieval includes the following steps:

[0044] S1: Establish industrial inspection dataset;

[0045] The data source of the industrial inspection dataset is to collect industrial pictures containing various defects, mark them in the industrial pictures, obtain defect location marking information, and reflect the specific location information of different defects.

[0046] S2: Build a feature extraction model, Backbone, and train it using an industrial inspection dataset.

[0047] The feature extraction model Backbone consists of a linear layer and various currently available visual basic models, including but not limited to ResNet, VGG, DINOv2, etc. Considering that different visual models extract different numbers of feature channels, the feature channels extracted by the visual basic model are converted to 512 dimensions through a linear layer.

[0048] Training the model using an industrial inspection dataset involves the following steps:

[0049] S21: Send an industrial image X to the feature extraction model Backbone. After passing through the visual base model and linear layer of the feature extraction model Backbone, the corresponding feature map F is obtained. The size is Where H and W represent the height and width of the industrial image X, respectively, and N is the downsampling multiple of the visual basic model;

[0050] S22: Based on the defect position annotation information of the industrial image X, the defect vector corresponding to each pixel is obtained, and these defect vectors are combined to obtain the defect feature F corresponding to the defect. pos , the defect feature F pos The dimension is m×512, where m represents the number of defect features.

[0051] The defect location annotation information is represented in the feature map F as a binary mask of the same size as the feature map. It is represented in the feature map as a mask, with each pixel value of 1 indicating a defective area and a value of 0 indicating a non-defective area. The defect vector corresponding to each pixel refers to the feature value of the corresponding pixel in the network feature map, which is a 512-dimensional vector extracted by a convolutional neural network. The defect vector merging forms a set of defect vectors. The defect feature is two-dimensional, indicating that there are m feature vectors, each of which has a dimension of 512.

[0052] The other regions are the feature vectors of normal samples, which are merged to obtain the normal feature F neg , the dimension is n×512, where n represents the number of normal features.

[0053] S23: First, calculate the similarity M between the defect feature and the normal feature through matrix multiplication pn =Fpos ×f neg , whose dimension is m×n; then calculate the similarity M between each defect feature and other defect features in turn p , whose dimension is m×(m-1); and the two are spliced ​​together to obtain the final similarity matrix M, whose dimension is m×(n+m-1), and the similarity matrix M is a two-dimensional matrix.

[0054] S24: After obtaining the similarity matrix, calculate the contrast loss, as shown in formula (1):

[0055]

[0056] L represents contrast loss, i and j represent integers, which are in the summation symbol; y and M are both two-dimensional matrices. Furthermore, y also represents the label corresponding to the similarity. The similarity between defect features and defect features corresponds to the label 1, and the similarity between defect features and normal features corresponds to the label 0. ij and M ij Indicates the specific value in the two-dimensional matrix is ​​retrieved based on the index ij. τ represents the temperature coefficient, which is set to 0.07 by default. The contrast loss L forces the network to distinguish between defect features and normal features, that is, to increase the similarity between defect features and reduce the similarity between defect features and normal features.

[0057] S3: After training the model using contrast loss, a normal feature library and a defect feature library are generated respectively;

[0058] S31: First, use the greedy algorithm to generate a normal feature library. Suppose that after the model extracts features, the normal feature is F A , whose dimension is n1×512, where n1 represents the number of normal features. First, randomly select a feature vector F1, calculate its distance with the other n1-1 feature vectors, and select the feature vector F2 with the largest distance to feature vector F1; then calculate the distance between feature vector F2 and the other n1-2 feature vectors, and select the feature vector F3 with the largest distance to F2; and so on, finally get n A feature vectors as the normal feature library;

[0059] S32: Use KNN clustering algorithm to generate defect feature library. Suppose that after the model extracts features, all defect features are spliced ​​and merged together, denoted as F B , whose dimension is n2×512. Then use KNN clustering algorithm to get n B Cluster centers are used as defect feature libraries.

[0060] S4: Using the normal feature library and defect feature library obtained in step S3, the test image is inspected;

[0061] S41: First, the test image is sent to the feature extraction model to extract features and obtain the feature map F C , whose dimensions are

[0062] S42: First calculate the feature map F C The cosine distance M between the corresponding feature matrix and the defect feature library CB , whose dimensions are Then in n B The smallest of these distances is taken as the detection value for each pixel. The detection value is then compared pixel by pixel with a pre-set first threshold (the threshold for similarity with the defect feature). If the value is less than the threshold, the pixel is defective. The closer the test image features are to the defect features, the higher the probability of a defect in that area. Setting the threshold too low results in more detections, while setting it too high results in fewer detections.

[0063] S43: Calculate feature map F C The cosine distance M between the corresponding feature matrix and the normal feature library CA , whose dimensions are Then in n A The smallest of these distances is taken as the detection value for each pixel. The detection value is then compared pixel by pixel with a pre-set second threshold (the similarity threshold with normal features). If the value is greater than the threshold, it indicates that the pixel is defective. In other words, the greater the difference between the test image's features and normal features, the higher the probability that a defect exists in that area. By comparing with normal features, unknown defects not seen during training can be detected, making the model's detection effect more stable.

[0064] S5: Generate detection results.

[0065] The present invention detects defects through vector retrieval, which can be compared with positive sample features to detect unknown defects; it uses the distance between features as the detection value, which has strong interpretability, stable detection effect, and is not easily affected by background texture; it has the advantages of high detection accuracy and strong robustness, and for false detection and missed detection data, there is no need for retraining, only offline updating of the feature library, which is more user-friendly.

Claims

1. A defect detection method based on local vector retrieval, comprising the following steps: S1: Collect and annotate industrial defect images and build an industrial inspection dataset; S2: Build a feature extraction model Backbone and train the model using industrial inspection datasets; S3: After training the model using contrast loss, generate the normal feature library and defect feature library offline; S4: Detect the test image using the normal feature library and the defect feature library; S5: Generate detection results and end.

2. The defect detection method based on local vector retrieval according to claim 1 is characterized in that: In step S2, the model is trained using the industrial inspection data set, including the following steps: S21: Send an industrial image X to the feature extraction model Backbone. After passing through the visual base model and linear layer of the feature extraction model Backbone, the corresponding feature map F is obtained. The size of the feature map F is Where H and W represent the height and width of the industrial image X, respectively, and N is the downsampling multiple of the visual basic model; S22: According to the defect position annotation information of the industrial image X, the defect vector corresponding to each pixel is obtained and combined to obtain the defect feature F pos , the defect feature F pos The dimension is m×512, where m represents the number of defect features; The area outside the defect position is the feature vector of the normal sample, which is merged to obtain the normal feature F neg , the normal characteristic F neg The dimension is n×512, where n represents the number of normal features; S23: Calculate the first similarity matrix M between the defect feature and the normal feature through matrix multiplication pn =F pos ×F neg , whose dimension is m×n; then calculate the second similarity matrix M between each defect feature and other defect features in turn p , whose dimension is m×(m-1); the first similarity matrix M pn and the second similarity matrix M p The third similarity matrix M is obtained by concatenating and combining, and its dimension is m×(n+m-1); S24: Calculate the contrast loss L according to the third similarity matrix. L represents contrast loss, i and j represent integers, which are in the summation symbol; y and M are both two-dimensional matrices, y ij and M ij It indicates that the specific value in the third similarity matrix M is obtained according to the index ij, and τ indicates the temperature coefficient, which is set to 0.07 by default.

3. The defect detection method based on local vector retrieval according to claim 2 is characterized in that: In step S24, y represents the label corresponding to the similarity, the label corresponding to the similarity between the defect features is 1, and the label corresponding to the similarity between the defect features and the normal features is 0. The contrast loss L increases the similarity between the defect features and reduces the similarity between the defect features and the normal features.

4. The defect detection method based on local vector retrieval according to claim 1 is characterized in that: In step S3, after the model is trained using contrast loss, a normal feature library and a defect feature library are generated offline, including the following steps: S31: Generate a normal feature library using a greedy algorithm; S32: Generate a defect feature library using the KNN clustering algorithm.

5. The defect detection method based on local vector retrieval according to claim 4 is characterized in that: In step S31, a normal feature library is generated using a greedy algorithm, including the following steps: S311: After the feature extraction model Backbone extracts the features, the normal feature is F A , whose dimension is n1×512, where n1 represents the number of normal features; S312: randomly select a feature vector F1, calculate its distance with other n1-1 feature vectors, and select the feature vector F2 with the largest distance to feature vector F1; S313: Then calculate the distance between the eigenvector F2 and other n1-2 eigenvectors, and select the eigenvector F3 with the largest distance to F2; S314: And so on, finally we get n A feature vectors as the normal feature library.

6. The defect detection method based on local vector retrieval according to claim 4 is characterized in that: In step S32, the defect feature library is generated using the KNN clustering algorithm, including the following steps: S321: Backbon after feature extraction model e After extracting the features, the features of all defects are combined and recorded as F B , whose dimension is n2×512; S322: Using KNN clustering algorithm, get n B Cluster centers are used as defect feature libraries.

7. The defect detection method based on local vector retrieval according to claim 1 is characterized in that: In step S4, the test image is detected using the normal feature library and the defect feature library, including the following steps: S41: Send the test image to the feature extraction model Backbone to extract features and obtain the feature map F C , whose dimensions are S42: Calculate feature map F C The cosine distance M between the corresponding feature matrix and the defect feature library CB , whose dimensions are Then in n B The smallest distance is taken as the detection value of each pixel, and then the detection value is compared with the set first threshold pixel by pixel. If it is less than the first threshold, it means that the pixel has a defect. The first threshold refers to the similarity threshold with the defect feature. S43: Calculate feature map F C The cosine distance M between the corresponding feature matrix and the normal feature library CA , whose dimensions are Then in n A The smallest distance is taken as the detection value of each pixel, and then the detection value is compared with the set second threshold pixel by pixel. If it is greater than the second threshold, it means that the pixel has a defect. The second threshold refers to the similarity threshold with the normal feature.

8. The defect detection method based on local vector retrieval according to claim 7 is characterized in that: In the step S42, if the value of the first threshold is set low, more pixels will be detected, and if the value of the first threshold is set high, fewer pixels will be detected.

9. The defect detection method based on local vector retrieval according to claim 7 is characterized in that: In step S43, the test pattern F C The greater the difference between the feature matrix and the normal feature library, the higher the probability that there is a defect in the area.

Images