A non-standard parts defect detection method based on pixel-level deep network feature matching

Abstract

The present invention relates to the field of non-standard parts defect detection methods, and in particular to a non-standard parts defect detection method based on pixel-level deep network feature matching. The method is based on a point matching and alignment algorithm based on deep features and a pixel comparison algorithm based on deep features. The network structure of the point matching and alignment algorithm includes an image encoder, a point feature extractor, a pixel feature extractor, and a pixel comparison network. The present invention proposes a novel deep learning defect detection method that can effectively make high-precision pixel-level predictions based on smaller samples. Since a pixel-level comparison strategy is adopted instead of pixel-level prediction or anomaly detection, deep learning can more easily transfer the knowledge learned from the original similar data set to the current data, thereby avoiding the overfitting problem often encountered in deep learning to a large extent.

Description

Technical Field

[0001] The present invention relates to the field of non-standard parts defect detection methods, and in particular to a non-standard parts defect detection method based on pixel-level deep network feature matching. Background Art

[0002] Deep learning, as the culmination of machine learning theory, has played an increasingly important role in people's production and life. In the field of industrial defect detection, the application of deep learning is becoming more and more extensive. Compared with traditional machine vision methods, deep learning defect detection has the advantages of high accuracy, strong robustness, and simple debugging. In the foreseeable future, industrial defect detection methods based on deep learning will play an increasingly important role in the field of industrial production.

[0003] Existing defect detection solutions are divided into two types: traditional methods and emerging deep learning methods. The traditional method mainly relies on engineers to observe defect features, manually select detection image features, and manually adjust the detection algorithm threshold to achieve the purpose of automatic defect detection. This method relies on the operation of engineers and requires a high level of experience from engineers. Moreover, difficult tasks often require engineers to debug repeatedly, which is time-consuming and costly. The deep learning defect detection method requires manual calibration of defect types and locations (or pixel attributes) and uses labeled data to train pre-designed deep learning models. When data collection and deep network training are normal, the resulting detection algorithm can better identify defects on components. However, ordinary deep learning methods are often good at predicting labels with a high degree of abstraction, such as defect types and locations, and often cannot achieve similar accuracy in pixel-level defect prediction; in addition, deep learning algorithms generally require more training data and lack the ability to quickly adapt to new detection objects. Summary of the invention

[0004] In view of the problems existing in the background technology, a non-standard defect detection method based on pixel-level deep network feature matching is proposed. This method is based on a deep learning point matching and alignment algorithm and a deep learning pixel-level comparison algorithm, which can effectively make high-precision defect pixel-level predictions based on a small sample.

[0005] The present invention proposes a non-standard parts defect detection method based on pixel-level deep network feature matching. The method is based on a point matching and alignment algorithm based on deep features and a pixel comparison algorithm based on deep features.

[0006] The network structure of the point matching and alignment algorithm includes an image encoder, a point feature extractor and an image alignment algorithm; the image encoder is used to extract basic features; the point feature extractor is used to extract point matching features based on the basic features; the image alignment algorithm calculates the transformation matrix between the test image and the standard image based on point matching and "twists" the test image to a posture aligned with the reference image.

[0007] The network structure of the pixel comparison algorithm includes an image encoder, a pixel feature extractor and a pixel comparison network; the image encoder is used to extract basic features; the pixel feature extractor is used to extract pixel comparison features based on the basic features; the pixel comparison network is used to compare the pixel features of two aligned images (one is a test image and the other is a standard image) to predict pixel-level defects.

[0008] Preferably, the above-mentioned non-standard parts defect detection method based on pixel-level deep network feature matching has a basic framework of: inputting the test image and the standard image into the deep learning point matching network, performing point matching feature comparison to obtain the aligned test image; inputting the aligned test image and the standard image into the deep learning pixel matching network to obtain the predicted defective pixel map.

[0009] Preferably, the deep learning point matching network and the deep learning pixel matching network share features.

[0010] Preferably, the above-mentioned non-standard parts defect detection method based on pixel-level deep network feature matching comprises the following steps:

[0011] S1, standard images and test images are input into the network at the same time;

[0012] S2, firstly, after being processed by the image encoder and the point feature extractor, the point matching features of the standard image and the test image are extracted and compared with each other, and the point matching and alignment algorithm then aligns the test image to the reference image according to the point matching results to obtain an aligned test image;

[0013] S3, then the pixel features of the aligned test image and the reference image are extracted by the pixel feature extractor and input into the pixel comparison network, the output of which is the predicted defect position (defect mask);

[0014] S4. Finally, as needed, the defect mask is converted back to the pose of the original test image through inverse pose transformation.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects:

[0016] The present invention proposes a non-standard defect detection method based on pixel-level deep network feature matching. The method is based on a point matching and alignment algorithm based on deep learning and a pixel-level comparison algorithm based on deep learning, which can effectively make high-precision defect pixel-level predictions based on a small sample. Traditional defect detection algorithms based on pixel property prediction are mostly based on pixel-level anomaly detection algorithms and semantic segmentation algorithms. The former cannot cope with large changes in lighting or large differences in itself; the latter requires users to provide a large number of labeled training samples, otherwise it is easy to over-fit and cause a decrease in recognition accuracy.

[0017] In contrast, the point matching deep learning model of the present invention can be trained on a data set that is unrelated to the current data and obtain better results. This is also the property of the point matching model currently recognized by academia and industry. The deep learning network based on pixel-level comparison of the present invention can more easily transfer information from other related data sets to the model of this task, thereby reducing the model's requirements on the number of labeled data and reducing the overfitting effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A network structure diagram of a point matching and alignment algorithm in an embodiment of the present invention;

[0019] Figure 2 A basic framework diagram of an embodiment of the present invention;

[0020] Figure 3 The figure is a flowchart of the steps of the point matching and alignment algorithm in one embodiment of the present invention. DETAILED DESCRIPTION

[0021] Embodiment 1

[0022] like Figure 1 As shown, the present invention proposes a non-standard part defect detection method based on pixel-level deep network feature matching. The method is based on a point matching and alignment algorithm based on deep features and a pixel comparison algorithm based on deep features.

[0023] The network structure of the point matching and alignment algorithm includes an image encoder, a point feature extractor and an image alignment algorithm; the image encoder is used to extract basic features; the point feature extractor is used to extract point matching features based on the basic features; the image alignment algorithm calculates the transformation matrix between the test image and the standard image based on point matching and "twists" the test image to a posture aligned with the reference image.

[0024] The network structure of the pixel comparison algorithm includes an image encoder, a pixel feature extractor and a pixel comparison network; the image encoder is used to extract basic features; the pixel feature extractor is used to extract pixel comparison features based on the basic features; the pixel comparison network is used to compare the pixel features of two aligned images (one is a test image and the other is a standard image) to predict pixel-level defects.

[0025] Embodiment 2

[0026] like Figure 2 As shown in the figure, the present invention proposes a non-standard defect detection method based on pixel-level deep network feature matching. The basic framework is: input the test image and the standard image into the deep learning point matching network, perform point matching feature comparison to obtain the aligned test image; input the aligned test image and the standard image into the deep learning pixel matching network to obtain the predicted defect pixel map. The deep learning point matching network and the deep learning pixel matching network share features.

[0027] Embodiment 3

[0028] like Figure 3 As shown, the present invention proposes a non-standard parts defect detection method based on pixel-level deep network feature matching, and the method steps are as follows:

[0029] S1, standard images and test images are input into the network at the same time;

[0030] S2, firstly, after being processed by the image encoder and the point feature extractor, the point matching features of the standard image and the test image are extracted and compared with each other, and the point matching and alignment algorithm then aligns the test image to the reference image according to the point matching results to obtain an aligned test image;

[0031] S3, then the pixel features of the aligned test image and the reference image are extracted by the pixel feature extractor and input into the pixel comparison network, the output of which is the predicted defect position (defect mask);

[0032] S4. Finally, as needed, the defect mask is converted back to the pose of the original test image through inverse pose transformation.

[0033] The present invention proposes a non-standard defect detection method based on pixel-level deep network feature matching. The method is based on a point matching and alignment algorithm based on deep learning and a pixel-level comparison algorithm based on deep learning, which can effectively make high-precision defect pixel-level predictions based on a small sample. Traditional defect detection algorithms based on pixel property prediction are mostly based on pixel-level anomaly detection algorithms and semantic segmentation algorithms. The former cannot cope with large changes in lighting or large differences in itself; the latter requires users to provide a large number of labeled training samples, otherwise it is easy to over-fit and cause a decrease in recognition accuracy.

[0034] In contrast, the point matching deep learning model of the present invention can be trained on a data set that is unrelated to the current data and obtain better results. This is also the property of the point matching model currently recognized by academia and industry. The deep learning network based on pixel-level comparison of the present invention can more easily transfer information from other related data sets to the model of this task, thereby reducing the model's requirements on the number of labeled data and reducing the overfitting effect.

[0035] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto, and various changes can be made within the knowledge scope of technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A non-standard parts defect detection method based on pixel-level deep network feature matching, characterized in that; This method is based on a point matching and alignment algorithm based on depth features and a pixel comparison algorithm based on depth features; The network structure of the point matching and alignment algorithm includes an image encoder, a point feature extractor, and an image alignment algorithm; the image encoder is used to extract basic features; the point feature extractor is used to extract point matching features based on basic features; The image alignment algorithm calculates the transformation matrix between the test image and the standard image based on point matching and "twists" the test image to align with the reference image; The network structure of the pixel comparison algorithm includes an image encoder, a pixel feature extractor, and a pixel comparison network; the image encoder is used to extract basic features; the pixel feature extractor is used to extract pixel comparison features based on basic features; the pixel comparison network is used to compare the pixel features of two aligned images, and then predict pixel-level defects; The basic framework is: input the test image and the standard image into the deep learning point matching network, perform point matching feature comparison to obtain the aligned test image; input the aligned test image and the standard image into the deep learning pixel matching network to obtain the predicted defect pixel map; Deep Learning Point Matching Network and Deep Learning Pixel Matching Network share features; The steps are as follows: S1, standard images and test images are input into the network at the same time; S2, firstly, after being processed by the image encoder and the point feature extractor, the point matching features of the standard image and the test image are extracted and compared with each other, and the point matching and alignment algorithm then aligns the test image to the reference image according to the point matching results to obtain an aligned test image; S3, then the pixel features of the aligned test image and the reference image are extracted by the pixel feature extractor and input into the pixel comparison network, the output of which is the predicted defect position; S4. Finally, as needed, the defect mask is converted back to the pose of the original test image through inverse pose transformation.

Images