Construction method of industrial defect detection data set based on multiple optical schemes

Abstract

The invention discloses an industrial defect detection data set construction method based on multiple optical schemes, and the method comprises the steps: constructing a Texture-AD unsupervised anomaly detection data set, and carrying out the defect marking of an image sample; dividing a training set and a test set, and performing data enhancement on textures and objects of the image samples; according to the method, a current advanced or common deep learning algorithm is selected for evaluation, the performance of different algorithms in an actual application scene is displayed by providing a reference, the performance of the algorithms is evaluated, the problems and defects of the algorithms are exposed, and the method is used for targeted optimization in subsequent improvement. According to the method, a Texture-AD unsupervised anomaly detection data set is constructed, a pixel precision ground truth value label of an abnormal area in an image is provided, and various image-level classification and pixel-level segmentation anomaly detection methods can be evaluated in an actual industrial development scene.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and data set construction, and in particular to a method for constructing an industrial defect detection data set based on a multi-optical scheme. Background Art

[0002] Anomaly detection is a key link in industrial manufacturing and has made significant progress in recent years. Anomaly detection algorithms based on computer vision are crucial to modern AOI (automated optical inspection) equipment.

[0003] At present, there are many commonly used datasets for anomaly detection, and the scenarios and scales contained in these datasets vary greatly. For example, datasets related to cloth textures usually have a large amount of data, but they are very different from the production scenarios of actual cloth. Cloth defects include pencil marks, cuts, marker stains, water stains, black and white dots, lines, inconsistent sewing distances, and color differences caused by dyeing. For another example, as chips become an increasingly important research field worldwide, wafer defect detection has become an indispensable part of the process. Defects in wafers and metal plates include scratches, stains, and inherent manufacturing defects, which are naturally generated during the production process. Therefore, the demand for wafer defect detection datasets in the field of industrial inspection is also growing, but there are few open source wafer defect detection datasets. In addition, there are more datasets related to metal defects in industrial production, which usually include material types and are applicable to a more limited range of scenarios. There are also datasets related to crack defects, such as cracks on bridge surfaces and concrete floors.

[0004] In industrial environments, having more normal samples than defective samples will cause an imbalanced dataset, affecting model training and generalization. Obtaining a large number of defective samples is expensive and time-consuming, especially for rare defects. Current datasets may not cover all defect types, limiting the model's ability to identify abnormal defects. The complexity of industrial product appearance and potential label inconsistencies increase the challenges of defect detection.

[0005] In addition, existing data sets often cannot meet the needs of real-time response in industry, and the data that needs to be processed after the algorithm is deployed is often quite different from the data that can be obtained during the development phase, especially for industrial products with repetitive textures. As a result, defect detection models based on existing data sets may perform poorly in actual industrial quality inspections. Summary of the invention

[0006] The problem to be solved by the present invention is to provide a method for constructing an industrial defect detection dataset based on a multi-optical scheme, which is used for constructing anomaly detection datasets in actual industrial scenarios, so as to evaluate the performance of various deep learning algorithms, expose problems and deficiencies in the algorithms, and facilitate targeted optimization of the weaknesses of the algorithms in subsequent improvements.

[0007] The present invention adopts the following technical solution: a method for constructing an industrial defect detection data set based on a multi-optical solution, comprising the following steps:

[0008] S1. Dataset collection and annotation: Collect Texture-AD image samples, build anomaly detection dataset, and annotate images for defects;

[0009] S2. Divide the data set and perform data training and testing: Divide the constructed anomaly detection data set into a training set and a test set, and verify the unsupervised anomaly detection algorithm on the test set to obtain the baseline evaluation results;

[0010] S3, Dataset expansion: Perform data enhancement on the texture and objects of image samples, adjust the image size to fit the shape of the deep learning algorithm model input, and apply additional mirroring to expand the image samples of each category to a preset number;

[0011] S4, Evaluation indicators: Select the deep learning algorithm to be tested, train and test it based on the anomaly detection dataset, use image-level AUROC evaluation for image-level anomaly detection performance, and use pixel-level AUROC evaluation for anomaly localization;

[0012] S5. Data evaluation: Based on the evaluation indicators, the abnormal image classification and abnormal region segmentation under the Texture-AD dataset are evaluated to expose and analyze the problems and deficiencies of the deep learning algorithm to be tested, so as to optimize it in a targeted manner in subsequent improvements.

[0013] Preferably, in step S1, the Texture-AD image is taken by a high-resolution industrial camera and combined with two light sources, the two light sources are respectively placed at the lower positions on both sides of the camera, and the optical scheme is changed by adjusting the positions and brightness of the two light sources; the defects are defects generated in the actual production environment, including artificial defects and natural defects, and Labelme is used to annotate the image with defects, including: defect category, defect size, number of defects, etc.

[0014] Preferably, in step S2, the training set includes defect-free image samples of multiple subclasses of various categories, obtained under different lighting conditions, for learning the characteristics and differences of each subclass;

[0015] The test set contains two types of image samples: image samples with various categories of defects and image samples without defects. The imaging difference between the training stage and the actual deployment is simulated by changing the optical scheme for defect recognition.

[0016] Preferably, in step S3, the data set expansion includes the following sub-steps:

[0017] S3.1. Data enhancement: Data enhancement of the texture and objects of the original image samples, including grayscale, denoising, cropping, and color adjustment, to simulate possible changes in the training data.

[0018] The denoising process includes Gaussian filtering and median filtering, and the color adjustment includes brightness, contrast, and saturation;

[0019] S3.2, adjust image size: scale the image sample to the width and height required by the deep learning algorithm model;

[0020] S3.3. Apply additional mirroring: flip the image horizontally and vertically to increase the diversity of the dataset.

[0021] Preferably, in step S4, the deep learning algorithm includes but is not limited to SimpleNet, PyramidFlow, Mean-Shift, DRAEM, MSFlow, and EfficientAD methods;

[0022] Select a deep learning algorithm, train and test it based on the anomaly detection dataset, and obtain the image-AUROC and pixel-AUROC evaluation results as well as the anomaly detection visualization heat map.

[0023] Preferably, in step S5, through data evaluation, the evaluation results of abnormal image classification and abnormal region segmentation of various deep learning algorithms to be tested under the Texture-AD dataset are obtained, and the problem domain of algorithm failure is exposed and analyzed, so as to carry out targeted optimization of the weak points of the algorithm in the future.

[0024] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0025] 1. The Texture-AD unsupervised anomaly detection dataset constructed by the present invention provides pixel-accurate ground truth labels of abnormal areas in images, which can be used to evaluate various image-level classification and pixel-level segmentation anomaly detection methods in actual algorithm development scenarios.

[0026] 2. Based on the dataset of the present invention, several currently advanced deep learning algorithms can be evaluated. By providing a benchmark, the performance of different algorithms in actual application scenarios can be displayed, the performance of the algorithms can be evaluated, and the shortcomings of the algorithms can be exposed. Tests show that the Texture-AD unsupervised anomaly detection dataset constructed by the present invention has an excellent effect in evaluating the performance of algorithms, exposing the shortcomings of algorithms, and improving the direction of the algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A diagram of the image acquisition and defect annotation process of the present invention;

[0028] Figure 2 This is a statistical graph of the anomaly detection data set of the present invention;

[0029] Figure 3 The comparison results of the average Image-AUROC of different algorithms of the present invention on Texture-AD and MVTec are shown;

[0030] Figure 4 This is a visualization result diagram of SimpleNet according to an embodiment of the present invention;

[0031] Figure 5 A cloth image used in the embodiment of the present invention;

[0032] Figure 6 A wafer image used in an embodiment of the present invention;

[0033] Figure 7 This is the metal plate image used in the embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the application is further elaborated in detail below in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in the field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0035] In one embodiment of the present invention, the research is based on representative texture product defect detection, and the Texture-AD dataset is introduced to evaluate the effect of the unsupervised anomaly detection algorithm in practical application scenarios.

[0036] The dataset includes images of 15 different fabrics, 14 semiconductor wafers, and 10 metal plates captured under different optical schemes. It also includes more than 10 different types of defects generated in the real production process, such as scratches, wrinkles, color changes, and point defects, which are usually more difficult to detect than the defects in existing datasets.

[0037] The method for constructing an industrial defect detection data set based on a multi-optical solution in this embodiment is as follows:

[0038] S1. Dataset collection and annotation

[0039] like Figure 1As shown, Texture-AD images are taken with a high-resolution industrial camera (MV-CS200-10 GC). The optical scheme is changed by adjusting the position of the light source and the brightness of the two light sources. The cloth image contains both artificial and natural defects, while the wafer and metal plate images contain only natural defects. Labelme is used to annotate the images for defects.

[0040] like Figure 2 As shown, the anomaly detection dataset proposed in this embodiment includes 15 cloth subcategories, covering various colors, materials and texture defects, 14 different wafer subcategories and 10 metal plate subcategories. The metal plates include 5 colors, each of which has two types: brushed and matte, for a total of 10 texture subcategories.

[0041] The defects in the dataset of this embodiment are defects generated in the actual production environment, which is very valuable for the research of industrial quality inspection algorithms. Fabric defects include pencil marks, cuts, marker stains, water stains, black and white dots, lines, inconsistent sewing distances, and color differences caused by dyeing. Defects in wafers and metal plates include scratches, stains, and inherent manufacturing defects, which are naturally generated during the production process.

[0042] All images were captured using a high-resolution industrial camera (MV-CS200-10GC) with a resolution of 5472 × 3648 pixels and combined with two light sources. The optical scheme was changed by adjusting the position and brightness of the light sources. The defects in our dataset were manually annotated using the Labelme annotation tool.

[0043] To better match the defects generated during industrial manufacturing, this embodiment creates some artificial defects on the cloth, while the wafer and metal plate present naturally occurring defects. Subsequently, the images are cropped to the appropriate output size. The resolution of all images is 1024×1024 pixels. The training set images are acquired under relatively stable lighting conditions. However, for the test set, the imaging differences between the algorithm training phase and actual deployment are simulated by intentionally changing the optical scheme.

[0044] Finally, pixel-level ground truth annotations are provided for each defect image region.

[0045] S2. Divide the data set and perform training and data testing

[0046] The constructed anomaly detection dataset is divided into a training set and a test set. The training set contains only defect-free images, and the test set contains two types of images: images with various types of defects and defect-free images. A variety of popular unsupervised anomaly detection algorithms on the market are further used to verify the test set to obtain the baseline evaluation results.

[0047] Specifically, in terms of the division of the data set, this embodiment provides good production images from multiple subcategories as training sets for each category, allowing the model to learn the characteristics and differences of each subcategory. At the same time, defective images and good production images of the same category are also provided for the test set to evaluate the model's recognition ability when faced with actual defects.

[0048] Preferably, the anomaly detection data set of this embodiment contains a total of 43,120 images, of which 28,973 are used for training and 14,147 are used for testing.

[0049] For training and testing, the information available during training is the same as that of MVTec, but the subcategory labels cannot be used during testing, as shown in Table 1.

[0050] Table 1:

[0051]

[0052] like Figure 3 As shown, the Texture-AD dataset of this embodiment is compared with the currently commonly used anomaly detection MVTec dataset, and it is found that the baseline results of the Texture-AD dataset show larger range and variance in performance evaluation indicators, which means that the dataset of this embodiment has higher discrimination than MVTec and is more suitable for evaluating deep learning algorithms of different technical routes. At the same time, compared with MVTec, the evaluation results of the dataset of this embodiment are generally lower, which can better expose the problem domain of algorithm failure and help to optimize the weak points of the algorithm in a targeted manner later. It can be seen that the method of constructing the Texture-AD dataset in this embodiment is more advanced and more in line with the needs of improving the current anomaly detection algorithm.

[0053] S3, Dataset Expansion

[0054] Since deep learning-based evaluation methods are usually trained on large datasets, these methods perform data augmentation on both textures and objects. In the field of deep learning, especially for image-based evaluation methods, having a large amount of diverse training data is crucial. This is because large datasets enable the model to learn rich and diverse features, thereby improving the model's generalization ability and evaluation performance.

[0055] To achieve this goal, this embodiment adopts the following data expansion strategy:

[0056] Data augmentation: We applied a series of data augmentation techniques to the existing texture and object images. This includes grayscale, denoising (Gaussian filtering and median filtering), cropping, and color adjustment (such as brightness, contrast, saturation), etc. These operations help simulate the possible changes in the training data and make the model work better.

[0057] Resizing images: To ensure that the images can fit into the model’s input requirements, they are resized to the dimensions required by the model (512×512), which involves scaling the images to a specific width and height.

[0058] Additional mirroring was applied: images were flipped horizontally and vertically to further increase the diversity of the dataset.

[0059] Through the above data augmentation strategy, the size of the original dataset can be significantly expanded, which not only improves the sample diversity during model training, but also helps reduce the risk of overfitting and provides richer and more diverse training data for deep learning-based evaluation methods. This will help improve the performance and reliability of the model in practical applications, especially when faced with complex defects. These methods are crucial in many computer vision tasks, including but not limited to image classification, object detection, and image segmentation.

[0060] After the data set of this embodiment is expanded, more than 10,000 training images of each category of cloth, wafer, and metal plate are created.

[0061] S4. Evaluation indicators

[0062] Specifically, the image-level anomaly detection performance is measured by the standard Receiver Operating Characteristic Curve (Area Under the Receiver Operator Curve), denoted as I-AUROC; for anomaly localization, it is evaluated by pixel-level AUROC, denoted as P-AUROC.

[0063] This embodiment uses existing popular deep learning algorithms, including SimpleNet, PyramidFlow, Mean-Shift, DRAEM, MSFlow and EfficientAD, as the dataset baseline method, and performs training and testing based on the TextureAD dataset to obtain image-AUROC and pixel-AUROC evaluation results and anomaly detection visualization heat map.

[0064] Furthermore, this embodiment gives the evaluation results of abnormal image classification and abnormal region segmentation under various deep learning algorithms and dataset categories to be tested, and finds that no method can perform consistently in all texture categories. In the cloth category, SimpleNet outperforms other methods. But in the wafer category, DRAEM performs better than SimpleNet. In the metal plate category, EfficientAD leads the second place by 4.38% in I-AUROC.

[0065] S5. Data Evaluation

[0066] Based on the evaluation indicators, the abnormal image classification and abnormal region segmentation under the Texture-AD dataset are evaluated, and the evaluation results of the above-mentioned SimpleNet, PyramidFlow, Mean-Shift, DRAEM, MSFlow and EfficientAD algorithms on the Texture-AD dataset are obtained.

[0067] The SimpleNet visualization results are as follows Figure 4 As shown, the cloth image, wafer image and metal plate image used in the embodiment are respectively Figure 5 , Figure 6 , Figure 7 As shown, the upper row is a normal image, and the lower row is a defective image.

[0068] Finally, the evaluation results of abnormal image classification and abnormal region segmentation of all methods in the Texture-AD dataset are given. Table 2, Table 3, and Table 4 are the I-AUROC and P-AUROC evaluation results of cloth, wafer, and metal plate, respectively. The upper data is the I-AUROC evaluation result, and the lower data is the P-AUROC evaluation result.

[0069] Table 2:

[0070]

[0071] Table 3:

[0072]

[0073] Table 4:

[0074]

[0075] As can be seen, no method performs consistently across all texture categories: in the cloth category, SimpleNet outperforms other methods; but in the wafer category, DRAEM outperforms SimpleNet; and in the metal sheet category, EfficientAD leads the second place by 4.38% in I-AUROC.

[0076] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for constructing an industrial defect detection dataset based on a multi-optical scheme, characterized in that: The steps include: S1. Dataset collection and annotation: Collect Texture-AD image samples, build anomaly detection dataset, and annotate images for defects; S2. Divide the data set and perform data training and testing: Divide the constructed anomaly detection data set into a training set and a test set, and verify the unsupervised anomaly detection algorithm on the test set to obtain the baseline evaluation results; S3, Dataset expansion: Perform data enhancement on the texture and objects of image samples, adjust the image size to fit the shape of the deep learning algorithm model input, and apply additional mirroring to expand the image samples of each category to a preset number; S4, Evaluation indicators: Select the deep learning algorithm to be tested, train and test it based on the anomaly detection dataset, use image-level AUROC evaluation for image-level anomaly detection performance, and use pixel-level AUROC evaluation for anomaly localization; S5. Data evaluation: Based on the evaluation indicators, the abnormal image classification and abnormal region segmentation under the Texture-AD dataset are evaluated to expose and analyze the problems and deficiencies of the deep learning algorithm to be tested, so as to optimize it in a targeted manner in subsequent improvements.

2. The method for constructing an industrial defect detection data set based on a multi-optical scheme according to claim 1, characterized in that: In step S1, the Texture-AD image is taken by a high-resolution industrial camera and combined with two light sources, which are placed at the lower positions on both sides of the camera, and the optical scheme is changed by adjusting the positions and brightness of the two light sources; The defects are defects generated in an actual production environment, including artificial defects and natural defects. Labelme is used to annotate the images with defects, including: defect category, defect size, and defect number.

3. The method for constructing an industrial defect detection dataset based on a multi-optical scheme according to claim 1, characterized in that: In step S2, the training set includes defect-free image samples of various categories and multiple subcategories, obtained under different lighting conditions, for learning the characteristics and differences of each subcategory; The test set includes two types of image samples: image samples with defects of various categories and image samples without defects. The imaging difference between the training stage and the actual deployment is simulated by changing the optical scheme for defect recognition.

4. The method for constructing an industrial defect detection data set based on a multi-optical scheme according to claim 1, characterized in that: In step S3, the data set expansion includes the following sub-steps: S3.

1. Data enhancement: Data enhancement of the texture and objects of the original image samples, including grayscale, denoising, cropping, and color adjustment, to simulate possible changes in the training data. The denoising process includes Gaussian filtering and median filtering, and the color adjustment includes brightness, contrast, and saturation; S3.2, adjust image size: scale the image sample to the width and height required by the deep learning algorithm model; S3.

3. Apply additional mirroring: flip the image horizontally and vertically to increase the diversity of the dataset.

5. The method for constructing an industrial defect detection data set based on a multi-optical scheme according to claim 1, characterized in that: In step S4, the deep learning algorithm includes but is not limited to SimpleNet, PyramidFlow, Mean-Shift, DRAEM, MSFlow, and EfficientAD methods; Select the deep learning algorithm to be tested, train and test it based on the anomaly detection dataset, and obtain the image-AUROC and pixel-AUROC evaluation results as well as the anomaly detection visualization heat map.

6. The method for constructing an industrial defect detection data set based on a multi-optical scheme according to claim 5, characterized in that: In step S5, through data evaluation, the evaluation results of abnormal image classification and abnormal region segmentation of the deep learning algorithm to be tested under the Texture-AD dataset are obtained, and the problem domain of failure of the deep learning algorithm to be tested is exposed and analyzed, which is used for subsequent targeted optimization of the weak points of the algorithm.

Images