A method and system for difficult case mining in defect detection

Abstract

The application provides a method and system for defect detection and difficult case mining, comprising the following steps: S1. labeled data acquisition; S2. pre-labeling model training; S3. unlabeled data acquisition; S4. initial label generation; S5. artificial review and correction; S6. initial difficult case data generation; S7. reasoning uncertainty calculation; and S8. final difficult case data generation. The application adopts a "defect pre-labeling + artificial review" strategy, only corrects inaccurate defect contours / types, can realize low-cost defect data labeling, can realize multi-class pixel-level defect contour labeling, and solves problems such as missing labeling, slow labeling and poor consistency caused by artificial fatigue.

Description

Technical Field

[0001] This invention relates to the fields of industrial visual inspection and defect detection, and in particular to a method and system for difficult case mining. Background Technology

[0002] Product appearance defects not only severely affect the product's appearance quality but can even damage its functional characteristics, making product appearance quality inspection crucial. However, relying on subjective human observation and judgment to detect product defects suffers from problems such as high recruitment and training costs, low efficiency, inconsistent standards, susceptibility to false positives and false negatives, and difficulty in accurately adapting personnel to changing orders. Therefore, there is an urgent need for an AI-based appearance quality inspection system to replace manual labor, reduce labor costs, seamlessly integrate with automated production lines, improve inspection efficiency and production cycle time, and pinpoint defect locations for subsequent data traceability and process optimization.

[0003] Artificial intelligence is data-driven; the higher the quality of the data, the better the model's performance. Gartner predicts that by 2025, 70% of enterprises will shift from "big data" to "small and broad data." A "small and broad" data strategy enables more powerful analytics and AI, reducing an organization's reliance on big data. In the field of industrial defect detection, "small and broad" data refers to training data with minimal redundant information, high quality, and coverage of defect types and characteristics in the test scenario. However, industrial defect detection struggles to exhaustively identify all defects in a test scenario. Models trained on limited data may experience over-detection or under-detection during the inference phase, requiring validation with massive amounts of online data. Therefore, efficiently mining challenging examples after model deployment is a key factor limiting model performance.

[0004] Hard example mining involves collecting overchecked and underchecked samples during testing to facilitate subsequent addition to training samples for iterative model optimization. Without an initial training model, relying entirely on manual hard example mining methods requires significant manpower for annotation and yields limited hard example information; the richer the hard example information, the more efficient the model training. Therefore, efficiently mining hard example information during testing with low manpower costs, maximizing the information contained within each hard example, is a problem that needs to be solved. Summary of the Invention

[0005] In view of the problems in the prior art, the present invention provides a method and system for difficult case mining in defect detection, aiming to solve at least to some extent one of the technical problems in the related art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for hard case mining in defect detection includes the following steps:

[0008] S1. Acquisition of a small amount of labeled data: Acquire a small number of images containing product defects, and manually label these images;

[0009] S2. Pre-labeling model training: Train the defect pre-labeling model M using a small amount of manually labeled defect data;

[0010] S3. Acquisition of large amounts of unlabeled data: Acquiring large amounts of unlabeled data during the testing process;

[0011] S4. Initial Label Generation: Use the pre-labeled model M to infer the unlabeled data and obtain the initial pre-labeled labels L for the unlabeled data. init ;

[0012] S5. Manual review and correction: Manually review and correct the initial labels L for unlabeled data. init The correction is performed to obtain the correction label L for the unlabeled data. refine ;

[0013] S6. Initial Hard Example Data Generation: This involves generating initial labels L by comparing them with unlabeled data. init and correction label L refine ,

[0014] Obtain the difficult example data D hard ;

[0015] S7. Calculation of Inference Uncertainty: For Non-Difficult Example Data D easy And difficult example data D hard The reasoning increases noise.

[0016] Perform multiple forward propagations to calculate the inference uncertainty for each sample;

[0017] S8. Final Difficult Case Data Generation: Set an uncertainty threshold t, and select samples with uncertainty greater than the threshold t.

[0018] and D hard Combined into the final difficult sample D hard-final .

[0019] Furthermore, the small amount of labeled data acquisition mentioned in step S1 involves manually labeling a small number of images of the product to be inspected. The labeling content includes the pixel-level outline of the defect, the type of defect, etc.

[0020] Furthermore, the pre-labeled model training described in step S2 yields model M. Based on a small amount of labeled data, the methods employed include traditional algorithms, supervised algorithms, and unsupervised algorithms. The trained model is able to obtain initial information such as defect contours and defect categories.

[0021] Furthermore, the large amount of unlabeled data acquired in step S3, including defect data covering various types of products from different batches and models, was not manually labeled.

[0022] Furthermore, the initial label generation described in step S4 involves using the pre-labeled model M from step 4 to infer the unlabeled data, generating initial labels L for the unlabeled data. init It can be loaded onto public or self-developed data annotation platforms, such as LabelMe and CVAT.

[0023] Furthermore, the manual review and correction described in step S5 yields the corrected label L for the unlabeled data. refine ,include:

[0024] a. Correct pixel-level contours of defects, including adding / deleting / adjusting contour points, and adding / deleting defect contours;

[0025] b. Correct the defect category, including adding / modifying the category information for each defect.

[0026] Furthermore, the generation of initial hard case data in step S6 is achieved by comparing the initial label L. init and correction label L refine Each label information obtained from the difficult case data includes:

[0027] a. Defect contour information of difficult cases: including newly added contours (missed detection), deleted contours (over-detected), modified contours (corrected contour points, i.e. the segmentation information of the defect is not accurate enough), etc.

[0028] b. Defect category information for difficult cases: including modified categories (category errors), newly added categories, etc.

[0029] Furthermore, the inference uncertainty calculation described in step S7 applies to the non-difficult example data D. easy And difficult example data D hard The inference process adds noise by performing multiple forward propagations. The added noise includes:

[0030] a. Add noise to the data: such as random cropping, mirroring, rotation, adding Gaussian noise, blurring, etc., to obtain the output of "different data";

[0031] b. Add noise to the model: Dropout can be used to obtain outputs with "different network structures";

[0032] Furthermore, the reasoning uncertainty calculation described in step S7 involves obtaining multiple outputs from multiple inferences with added noise (data or model), and calculating the variance of the multiple outputs as the uncertainty of each sample.

[0033] Furthermore, in step S8, the final difficult case data generation involves randomly selecting a certain proportion of D. easy Sample D greater than the uncertainty threshold t easy-uncertain , with D hard and all samples D greater than the uncertainty threshold t hard-uncertain Together they form the final difficult case D. hard-final =D easy-uncertain+ D hard-uncertain+ D hard ;

[0034] Another object of the present invention is to provide a system for hard case mining in defect detection, comprising the following modules:

[0035] Small amount of labeled data acquisition module: Manually labeled a small number of images of the products to be inspected, including pixel-level outlines of defects, defect categories, etc.

[0036] Pre-labeled model training module: Based on a small amount of labeled data, the pre-labeled model is trained using traditional algorithms, supervised algorithms, and unsupervised algorithms. The model outputs initial information such as defect contours and defect categories.

[0037] Large-scale unlabeled data acquisition module: Acquires defect data covering various types of products from different batches and models;

[0038] Initial label generation module: Uses a pre-labeled model to infer from a large amount of unlabeled data to obtain initial labels;

[0039] Manual review and correction module: Load initial labels of unlabeled data onto a public or self-developed labeling platform, and manually correct them, such as the information on the defect contour and the category of the defect;

[0040] Initial difficult example generation module: By comparing each annotation information of the initial label and the corrected label of the unlabeled data, difficult example data is obtained. The information of difficult examples includes missed detections, over-detections, insufficient segmentation accuracy, new categories, and category errors.

[0041] Inference uncertainty calculation module: Add noise to the inference of non-difficult example data and difficult example data, perform multiple forward propagations, and calculate the inference uncertainty of each sample;

[0042] Final difficult case data generation module: Set an uncertainty threshold, select samples with uncertainty greater than the threshold, and combine them with the initial difficult case samples to form the final difficult case samples.

[0043] The beneficial effects of the method and system for hard case mining in defect detection provided in this application include:

[0044] By employing a "defect pre-labeling + manual review" strategy, only inaccurate defect outlines / types are corrected, enabling low-labor-cost defect data annotation. This allows for pixel-level defect outline annotation across multiple categories, resolving issues such as missed annotations, slow annotation speed, and poor consistency caused by human fatigue. The "initial pre-labeling vs. corrected labeling comparison" strategy provides rapid and flexible acquisition of difficult case information from the labeled data. The compared difficult case information is rich, including not only over-detected and under-detected samples but also samples with inaccurate segmentation, new categories, and category errors. Furthermore, difficult case samples can be categorized, facilitating subsequent iterative optimization of the model with different weights. Finally, the "inference uncertainty calculation" strategy adds noise (data noise or model noise) to the samples and calculates the variance of multiple inference results to obtain the sample uncertainty. This further mines samples beneficial to enhancing the model's robustness and generalization ability from both the data level and model structure perspectives. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of a method for difficult case mining in defect detection according to Example 1.

[0046] Figure 2 This is a schematic diagram of the mask for some difficult sample examples in Example 1.

[0047] Figure 3 This is a schematic diagram of a system for difficult case mining in defect detection, as shown in Example 2. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] Conversely, this application covers any substitutions, modifications, equivalent methods, and schemes made within the spirit and scope of this application as defined in the claims. Furthermore, to provide the public with a better understanding of this application, certain specific details are described in detail below. However, this application can be fully understood by those skilled in the art even without these detailed descriptions.

[0050] The following will combine Figure 1 This application provides a detailed description of a method and system for hard case mining in defect detection, as illustrated in the embodiments of this application. It is worth noting that the following embodiments are merely illustrative of this application and do not constitute a limitation thereof.

[0051] like Figure 1 As shown, a method for hard case mining in defect detection specifically includes the following steps:

[0052] S1. Acquisition of a small amount of labeled data: Acquire a small number of images containing product defects, and manually label these images.

[0053] Furthermore, a small number of images of the products to be inspected are manually annotated, including the pixel-level outline of the defects and the type of defects.

[0054] In this embodiment, the specific implementation is as follows: Detecting appearance defects of industrial products, including fiber filaments, damage, misaligned needles, metal shavings, etc., using a 5-megapixel industrial color camera + dome light + telecentric lens to acquire more than 300 defect images of the surface to be inspected of the industrial product, and using the open-source Labelme annotation tool to annotate the outline of the defects with polygons. The defect types include "XianWeiSi", "SunShang", "WaiZhen", and "JinShuXie", and the annotation format is JSON.

[0055] S2. Pre-labeled model training: The defect pre-labeled model M is trained using a small amount of manually labeled defect data.

[0056] Furthermore, model M is obtained based on a small amount of labeled data. The methods used include traditional algorithms, supervised algorithms, and unsupervised algorithms. The trained model can obtain initial information such as defect contours and defect categories.

[0057] In this embodiment, a "segmentation + classification" algorithm is used for coarse-grained defect detection. The backbone network consists of 8 sets of convolutional layers, instance normalization layers, ReLU layers, max pooling layers, and average pooling layers, plus fully connected layers. The model input is the region of interest (ROI) of the surface to be detected, which is 512x512x3. After training, the model output is whether there is a defect in the image and the approximate area of ​​the defect. A 128x128 sub-image region is cropped centered on the coarse-grained detection region to achieve "fine-grained segmentation and classification" of the defect. A multi-class semantic segmentation algorithm such as U2Net is used. Each pixel in the output mask represents the category corresponding to the highest probability of different defect categories, such as 0 / 1 / 2 / 3, a total of 4 defect categories, corresponding to "XianWeiSi", "SunShang", "WaiZhen", and "JinShuXie" respectively.

[0058] S3. Acquisition of large amounts of unlabeled data: Acquiring large amounts of unlabeled data during the testing process;

[0059] Furthermore, defect data covering various types of products from different batches and models were not manually labeled.

[0060] In this embodiment, the specific implementation involves acquiring over 5,000 images of five batches of products (the raw materials and processes may differ, resulting in variations in the type and form of defects) through an image acquisition system, including both defective and non-defective images.

[0061] S4. Initial Label Generation: Use the pre-labeled model M to infer the unlabeled data and obtain the initial pre-labeled labels L for the unlabeled data. init ;

[0062] Furthermore, the pre-labeled model M is used to infer the unlabeled data, generating initial labels L for the unlabeled data. init It can be loaded onto public or self-developed data annotation platforms, such as LabelMe and CVAT.

[0063] In this embodiment, the specific implementation involves building a defect detection service based on the pre-labeled model M, using the HTTP POST protocol, and modifying the source code of the LabelMe tool so that the defect detection service is called first when loading images during annotation, uploading the data to the server to obtain the pre-labeled JSON format tags L. init This information is displayed and stored in memory variables, with different defects represented by polygons of different colors. Thus, each labeled image will have a pre-labeled L. init It will be displayed.

[0064] S5. Manual review and correction: Manually review and correct the initial labels L for unlabeled data. init The correction is performed to obtain the correction label L for the unlabeled data. refine ;

[0065] Furthermore, manual review and correction were performed to obtain the correction label L for the unlabeled data. refine ,include:

[0066] a. Correct pixel-level contours of defects, including adding / deleting / adjusting contour points, and adding / deleting defect contours;

[0067] b. Correct the defect category, including adding / modifying the category information for each defect.

[0068] In this embodiment, after LabelMe loads the initial labels, the outline of each defect is manually reviewed. If it is a non-defect outline that has passed inspection, it is deleted; if a defect is not labeled, a new labeled outline is added; if the outline points are not accurate enough, the outline points are modified; if the category is incorrect, the defect category is modified; if it is an unseen category, a new category label is created, and the category label of the defect is modified simultaneously. After labeling is completed, label L is corrected. refine Save the final tags as a file *_f.json, and add L refine With L initThe modified tags after comparison are saved as *_ai.json.

[0069] S6. Initial Hard Example Data Generation: This involves generating initial labels L by comparing them with unlabeled data. init and correction label L refine The difficult example data D is obtained. hard .

[0070] Furthermore, by comparing the initial label L init and correction label L refine Each label information obtained from the difficult case data includes:

[0071] a. Defect contour information of difficult cases: including newly added contours (missed detection), deleted contours (over-detected), modified contours (corrected contour points, i.e. the segmentation information of the defect is not accurate enough), etc.

[0072] b. Defect category information for difficult cases: including modified categories (category errors), newly added categories, etc.

[0073] In this embodiment, the specific implementation involves using the status records "add", "remove", "edit", "new_class", and "error_class" in the modified tag *_ai.json to obtain difficult examples such as missed detections, overdetections, inaccurate segmentation, new categories, and incorrect categories. Figure 2 As shown.

[0074] S7. Calculation of Inference Uncertainty: For Non-Difficult Example Data D easy And difficult example data D hard The inference is increased with noise, multiple forward propagations are performed, and the inference uncertainty for each sample is calculated.

[0075] Furthermore, the inference uncertainty calculation described in step S7 applies to the non-difficult example data D. easy And difficult example data D hard The inference process adds noise by performing multiple forward propagations. The added noise includes:

[0076] a. Add noise to the data: such as random cropping, mirroring, rotation, adding Gaussian noise, blurring, etc., to obtain the output of "different data";

[0077] b. Add noise to the model: Dropout can be used to obtain outputs with "different network structures";

[0078] Furthermore, the reasoning uncertainty calculation described in step S7 involves obtaining multiple outputs from multiple inferences with added noise (data or model), and calculating the variance of the multiple outputs as the uncertainty of each sample.

[0079] In this embodiment, the data noise is achieved by randomly flipping the data horizontally / vertically and adding random pixel value perturbation, with each sample corresponding to 3 data noise outputs; dropout is used in the last layer with a dropout rate of 0.5, with each sample corresponding to 1 model noise output; the variance of the 5 output results is calculated by adding the original output.

[0080] S8. Final Difficult Example Data Generation: Set an uncertainty threshold t, select samples with uncertainty greater than the threshold t, and D hard Combined into the final difficult sample D hard-final .

[0081] Furthermore, in step S8, the final difficult case data generation involves randomly selecting a certain proportion of D. easy Sample D greater than the uncertainty threshold t easy-uncertain , with D hard and all samples D greater than the uncertainty threshold t hard-uncertain Together they form the final difficult case D. hard-final =D easy-uncertain+ D hard-uncertain+ D hard ;

[0082] In this embodiment, the specific implementation involves setting an uncertainty threshold of 0.4 and randomly sampling 5% of D. easy Sample D with uncertainty greater than 0.4 easy-uncertain , with D hard and all samples D with uncertainty greater than 0.4 hard-uncertain Together they form the final difficult case D. hard-final =D easy-uncertain+ D hard-uncertain +D hard ;

[0083] This embodiment provides a method for hard case mining in defect detection. Its advantages are: by manually reviewing the interactive annotation method, only inaccurate defect contours / types are corrected, which can greatly reduce annotation time, while obtaining pixel-level defect contours and category annotations with high consistency; the hard case generation method is fast and flexible, and the hard case information is rich, not only obtaining over-detected and under-detected samples, but also obtaining hard case samples of defects such as inaccurate segmentation, new categories, and category errors; by adding noise (data noise or model noise) to the samples and calculating the variance of multiple inference results, the uncertainty of the samples is obtained, and samples that are beneficial to enhancing the robustness and generalization ability of the model are further mined from the data level and model structure.

[0084] Example 2:

[0085] like Figure 3As shown, a system for hard case mining in defect detection specifically includes the following modules:

[0086] Small amount of labeled data acquisition module: Manually labeled a small number of images of the products to be inspected, including pixel-level outlines of defects, defect categories, etc.

[0087] In practice, a 2-megapixel monochrome industrial camera with coaxial light and FA lens is used to capture defect images of the surface to be inspected. The open-source OpenCV CVAT annotation platform is used to annotate the defects with polygons. The annotation categories are three: "LieWen", "SunShang", and "YiWu".

[0088] Pre-labeled model training module: Based on a small amount of labeled data, the pre-labeled model is trained using traditional algorithms, supervised algorithms, and unsupervised algorithms. The model outputs initial information such as defect contours and defect categories.

[0089] In practice, based on a small amount of labeled data, a segmentation + classification algorithm is used for coarse-grained detection. The model is trained until it converges, and the output consists of two parts: the classification score and the mask region of the defect. A 124x124 ROI is extracted centered on the defect region output by the coarse-grained detection. By adding a category classification branch to the UET network and performing multi-task learning, the precise defect region and category can be output simultaneously.

[0090] Large-scale unlabeled data acquisition module: Acquires defect data covering various types of products from different batches and models;

[0091] In practice, an image acquisition system was used to collect more than 2,000 images of the surfaces to be inspected from three batches of two different models. These images included defect-free images and images of various types of defects.

[0092] Initial label generation module: Uses a pre-labeled model to infer from a large amount of unlabeled data to obtain initial labels;

[0093] In practice, more than 2,000 unlabeled data images in CVAT 1.1 format, labeled_init.xml, were generated using the pre-labeled model and imported into the CVAT platform along with the labeled data.

[0094] Manual review and correction module: Load initial labels of unlabeled data onto a public or self-developed labeling platform, and manually correct them, such as the information on the defect contour and the category of the defect;

[0095] In practice, the outline points and categories of each defect are corrected manually on the CVAT platform based on the pre-labeled data to obtain the final annotation file annotations_refine.xml.

[0096] Initial difficult example generation module: By comparing each annotation information of the initial label and the corrected label of the unlabeled data, difficult example data is obtained. The information of difficult examples includes missed detections, over-detections, insufficient segmentation accuracy, new categories, and category errors.

[0097] In practice, by comparing annotations_refine.xml and annotations_init.xml, the contour points, contour regions and category information of each contour are compared to obtain difficult sample samples that are missed, over-detected, not accurately segmented, new categories, and misclassified, which can be flexibly used for subsequent iterative optimization of the model.

[0098] Inference uncertainty calculation module: Add noise to the inference of non-difficult example data and difficult example data, perform multiple forward propagations, and calculate the inference uncertainty of each sample;

[0099] In practice, data noise (mirroring, cropping, rotation, etc.) and model noise (dropout added to the last layer) are added to the inference, and together with the original output, the variance of multiple outputs is calculated.

[0100] Final difficult case data generation module: Set an uncertainty threshold, select samples with uncertainty greater than the threshold, and combine them with the initial difficult case samples to form the final difficult case samples.

[0101] In practice, an uncertainty threshold of 0.4 is set, and the initial difficult case data, all samples with uncertainty greater than the threshold, and a certain proportion of non-difficult case data samples with uncertainty greater than the threshold are used to form the final difficult case.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for hard case mining in defect detection, characterized in that, Includes the following steps: S1. Data Acquisition: Acquire images containing product defects and defect data obtained by manually annotating the images; S2. Pre-labeling model training: Train the defect pre-labeling model M using manually labeled defect data; S3. Unlabeled Data Acquisition: Acquire unlabeled data during the testing process; S4. Initial Label Generation: Use the pre-labeled model M to infer the unlabeled data and obtain the initial pre-labeled labels L for the unlabeled data. init ; S5. Manual review and correction: Manually review and correct the initial labels L for unlabeled data. init The correction is performed to obtain the correction label L for the unlabeled data. refine ; S6. Initial Hard Example Data Generation: This involves generating initial labels L by comparing them with unlabeled data. init and correction label L refine The difficult example data D is obtained. hard ; S7. Calculation of Inference Uncertainty: For Non-Difficult Example Data D easy And difficult example data D hard The inference is increased with noise, multiple forward propagations are performed, and the inference uncertainty for each sample is calculated. S8. Final Difficult Example Data Generation: Set an uncertainty threshold t, select samples with uncertainty greater than the threshold t, and D hard Combined into the final difficult sample D hard-final ; The manual review and correction described in step S5 yields the corrected label L for the unlabeled data. refine ,include: a. Correct pixel-level contours of defects, including adding / deleting / adjusting contour points, and adding / deleting defect contours; b. Correct the category of defects, including adding / modifying the category information for each defect; The reasoning uncertainty calculation described in step S7 applies to the non-difficult example data D. easy And difficult example data D hard The inference process adds noise by performing multiple forward propagations. The added noise includes: a. Add noise to the data: random cropping, mirroring, rotation, adding Gaussian noise, blurring, to obtain the output of "different data"; b. Add noise to the model: Dropout can be used to obtain outputs with "different network structures".

2. The method for hard case mining in defect detection according to claim 1, characterized in that, In step S1, the defect data is obtained by manually annotating the image; the annotation content includes the pixel-level outline of the defect and the category of the defect.

3. The method for hard case mining in defect detection according to claim 1, characterized in that, The pre-labeled model training described in step S2 yields model M, using methods including traditional algorithms, supervised algorithms, and unsupervised algorithms. The trained model can obtain initial defect contours and defect category information. The unlabeled data acquisition described in step S3 includes defect data covering various types of products from different batches and models, none of which have been manually labeled.

4. The method for hard case mining in defect detection according to claim 1, characterized in that, Step S4 involves generating initial labels using a pre-labeled model M to infer the unlabeled data, resulting in initial labels L for the unlabeled data. init Used for loading on public or self-developed data annotation platforms.

5. The method for hard case mining in defect detection according to claim 1, characterized in that, Step S6 involves generating initial hard case data by comparing it with the initial label L. init and correction label L refine Each label information obtained from the difficult case data includes: a. Defect profile information for difficult cases: including newly added profiles, deleted profiles, and modified profiles; b. Defect category information for difficult cases: including modified categories and newly added categories.

6. The method for hard case mining in defect detection according to claim 1, characterized in that, The inference uncertainty calculation described in step S7 involves inferring multiple outputs from the data with added noise and the model with added noise, and calculating the variance of the multiple outputs as the uncertainty of each sample.

7. The method for hard case mining in defect detection according to claim 1, characterized in that, Step S8 describes the generation of the final difficult case data, which involves randomly selecting a certain proportion of D. easy Sample D greater than the uncertainty threshold t easy-uncertain , with D hard and all samples D greater than the uncertainty threshold t hard-uncertain Together they form the final difficult case D. hard-final =D easy-uncertain +D hard-uncertain +D hard .

8. A system for hard case mining in defect detection, characterized in that, include: Small amount of labeled data acquisition module: Manually labeled the collected images of the products to be inspected, including the pixel-level outline of defects and the type of defects; Pre-labeled model training module: Based on labeled data, the pre-labeled model is trained using traditional algorithms, supervised algorithms, and unsupervised algorithms. The model outputs initial defect contours and defect category information. Large-scale unlabeled data acquisition module: Acquires defect data covering various types of products from different batches and models; Initial label generation module: Uses a pre-labeled model to infer from unlabeled data to obtain initial labels; Manual review and correction module: Loads initial labels for unlabeled data, and manually corrects them, including the information on defect contours and the category of defects; Initial difficult example generation module: By comparing each annotation information of the initial label and the corrected label of the unlabeled data, difficult example data is obtained. The information of difficult examples includes missed detections, over-detections, insufficient segmentation, new categories, and category errors. Inference uncertainty calculation module: Add noise to the inference of non-difficult example data and difficult example data, perform multiple forward propagations, and calculate the inference uncertainty of each sample; Final difficult case data generation module: Set an uncertainty threshold, select samples with uncertainty greater than the threshold, and combine them with the initial difficult case samples to form the final difficult case samples.

Images