A defect detection evaluation method based on image segmentation

Abstract

This invention relates to the field of surface defect detection technology, specifically to a defect detection evaluation method based on image segmentation, comprising the following steps: calculating the cumulative overlap score of the segmentation prediction map and the label map; calculating the cumulative over-detection rate of the segmentation prediction map and the label map; and obtaining image segmentation indicators based on the cumulative overlap score and the cumulative over-detection rate. This invention effectively evaluates the segmentation of small defects without affecting the structure of the image segmentation method. Furthermore, this method can additionally calculate the over-detection rate. The cumulative overlap score can equalize the importance of small and large defects, and the calculation of the cumulative over-detection rate can evaluate the over-detection rate of the model, thereby solving the problem of poor segmentation performance in existing defect detection methods.

Description

Technical Field

[0001] This invention relates to the field of surface defect detection technology, and in particular to a defect detection and evaluation method based on image segmentation. Background Technology

[0002] Defect detection is generally divided into three tasks: image classification, object detection, and image segmentation.

[0003] Image segmentation primarily focuses on class segmentation, treating different instances of the same class as a single entity. Defect detection, however, considers not only the defect category but also different instances of the same category, treating each instance as an independent entity. Image segmentation typically uses Intersection over Union (IOU) as its evaluation metric. IOU only considers the class level; therefore, if the differences between different instances of the same class are too large, IOU becomes insensitive to missing small defects. However, defect detection precisely needs to address this, as small and large defects are generally of equal importance in defect detection tasks. Furthermore, defect detection tasks are not overly concerned with the segmentation effect of a single defect instance (i.e., whether the IOU value of a defect instance is 0.5 or 1 is not significantly different; the key is that most of the defect instance's area is segmented). Therefore, there is an urgent need to develop an image segmentation metric suitable for defect detection. Summary of the Invention

[0004] The purpose of this invention is to provide a defect detection and evaluation method based on image segmentation, which aims to solve the problem of poor segmentation performance in existing defect detection methods.

[0005] To achieve the above objectives, this invention provides a defect detection and evaluation method based on image segmentation, comprising the following steps:

[0006] Calculate the cumulative overlap score of the segmentation prediction map and the label map;

[0007] Calculate the cumulative overkill rate of the segmentation prediction map and the label map;

[0008] The image segmentation index is obtained based on the segmentation cumulative overlap score and the segmentation cumulative overkill rate.

[0009] The calculation of the segmentation cumulative overlap score of the segmentation prediction map and the label map includes:

[0010] The segmentation prediction map and the label map are divided into multiple first sub-images and multiple second sub-images;

[0011] Multiple first sub-images and multiple second sub-images are labeled to obtain multiple first labeled sub-images and multiple second labeled sub-images;

[0012] Extract defect images from multiple sub-images labeled with the second number to obtain the first image set;

[0013] Based on the first image set, extract the corresponding defect images from multiple sub-images labeled with the first number to obtain the second image set;

[0014] The segmentation overlap score is calculated based on the first image set and the second image set.

[0015] The step of calculating the segmentation overlap score based on the first image set and the second image set includes:

[0016] The first sequence value is obtained by calculating based on the first image set and the second image set;

[0017] The first sequence value is set to obtain the second sequence value;

[0018] The second sequence value is accumulated and then divided by the sequence value length to obtain the segmentation overlap score.

[0019] The calculation of the cumulative overkill rate of the segmentation prediction map and the label map includes:

[0020] The segmentation prediction map and the label map are divided into multiple third sub-images and multiple fourth sub-images;

[0021] The multiple third sub-images and the multiple fourth sub-images are labeled to obtain multiple third labeled sub-images and multiple fourth labeled sub-images;

[0022] Defect images are extracted from multiple fourth-labeled sub-images to obtain a third image set;

[0023] Based on the third image set, extract the corresponding defect images from multiple third labeled sub-images to obtain the fourth image set;

[0024] The segmentation cumulative overkill rate is calculated by performing calculations on the third and fourth image sets.

[0025] The calculation of the segmentation cumulative overkill rate for the third and fourth image sets includes:

[0026] The first overkill rate sequence is obtained by calculating based on the third and fourth image sets.

[0027] The cumulative overkill rate sequence value is obtained by summing the first overkill rate sequence value and dividing it by the sequence value length.

[0028] This invention provides a defect detection evaluation method based on image segmentation. It calculates the cumulative overlap score of the segmentation prediction map and the label map; calculates the cumulative over-detection rate of the segmentation prediction map and the label map; and obtains an image segmentation index based on the cumulative overlap score and the cumulative over-detection rate. This invention effectively evaluates the segmentation of small defects without affecting the structure of the image segmentation method. Furthermore, this method can additionally calculate the over-detection rate. The cumulative overlap score can equalize the importance of small and large defects, and the calculation of the cumulative over-detection rate can evaluate the over-detection rate of the model, thus solving the problem of poor segmentation performance in existing defect detection methods. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of a defect detection and evaluation method based on image segmentation provided by the present invention.

[0031] Figure 2 This is a flowchart for calculating the cumulative overlap score.

[0032] Figure 3 This is a flowchart for calculating the cumulative overkill rate.

[0033] Figure 4 It is a segmentation prediction graph.

[0034] Figure 5 It's a label image. Detailed Implementation

[0035] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0036] Please see Figures 1 to 5 This invention provides a defect detection and evaluation method based on image segmentation, comprising the following steps:

[0037] S1 calculates the cumulative overlap score of the segmentation prediction map and the label map;

[0038] S11 divides the segmentation prediction map and the label map into multiple first sub-images and multiple second sub-images;

[0039] Specifically, the segmentation prediction image and the label image are each divided into n*n sub-images with equal area.

[0040] S12 labels the multiple first sub-images and multiple second sub-images to obtain multiple first labeled sub-images and multiple second labeled sub-images;

[0041] Specifically, the multiple sub-images are labeled with serial numbers in sequence.

[0042] S13 Extracts defect images from multiple sub-images labeled with the second number to obtain a first image set;

[0043] Specifically, extract the set A of sub-images containing defects from all the labeled sub-images, obtain their serial numbers, and obtain a set of labeled sub-image serial numbers containing defects.

[0044] S14 Extracts the corresponding defect images from multiple first labeled sub-images based on the first image set to obtain the second image set;

[0045] Specifically, based on the sequence set obtained above, the segmentation prediction sub-image set B corresponding to the sequence number is extracted.

[0046] S15 calculates the segmentation overlap score based on the first image set and the second image set.

[0047] S151 calculates based on the first image set and the second image set to obtain the first sequence value;

[0048] Specifically, the IOU is calculated between the image and the corresponding labeled sub-image, resulting in the IOU value sequence C. The IOU calculation formula is as follows:

[0049]

[0050] S152 sets the first sequence value to obtain the second sequence value;

[0051] Specifically, a threshold h is set for the IOU sequence C, and all values ​​greater than h are set to h, resulting in a new IOU sequence D.

[0052] S153 accumulates the second sequence value and divides it by the sequence value length to obtain the segmentation overlap score.

[0053] Specifically, the IOU value sequence D is accumulated, and then the resulting value is divided by the length l of the IOU value sequence. The formula for obtaining the segmented and accumulated IOU score m is shown below.

[0054]

[0055] S2 calculates the cumulative overkill rate of the segmentation prediction map and the label map;

[0056] S21 divides the segmentation prediction map and the label map into multiple third sub-images and multiple fourth sub-images;

[0057] Specifically, the segmentation prediction image and the label image are each divided into n*n sub-images with equal area.

[0058] S22 labels the multiple third sub-images and the multiple fourth sub-images to obtain multiple third labeled sub-images and multiple fourth labeled sub-images;

[0059] Specifically, the multiple sub-images are labeled with serial numbers in sequence.

[0060] S23 Extract defect images from multiple fourth-labeled sub-images to obtain a third image set;

[0061] Specifically, extract the set C of sub-images that do not contain defects from all the labeled sub-images, obtain their serial numbers, and obtain a set of labeled sub-image serial numbers that contain defects.

[0062] S24 Based on the third image set, extract the corresponding defect images from multiple third labeled sub-images to obtain a fourth image set;

[0063] Specifically, based on the sequence set obtained above, the corresponding sequence number of the segmentation prediction sub-image set D is extracted.

[0064] S25 calculates the segmentation cumulative overkill rate by performing calculations on the third image set and the fourth image set.

[0065] S251 calculates based on the third image set and the fourth image set to obtain the first overkill rate sequence;

[0066] Specifically, the pass rate is calculated by comparing it with the corresponding numbered tag sub-images, resulting in the pass rate sequence C. The pass rate calculation formula is as follows:

[0067]

[0068] S252 sums the first overkill rate sequence value and divides it by the sequence value length to obtain the segmented accumulated overkill rate.

[0069] Specifically, the overkill rate sequence C is accumulated, and then the resulting value is divided by the length l of the IOU value sequence. The formula for obtaining the segmented and accumulated overkill rate score n is shown below.

[0070]

[0071] S3 obtains the image segmentation index based on the segmentation cumulative overlap score and the segmentation cumulative overkill rate.

[0072] Specifically, the image segmentation index is obtained based on the segmentation accumulated IOU score and the segmentation accumulated overkill rate.

[0073] This invention effectively evaluates the segmentation of small defects without affecting the structure of the image segmentation method, and the method can also calculate the over-detection rate.

[0074] Segmented cumulative IOU score:

[0075] It can make the importance of minor flaws and major flaws equal;

[0076] Calculation of cumulative kill rate by segmentation:

[0077] The overkill rate of the model can be evaluated.

[0078] (when h is 0.5) Predicted value The difference between the correct prediction and the actual prediction IOU 0.81 0.19 Segmented cumulative IOU 0.59 0.41

[0079] Generally, small defects are as important as large defects in defect detection. When using IOU as an evaluation metric, the table above shows that even when small defects are missed, the IOU still has a high score of 0.81. However, the improved segmented cumulative IOU has a predicted value of 0.59, which better balances the importance of small and large defects.

[0080] The above-disclosed embodiments are merely preferred embodiments of the defect detection and evaluation method based on image segmentation of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art can understand that implementing all or part of the above embodiments and making equivalent changes in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A defect detection and evaluation method based on image segmentation, characterized in that, Includes the following steps: Calculate the cumulative overlap score of the segmentation prediction map and the label map; Calculate the cumulative overkill rate of the segmentation prediction map and the label map; An image segmentation index is obtained based on the segmentation cumulative overlap score and the segmentation cumulative overkill rate; The calculation of the segmentation prediction map and the label map's cumulative overlap score includes: The segmentation prediction map and the label map are divided into multiple first sub-images and multiple second sub-images; Multiple first sub-images and multiple second sub-images are labeled to obtain multiple first labeled sub-images and multiple second labeled sub-images; Extract defect images from multiple sub-images labeled with the second number to obtain the first image set; Based on the first image set, extract the corresponding defect images from multiple sub-images labeled with the first number to obtain the second image set; Based on the first image set and the second image set, a segmentation overlap score is obtained. The calculation based on the first image set and the second image set to obtain the segmentation overlap score includes: The first sequence value is obtained by calculating based on the first image set and the second image set; The first sequence value is set to obtain the second sequence value; The summation of the second sequence values ​​is divided by the sequence value length to obtain the segmentation overlap score; The calculation of the cumulative overkill rate of the segmentation prediction map and the label map includes: The segmentation prediction map and the label map are divided into multiple third sub-images and multiple fourth sub-images; The multiple third sub-images and the multiple fourth sub-images are labeled to obtain multiple third labeled sub-images and multiple fourth labeled sub-images; Defect images are extracted from multiple fourth-labeled sub-images to obtain a third image set; Based on the third image set, extract the corresponding defect images from multiple third labeled sub-images to obtain the fourth image set; The segmentation cumulative overkill rate is calculated by performing calculations on the third image set and the fourth image set. The calculation of the segmentation cumulative overkill rate for the third image set and the fourth image set includes: The first overkill rate sequence is obtained by calculating based on the third and fourth image sets. The cumulative overkill rate sequence value is obtained by summing the first overkill rate sequence value and dividing it by the sequence value length.

Images