Systems, methods, and non-transitory machine-readable storage media

Abstract

The present application provides a system, a method and a non-transitory machine-readable storage medium. Through a unique evaluation technique, the performance of a defect detection model is evaluated on a per-label basis rather than on a per-pixel or per-image basis. For the label mask of an image (ground truth of the image), one or more contours are extracted from the mask using computer vision, and the same is performed on the predicted mask. Using an exhaustive pairing approach, each ground truth contour is compared to each predicted contour using a new metric called Intersection over Prediction (IoP). If the IoP is above a threshold, it is recorded as a true positive. If not, another new metric called Ground Truth over Intersection (IoGT) is calculated, and if it is above a threshold, it is recorded as a true positive. If the ratio of unmatched ground truth area to total ground truth area exceeds another threshold, the ground truth contour is a true positive.

Images