Composite defect detection method based on semantic segmentation and target detection fusion model

Abstract

The invention provides a composite defect detection method based on a semantic segmentation and target detection fusion model, and the method comprises the steps of carrying out the composite defect image preprocessing of an original image, obtaining a plurality of defect images, obtaining the edge shapes of defects in the images, and distinguishing the defect types of the images; performing defect image preprocessing on the original image according to the defect type of the image, and inputting a processing result to construct and train a Unet semantic segmentation network to obtain a defect mask pattern of the corresponding image; calculating the similarity between the defect mask pattern of the corresponding image and the mask label of the corresponding defect of the original image, obtaining a superposed image according to the similarity, inputting the obtained superposed image into the Faster R-CNN network for defect target detection, obtaining a Faster R-CNN network detection result image, and completing the detection of the composite defect. According to the composite defect detection method provided by the invention, various defects of the original image are respectively detected, so that the defect detection problem of the mobile phone screen image with multiple types of overlapped defects can be effectively solved.

Description

technical field [0001] The invention relates to the technical field of screen surface defect detection, in particular to a composite defect detection method based on a fusion model of semantic segmentation and target detection. Background technique [0002] During the use of mobile phones, it is inevitable that there will be defects on the screen surface. The surface defects of existing mobile phone screens can be mainly divided into single type of defects, multiple types of defects and mixture of multiple types of defects according to the existence of defects. Among the three defect situations, the detection process of the mixture of multiple defect types is relatively difficult. A single type of defect means that there is only one type of defect on a mobile phone screen image, which is relatively difficult for the detection model to detect; multiple types of defects refer to a mobile phone screen image There are many different types of defects, but there is no mixed and ...

AI Technical Summary

Problems solved by technology

Among the three defect situations, the detection process of the mixture of multiple defect types is relatively difficult

A single type of defect means that there is only one type of defect on a mobile phone screen image, which is relatively difficult for the detection model to detect; multiple types of defects refer to a mobile phone screen image There are many different types of defects, but there is no mixed and overlapping situation in these defect images. In this case, the detection model will be more difficult to detect than a single defect detection; multiple types of defects overlap, usually manifested as a mobile phone screen image. There are multiple defects overlapping and interlacing in a certain area at the same time, and the existence of defects is difficult to judge. In this case, the difficulty of detection model detection is the highest among the three types of defects, and it is difficult to successfully and accurately detect all defects in the image.

In addition, in addition to the above-mentioned conditions on the surface of the mobile phone screen, there may also be the following conditions that cause the detection accuracy of the detection model to be low

If it exists: 1. The pixel chromaticity value of some defects is relatively low, and the features are extremely inconspicuous

2. The size of various defects existing on the mobile phone screen is very different, and the feature sizes extracted from various defects are not uniform, which is difficult to use for detection targets

3. The division method of positive and negative samples is not particularly reasonable, especially in some special cases

The existence of these mobile phone screens may lead to more difficult detection, lower detection accuracy, and poor detection effect

[0003] According to some existing detection models, such as the classic target detection network Faster R-CNN, after multiple training iterations, it can achieve good detection results for single-type defects and multiple types of defects. In the case of multiple types of defects overlapping, until now there is no very suitable detection model

For the situation where multiple types of defects overlap, it is obviously insufficient for the detection ability of general detection models. The feature extraction network will be used to extract features based on the most obvious defects among all existing defects in an area, and then the extracted features will be used to perform subsequent detection tasks, and then classification prediction and position prediction will be performed on them. However, in this area However, there are multiple types of defects, so it leads to the problem of missed detection or the defect cannot be detected due to the mixture of defect characteristics.

Secondly, the pixel chromaticity value of the image defect on the surface of the mobile phone screen is low, which leads to the fact that the feature extraction network in the target detection network cannot extract the feature map in the image, which makes it difficult for the subsequent network of the target detection network to detect defects.

Finally, the size of various types of defects varies greatly. If the characteristics of defects are used to perform subsequent target detection, it may be due to the size difference of various types of defects. Large defects need to be judged by deep features, and small defects need to be judged. Using shallow features to judge, resulting in poor detection results

Images