A glass panel surface defect detection method based on small sample learning

Abstract

The application discloses a kind of glass panel surface defect detection methods based on small sample learning, comprising the following steps: a small amount of defective glass panel image is collected and labeled boundary frame and defect category;A small amount of glass panel defect image is preprocessed and expanded;Network for identifying and positioning glass panel defect image is constructed;Defect detection is carried out on glass panel image using trained defect detection model, and defect frame and category are output.The method uses data enhancement, transfer learning, L2 regularization to alleviate small sample problem, randomly shakes the labeled frame to increase the diversity of frame, to improve the robustness of model, using global ROI extraction layer introduces background information for candidate region feature, adaptively changes the weight of each sample in loss to improve the performance of model.The method is suitable for surface defect detection task of only a small amount of glass panel image, and the detection precision is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of glass panel detection, and particularly relates to a glass panel surface defect detection method based on small sample learning. BACKGROUND

[0002] Glass panels are in great demand in the computer, communication and consumer electronics industries, and as market demand continues to grow, the quality requirements for glass panels are also becoming higher and higher. Traditional manual inspection methods require a large number of trained workers, often consuming a lot of manpower, and being inefficient, and due to personal subjectivity, there can be a big difference in standards, in addition, due to the special optical properties of glass, long-term detection work can cause certain damage to the eyes of workers. With the development of optical technology and computer technology, many automatic optical detection solutions have been proposed for surface defect detection tasks. This non-contact detection method can fundamentally improve the detection accuracy and efficiency, provide guidance for production, and reduce the burden on manual labor.

[0003] Traditional machine vision methods are often designed for several specific defects and are not universal. Compared with traditional machine vision methods, deep convolutional neural networks have shown strong ability to learn high-level features, improving the accuracy and efficiency of image classification and detection. Currently, in the field of target detection, there are two-stage detectors represented by Faster R-CNN, and single-stage detectors represented by SSD and YOLO, among which Faster R-CNN is widely used in industrial product defect detection due to its high detection accuracy and speed. However, when Faster R-CNN is directly applied to glass panel surface defect detection, there are still several difficulties:

[0004] (1) Most current defect detection methods are based on data-driven methods to train related detection models, and it is very difficult to collect defect sample data in the actual glass panel industry because most glass panels are of qualified quality. Therefore, when Faster R-CNN is applied to glass panel surface defect detection, the problem of insufficient data needs to be considered.

[0005] (2) The boundaries of some defects of the actual glass panel are relatively fuzzy, and defects such as scratches are intermittent, making it difficult to label with a very clear rule, and there may be a problem of inaccurate labeling.

[0006] (3) The defects such as pinholes of the actual glass panel are usually small, and the detection accuracy of most current defect detection methods for small targets is not high, and it is easy to miss detection, etc. SUMMARY

[0007] To solve the above problems, the present application provides a glass panel surface defect detection method based on small sample learning, which can efficiently and accurately detect glass panel surface defects under the condition of a small number of glass panel defect sample images, inaccurate labeling and a large number of small targets, and the specific steps are as follows:

[0008] S1: Collect a small number of glass panel images with defects, and label the boundary box and defect category;

[0009] S2: Preprocess and expand the number of glass panel images to construct a glass panel surface defect detection dataset;

[0010] S3: Construct a defect detection network for recognizing and positioning the glass panel image, wherein the defect detection network comprises a backbone feature extraction network composed of ResNet101 and a feature pyramid network, an improved RPN network, an ROIpooling layer, a global ROI extraction layer, a bounding box regression network and a classification network;

[0011] The backbone feature extraction network is used to extract the global features of the image, the improved RPN network and the ROIpooling layer are used to extract the candidate region features of the image, the global ROI extraction layer is used to fuse the global features and the candidate region features, update the candidate region features, and the bounding box regression network and the classification network are used to generate the positioning boundary box and the defect category according to the updated candidate region features;

[0012] S4: Train the constructed defect detection network using the glass panel surface defect detection dataset to obtain a trained defect detection model;

[0013] S5: Use the trained defect detection model to detect defects in the glass panel image, and output the defect positioning boundary box and the defect category to which it belongs.

[0014] Further, the backbone feature extraction network comprises ResNet101 and a feature pyramid network;

[0015] The ResNet101 comprises one convolutional layer and four residual blocks connected in turn from bottom to top; the output of each residual block is connected in turn with the corresponding layer of the feature pyramid network from top to bottom, the output results of each layer of the feature pyramid network are subjected to L2 regularization processing to form a multi-scale feature map.

[0016] The improved RPN network works in the following manner: the multi-scale feature maps output by the feature pyramid network are taken as the input of the RPN network, p kinds of anchor boxes are generated in each scale of feature map, the length and width dimensions of the anchor boxes are obtained through clustering, the labeled boxes in the training set are clustered into p*q kinds by using the k-means method, wherein q is the number of different scales of the multi-scale feature map; p*q anchor boxes are generated according to the cluster centers, and the anchor boxes are sorted according to the area, and each scale of feature map corresponds to p anchor boxes. An adaptive training sample selection method is used to automatically select positive samples and negative samples for classification and bounding box regression training. The anchor boxes are subjected to bounding box regression and scoring by the RPN network to obtain initial candidate regions, and after NMS filtering, the final candidate regions are obtained.

[0017] The candidate regions generated by the improved RPN network are input into the ROI pooling layer, the candidate region features are extracted from the corresponding scale feature maps output by the feature pyramid network, and the size is unified; the multi-scale feature maps output by the feature pyramid network are taken as global features, the candidate region features and the global features after average pooling are processed based on the attention mechanism, the candidate region features are taken as a mask, and background features are generated; then the background features and the candidate region features are dynamically fused to obtain updated candidate region features.

[0018] The updated candidate region features are sent to the bounding box regression network and the classification network, the total loss of the defect detection network includes the classification loss and the bounding box loss, the classification loss uses the cross entropy function, and the bounding box loss uses the smoothL1 function.

[0019] Further, the positive samples and the negative samples are obtained according to the adaptive training sample selection method, first, the samples with IOU lower than the threshold value are filtered according to the IOU statistics of the anchor boxes and the labeled boxes, then it is determined whether the center of the anchor box falls into the labeled box, if yes, it is a positive sample, if not, it is a negative sample.

[0020] The application uses limited contrast self-adaptive histogram equalization to improve the contrast of the picture, adds noise simulation of the picture without accurate focusing, realizes the expansion of the small sample image by randomly taking the image block according to the intersection-over-union of the annotation box and the image block, uses data enhancement, transfer learning and L2 regularization to relieve the small sample problem; the annotation box is randomly shaken to increase the diversity of the frame, and the robustness of the model under the condition that the annotation is not accurate is enhanced; the feature pyramid is used to fuse the bottom structure information and the high-level semantic information, generate multi-scale features, and improve the detection accuracy of small target defects such as pinholes of the glass panel; the candidate region feature is fused with the whole image feature, the background information is introduced for the candidate region feature, which is helpful for the classification and regression of the frame smaller than the annotation box; the weight of each sample in the loss is changed to improve the performance index of the model as a whole. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is the overall flow chart of the glass panel surface defect detection method based on small sample learning disclosed by the application.

[0022] Figure 2 is a schematic diagram of a glass panel surface defect detection network model.

[0023] Figure 3 is a structural schematic diagram of a global ROI extraction layer.

[0024] Figure 4 is an experimental result diagram of the method provided by the application.

[0025] Figure 5 is a local schematic diagram of the glass panel surface defect detection result, wherein (a) is a bubble defect; (b) is a tin ash defect; (c) is a pinhole defect; (d) is a scratch defect. DETAILED DESCRIPTION

[0026] The application will be further described and explained in conjunction with the drawings and specific embodiments. The technical features of each embodiment in the application can be combined accordingly without conflict.

[0027] The overall flow chart of the glass panel surface defect detection method based on small sample learning disclosed by the application is as shown in Figure 1 The specific implementation process is as follows:

[0028] (1) Collect a small amount of glass panel images with defects. Use the annotation software labelImg to annotate each picture to generate an xml file, the xml file contains the boundary box of the defect and the defect category, and the defect category includes bubble, tin ash, pinhole and scratch.

[0029] (2) Preprocess and quantity expansion of glass panel image, and construct glass panel surface defect detection dataset.

[0030] The specific steps are as follows:

[0031] a. The preprocessing includes limited contrast adaptive histogram equalization, adding noise, etc.

[0032] The limited contrast adaptive histogram equalization method is used, the upper threshold of the contrast limit is 4, and the grid size of the histogram equalization is selected as 8. Since the collected glass panel image is relatively dark, the contrast of the image can be enhanced by using this method.

[0033] The specific process of the limited contrast adaptive histogram equalization method is as follows: first, convert the image from RGB color mode to LAB color mode, then divide the image into equal size 8x8 rectangular blocks, then count the histogram distribution of the luminance channel (L channel) of each rectangular block, and cut off the part of each histogram that exceeds the threshold and average it to each gray level, then equalize each histogram, calculate the equalized luminance value of the center point of each rectangular block, then for each pixel point, use the center points of the adjacent rectangular blocks for bilinear interpolation to obtain the luminance value of the pixel point, and finally convert the difference image from LAB color mode to RGB color mode.

[0034] Randomly add Gaussian or salt and pepper noise to simulate pictures without accurate focusing.

[0035] b. Quantity expansion is realized by random sampling.

[0036] Randomly crop each glass panel image with defects to obtain a plurality of image blocks, and each image block includes one or more boundary boxes with defect categories.

[0037] In this embodiment, when the glass panel image with defects is cropped, a threshold set {0.1, 0.3, 0.5, 0.7, 0.9} of the intersection over union of the image block and the boundary box is set, and the multiple rounds of cropping are performed in order from small to large threshold. In each round of cropping, the intersection over union of the boundary box with the defect category in the obtained image block and the image block is greater than the threshold. For example, first set the threshold to 0.1, randomly sample the collected area of the glass panel image with defects to expand the dataset, and after one round of sampling, set the threshold to 0.3, repeat the above process, and so on.

[0038] After the cropping, random jittering is performed on the labeled boxes to increase the diversity of the labeled boxes, so as to improve the adaptability and robustness of the model. The center point of the labeled box is unchanged, and the scaling ratio of the length and width is uniformly sampled between [0.9, 1.1]. The labeled boxes are transformed at the same time of data augmentation, and finally manually checked to ensure that the images and labeled boxes generated by data augmentation meet the actual situation and meet the construction principles of the data set.

[0039] (3) Construct a glass panel surface defect detection network model for recognizing and positioning the glass panel defect image. As shown in FIG. 8, the defect detection network model includes a backbone feature extraction network composed of ResNet101 and a feature pyramid network, an improved RPN network, an ROI pooling layer, a global ROI extraction layer, a bounding box regression network, and a classification network. Figure 2

[0040] The following describes each sub-network structure included in the glass panel surface defect detection network model.

[0041] (3.1) Backbone feature extraction network

[0042] The backbone feature extraction network is composed of ResNet101 and a feature pyramid network. In this embodiment, ResNet101 adopts the weight parameters pre-trained on a large-scale data set ImageNet for classification tasks, and uses transfer learning to alleviate overfitting of the deep model on small data.

[0043] ResNet101 includes one convolutional layer and four residual blocks. The feature outputs of the four residual blocks of ResNet101 are fused using the feature pyramid network, i.e., the bottom structure information and the high-level semantic information are combined to strengthen the feature expression and improve the detection performance of small targets. The output of the feature pyramid network is composed of five feature maps. The feature maps of each branch of the feature pyramid network are subjected to L2 regularization processing to form multi-scale feature maps.

[0044] (3.2) Improved RPN network

[0045] The multi-scale features output by the feature pyramid network are input into the improved RPN network for extracting candidate regions. Specifically, the multi-scale feature maps output by the feature pyramid network are used as the input of the RPN network. Three kinds of anchor boxes are generated in each scale of the feature map. The length and width dimensions of the anchor boxes are obtained by clustering, and a set of prior values more suitable for the data set is automatically generated using the k-means method. The IOU of the anchor box and the labeled box is improved, which is conducive to improving the detection accuracy of the model. Therefore, in order to improve the IOU of the anchor box and the labeled box, the IOU value is used as the evaluation criterion for clustering, and the distance measurement formula is as follows:

[0046] ​d(box, centroid) = 1 - IOU(box, centroid)

[0047] wherein, IOU(box, centroid) represents the intersection over union of the label box and the clustering cluster center box.

[0048] An adaptive training sample selection method is used to automatically select positive samples and negative samples for classification and bounding box regression training. Anchor boxes are subjected to bounding box regression and scoring through an RPN network to obtain initial candidate regions. The candidate regions filtered through NMS are sent to the ROI pooling layer for feature extraction, and the feature size is uniformly set to 7x7.

[0049] In this embodiment, the label boxes in the training set are clustered into 15 categories; and 15 anchor boxes are generated according to the respective clustering centers. The feature map with a large resolution contains more fine structure features, which is suitable for detecting small targets. The feature map with a small resolution has a larger receptive field, which is suitable for detecting large targets. Therefore, the generated anchor boxes are sorted according to the area, and the anchor boxes with small areas are used as the prior boxes of the feature map with a large resolution, and each feature map is allocated with 3 anchor boxes.

[0050] The common positive and negative sample selection method selects positive and negative samples by calculating the IOU of the anchor box and the label box, and comparing the IOU with the set threshold. The defect of this method is obvious. Whether the anchor box is a positive sample or a negative sample is very limited by the design of the anchor box length and width and the selection of the threshold. The present application proposes an adaptive training sample selection method. First, the IOU statistics of the anchor box and the label box are calculated to filter out a part of anchor boxes with small IOU. Then, the positive and negative samples are determined by determining whether the center of the anchor box falls into the label box. Obviously, this method eliminates the defect.

[0051] (3.3) ROI pooling layer

[0052] The candidate regions generated by the improved RPN network are input into the ROI pooling layer to extract the candidate region features from the corresponding scale feature maps output by the feature pyramid network, and the size is unified to 7x7.

[0053] (3.4) Global ROI extraction layer

[0054] The global ROI extraction layer is used to extract the features of the whole image. The residual network is used to fuse the global features and the candidate region features, which introduces background information for the candidate region features, and helps the classification and regression of the bounding boxes that are smaller than the label box.

[0055] In this embodiment, the global ROI extraction layer adopts a residual structure, and the side branch of the residual structure is beneficial to avoid the degradation of the network. The specific structure is as follows: Figure 3As shown, the global feature is first changed to 7x7 in resolution by adaptive average pooling, and then based on the attention mechanism, the candidate region feature X_pro is used as a mask to further optimize the generation of the background feature X_bg. Considering that simply adding the positions of the two features means that the two features are equally important, however, the background feature can only play an auxiliary role, therefore, a dynamic fusion strategy is used to fuse the candidate region feature X_pro and the background feature X_bg, w_pro and w_bg are both used as learnable parameters, and finally an updated candidate region feature is output through a convolution layer.

[0056] (3.5) bounding box regression network and classification network

[0057] The updated candidate region feature is sent to the bounding box regression network and the classification network to realize the detection of the surface defects of the glass panel.

[0058] The total loss of the detection network includes the classification loss and the bounding box loss, the classification loss uses the cross-entropy function, and the bounding box loss uses the smoothL1 function. From the perspective of the influence of each predicted box on the average precision mean (mAP), the importance of the predicted box is weighted.

[0059] The importance of the predicted box is specifically: considering the influence of each sample on mAP, the classification weight is adjusted. For positive samples, the larger the IOU with the labeled box, the larger the classification weight w i ; for negative samples, the larger the score of predicting as a positive sample (i.e. the probability of being classified as a defect), the larger the classification weight w j ; the bounding box regression weight is adjusted. For positive samples, the larger the score of predicting as a certain type of defect, the larger the bounding box regression weight c i .

[0060] The loss function is as follows:

[0061] L = λL cls + μL loc

[0062] Wherein, λ and μ are parameters for balancing the classification loss and the bounding box loss, and L is the total loss.

[0063] The classification loss is:

[0064]

[0065]

[0066]

[0067] Wherein, L clsis the classification loss, cross_entropy(.) is the cross-entropy function, n is the number of positive samples, m is the number of negative samples, s i is the predicted score of the i-th positive sample, s j is the predicted score of the j-th negative sample, is the true defect class of the i-th positive sample, is the class (i.e., background class) of the j-th negative sample, w i and w j are weights, β and γ are hyperparameters, r i is the ranking order of the i-th positive sample, r j is the ranking order of the j-th negative sample, where the positive samples are ranked according to the IOU of the bounding box and the annotation box, the larger the IOU, the smaller the order number, and the negative samples are ranked according to the probability of being predicted as a defect class, the larger the probability, the smaller the order number, n max is the total number of classes to which the sample belongs;

[0068] The bounding box loss is:

[0069]

[0070]

[0071] where L loc is the bounding box loss, smoothL1(.) is the smoothL1 function, n is the number of positive samples, d i is the predicted bounding box offset of the i-th positive sample, is the deviation of the annotation box and the predicted bounding box, c i is the weight, b and k are hyperparameters, p i is the predicted score of the i-th positive sample corresponding to the true defect class of the predicted bounding box. Therefore, for positive samples, the larger the score of being predicted as a certain defect class, the larger the weight in the loss.

[0072] The model is trained using a glass panel surface defect detection dataset, the initial learning rate is 0.004, the learning rate is changed using periodic learning rate and cosine annealing, the training period is 50 rounds, the momentum is set to 0.9, and the weight decay is set to 0.0001.

[0073] In actual application, the trained defect detection model is used to detect defects in glass panel images, and the defect bounding box and class are output. As Figure 5 shown, the detection results of four kinds of defects are respectively shown. Since the detection picture is large, local pictures are shown for convenience.

[0074] The embodiment adopts mAP, precision and recall to evaluate the defect detection result, and the higher the mAP is, the better the defect detection effect of the method on the glass panel is. The defect detection result of the algorithm in the embodiment is as shown in Table 1. Figure 4 As shown in Table 1, the method provided in the embodiment has obvious advantages in accuracy or recall relative to the conventional Faster R-CNN.

[0075] Table 1: Detection results of the conventional Faster R-CNN and the algorithm in the embodiment on the glass panel defect data set

[0076] Detection network mAP (%) Precision (%) Recall (%) Regular Faster R-CNN 72.8 73.1 70.1 The method of the invention 85.1 82.2 82.8

[0077] In summary, under the condition that only a small amount of glass panel defect pictures can be obtained, the picture contrast is improved by using the limited contrast adaptive histogram equalization, noise is added to simulate pictures without accurate focusing, the image blocks are randomly sampled according to the intersection-over-union of the labeled boxes and the image blocks to realize the expansion of the small sample images, the small sample problem is relieved by using data enhancement, transfer learning and L2 regularization; the labeled boxes are randomly shaken to increase the diversity of the boxes and enhance the robustness of the model under the condition that the labeling is not accurate enough; the bottom structure information and the high-level semantic information are fused by using the feature pyramid to generate multi-scale features and improve the detection accuracy of small target defects such as pinholes of the glass panel; the whole picture features are fused with the candidate region features to introduce background information into the candidate region features, which is helpful for the classification and regression of the boxes that are smaller than the true boxes; considering the influence of each sample on mAP, the weights of various samples in the loss are changed to improve the performance indicators of the model as a whole.

Claims

1. A method for detecting surface defects of a glass panel based on small sample learning, characterized in that, The method comprises the following steps: S1: Collecting a small amount of images of defective glass panels and labeling the boundary boxes and defect categories; S2: Preprocessing and quantity expansion of the glass panel images to construct a glass panel surface defect detection dataset, specifically: a. Preprocessing: including limited contrast adaptive histogram equalization and adding noise; The limited contrast adaptive histogram equalization is: using the limited contrast adaptive histogram equalization method, converting the glass panel image from the RGB color mode to the LAB color mode, dividing the image into 8x8 rectangular blocks of equal size, and counting the histogram distribution of the luminance channel of each rectangular block. The gray level part exceeding the threshold in each histogram is cut off and evenly distributed to the gray levels within the threshold, and then each histogram is equalized. The equalized luminance value of each rectangular block center point is calculated, and the bilinear interpolation of the adjacent rectangular block center point is used for each pixel point to obtain the luminance value of the pixel point. The interpolated image is converted from the LAB color mode to the RGB color mode; The noise is random Gaussian or salt and pepper noise; b. Quantity expansion: setting a threshold set {0.1, 0.3, 0.5, 0.7, 0.9} of the intersection over union of the image block and the boundary box, and cutting a plurality of image blocks from each glass panel image with defects according to the threshold from small to large. Each image block includes one or more boundary boxes with defect categories. In each round of cutting, the intersection over union of the boundary box with the defect category in the obtained image block and the image block is greater than the threshold. After cutting, the labeled box is randomly jittered; S3: Constructing a defect detection network for recognizing and positioning the glass panel image, wherein the defect detection network comprises a backbone feature extraction network composed of ResNet101 and a feature pyramid network, an improved RPN network, an ROI pooling layer, a global ROI extraction layer, a bounding box regression network, and a classification network; The backbone feature extraction network is used to extract the global features of the image, the improved RPN network and the ROI pooling layer are used to extract the candidate region features of the image, the global ROI extraction layer is used to fuse the global features and the candidate region features, update the candidate region features, and the bounding box regression network and the classification network are used to generate the positioning boundary box and the defect category according to the updated candidate region features; The ResNet101 comprises one convolutional layer and four residual blocks connected from bottom to top; the output of each residual block is connected to the corresponding layer of the feature pyramid network from top to bottom in turn, the output results of each layer of the feature pyramid network are subjected to L2 regularization processing to obtain multi-scale feature maps. The improved RPN network works in the following manner: multi-scale feature maps output by the feature pyramid network are taken as inputs of the RPN network, p kinds of anchor boxes are generated in each scale of feature map, the length and width dimensions of the anchor boxes are obtained through k-means clustering, and an IOU value is taken as a clustering evaluation criterion; an adaptive training sample selection method is used to automatically select positive samples and negative samples for classification and bounding box regression training; the anchor boxes are subjected to bounding box regression and scoring through the RPN network to obtain initial candidate regions, and after NMS filtering, final candidate regions are obtained; The candidate regions generated by the improved RPN network are input into the ROI pooling layer, candidate region features are extracted from the corresponding scale feature maps output by the feature pyramid network, and the sizes are unified; the multi-scale feature maps output by the feature pyramid network are taken as global features, the candidate region features and the global features after average pooling are subjected to attention mechanism-based processing, and the candidate region features are taken as masks to generate background features; The background features and the candidate region features are dynamically fused, w_pro and w_bg are taken as learnable parameters, and updated candidate region features are obtained; S4: using the glass panel surface defect detection dataset to train the constructed defect detection network to obtain a trained defect detection model; S5: using the trained defect detection model to perform defect detection on the glass panel image to output a defect positioning bounding box and a defect category to which the defect belongs. 2.The small sample learning based glass panel surface defect detection method of claim 1, wherein, The size of the bounding box labeled in the cropped image block is randomly scaled, and the center point position of the bounding box remains unchanged during scaling, and the length-width scaling ratio of the bounding box is between 0.9 and 1.

1. 3.The small sample learning based glass panel surface defect detection method of claim 1, wherein, The labeled boxes in the training set are clustered into p*q classes by using the k-means method, where q is the number of different scales of the multi-scale feature maps; p*q anchor boxes are generated according to the cluster centers, and the anchor boxes are sorted according to their areas, and each scale of feature map corresponds to p anchor boxes. 4.The small sample learning based glass panel surface defect detection method of claim 1, wherein, The positive samples and negative samples are obtained according to the adaptive training sample selection method, first, the samples with an IOU lower than a threshold value are filtered according to the IOU statistics of the anchor boxes and the labeled boxes, then it is determined whether the center of the anchor box falls within the labeled box, if yes, it is a positive sample, and if no, it is a negative sample. 5.The small sample learning based glass panel surface defect detection method of claim 1, wherein, The updated candidate region features are input into the bounding box regression network and the classification network, the total loss of the defect detection network includes a classification loss and a bounding box loss, the classification loss uses a cross-entropy function, and the bounding box loss uses a smoothL1 function, and the specific formulas are as follows: L = λL cls + μL loc Where λ and μ are parameters for balancing the classification loss and the bounding box loss, and L is the total loss; The classification loss is: where L cls is the classification loss, cross_entropy(.) is the cross-entropy function, n is the number of positive samples, m is the number of negative samples, s i is the predicted score of the i-th positive sample, s j is the predicted score of the j-th negative sample, is the true defect class of the i-th positive sample, is the class of the j-th negative sample, w i and w j are weights, β and γ are hyperparameters, r i is the ranking order of the i-th positive sample, r j is the ranking order of the j-th negative sample, n max is the total number of classes to which the samples belong. The bounding box loss is: where L loc is the bounding box loss, smoothL1(.) is the smoothL1 function, n is the number of positive samples, d i is the predicted bounding box offset of the i-th positive sample, is the deviation between the labeled box and the predicted bounding box, c i is the weight, b and k are hyperparameters, p i is the predicted score of the real defect class corresponding to the predicted bounding box of the i-th positive sample.

Images