Glass defect detection method adopting semi-supervised learning

Abstract

The invention discloses a glass defect detection method adopting semi-supervised learning, and relates to the technical field of image detection. According to the method, a semi-supervised learning method is adopted to construct a glass cup detection model, and the model is mainly used for realizing defect detection work before the glass cup leaves a factory. In the model, firstly, correlation between feature maps is learned through a feature extraction layer, so that more beneficial glass defect feature information is filtered out; secondly, adding a feature map attention layer to weight an output feature map so as to improve the feature extraction capability of a feature extraction layer; and finally, a feature enhancement feature pyramid layer is proposed to solve the defect that an original FPN cannot determine to detect effective feature information, feature maps of different scales are fused to facilitate detection of small target defects, and the overall detection performance of the model is improved.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, specifically to a method for detecting defects in glass cups using semi-supervised learning. Background Technology

[0002] Glass cups are a common drinking utensil in daily life, used in homes, schools, restaurants, and other places. Currently, glass cups have become the mainstream choice, with people increasingly opting for them over plastic or metal cups in their daily lives.

[0003] With the continuous growth of demand in my country's glass cup market, the quality requirements for glass cups are also increasing. The main focus of glass cup production is the detection of surface defects to determine whether the product meets standards. With the rapid development of machine vision and deep learning technologies, many target inspection methods have been widely applied in the field of defect detection for industrial products.

[0004] Traditional machine vision mainly involves acquiring images and using specific algorithms to analyze and process them. However, there are some problems in the image processing process, such as the inability to select suitable image features, susceptibility to complex interference, and inability to perform accurate defect detection.

[0005] Deep learning methods have demonstrated powerful capabilities in feature learning, extracting deeper-level features and improving the accuracy and efficiency of image classification, detection, and recognition. Currently, in the field of object detection, there are multi-stage detectors and single-stage detectors, with Faster R-CNN and YOLO being representative models. The YOLO model performs well in detection and is more suitable for defect detection. In actual production processes, tiny defects often appear on the surface of glass cups. Directly using the YOLO model for defect detection will not yield accurate results.

[0006] To address the issue of insufficient accuracy in defect detection using existing conventional inspection models during glass production, this invention proposes a semi-supervised learning method for glass defect detection. Summary of the Invention

[0007] The purpose of this invention is to provide a glass defect detection method using semi-supervised learning to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for detecting defects in glass cups using semi-supervised learning includes the following steps:

[0010] Obtain image samples of glass defects;

[0011] Collect images of defective glass cups. Further, label them according to the defect types of the glass cups and frame the positions of the defects in the images. Among them, the defect types of the glass cups are divided into cracks, cold spots, wrinkles, bubbles and dirt.

[0012] Preprocess the images to construct a training set for glass cup defects;

[0013] Further, in order to improve the performance of defect detection, it is necessary to preprocess the captured images of glass cups.

[0014] Among them, the preprocessing includes uniformly scaling the glass cup images to 512x512 for subsequent training of the model, and performing data augmentation on the scaled glass cup images. The main technical means include: limiting contrast adaptive histogram equalization to increase the contrast of the images and using Gaussian filtering to remove noises such as light and dust generated during the shooting process to obtain defect images of glass cups;

[0015] Among them, the limited contrast adaptive histogram equalization limits noise amplification and local contrast enhancement by restricting the height of the local histogram. This method divides the image into multiple sub-regions; then classifies the histograms of each sub-region; then performs histogram equalization on each sub-region respectively, and finally obtains the transformed gray values by performing interpolation operations on each pixel, so as to achieve contrast-limited adaptive histogram equalization image enhancement.

[0016] The training set for glass cup defects includes N labeled glass cup defect samples and M unlabeled glass cup defect samples; among them, N < M, and both N and M are natural numbers greater than 1; the unlabeled glass cup defect samples are only labeled with defect bounding boxes.

[0017] Further, use the training set for glass cup defects to train the glass cup detection model.

[0018] Among them, the glass cup detection model includes a feature extraction network layer, a feature map attention layer, a feature enhancement feature pyramid layer network layer, a region proposal network layer, a region of interest layer and a prediction layer;

[0019] The feature extraction layer is improved from the VoVNet network and is used for feature extraction of images;

[0020] Among them, the feature extraction layer includes three convolutional modules, four single aggregation modules with different numbers and a weighted pooling effective compression excitation module;

[0021] Further input the defect images of glass cups into the feature extraction layer to obtain four first-level feature images of glass cups, namely the first feature map, the second feature map, the third feature map and the fourth feature map. The specific steps are as follows:

[0022] Step 1: Input an image of the glass defect;

[0023] Step 2: After performing a 64-dimensional convolution with a kernel size of 3x3 and a stride of 2, the defect image of the glass is output as a 256x256 image.

[0024] Step 3: The 256x256 image is convolved with a 64-dimensional convolution kernel of 3x3 and a stride of 1 to output a 254x254 image;

[0025] Step 4: The 254x254 image is convolved with a 128-dimensional convolution kernel of 3x3 and a stride of 1 to output a 252x252 image;

[0026] Step 5: The 52x252 image is processed by five 128-dimensional convolutional kernels of the single aggregation module to form a 3x3 image, and then processed by the weighted pooling effective compression activation module to output the first feature map;

[0027] Step 6: The first feature map is processed by five 160-dimensional convolutional kernels of the single aggregation module 2 to form a 3x3 matrix, and then by the weighted pooling effective compression activation module to output the second feature map;

[0028] Step 7: The second feature map is processed by the five 192-dimensional convolutional kernels of the single aggregation module three to form a 3x3 matrix, and then by the weighted pooling effective compression activation module to output the third feature map;

[0029] Step 8: The output of the third feature map is processed by five 224-dimensional convolutional kernels of the single aggregation module four to form a 3x3 matrix, and then by the weighted pooling effective compression activation module to output the fourth feature map;

[0030] The four primary feature images can be represented as follows:

[0031]

[0032] Among them, A WP-esE This is a function for the weighted pooling effective compression excitation module;

[0033] The weighted pooling effective compression activation module can fully learn the correlation between the feature maps of the glass, thereby filtering out more beneficial feature maps. Specifically, the weighted pooling effective compression activation module includes: average pooling, max pooling, a 1x1 convolutional kernel, a weighting mechanism, and a sigmoid activation function. The following describes how the weighted pooling effective compression activation module processes the feature map output from the single aggregation module:

[0034]

[0035]

[0036] Among them, F gapavg Perform global average pooling on the four primary feature images, F gapmax Perform global max pooling on the four first-level feature images, a i For U i The activation value of the i-th pixel, w i For U i The i-th weight;

[0037] F gapavg and F gapmax They are defined in the following forms respectively:

[0038]

[0039] F gapmax (U i ) = max(U n,m |n∈W,m∈H)

[0040] Wherein, W and H represent the width and height of the feature map output by the aggregation module, respectively.

[0041] Furthermore, the first feature map, the second feature map, the third feature map, and the fourth feature map are respectively input into the feature map attention layer;

[0042] The feature map attention layer weights the input feature map, thereby improving the feature extraction capability of the feature extraction layer.

[0043] The four primary feature images are input into the feature attention layer for weighting to obtain four glass feature attention images, namely the first attention feature map, the second attention feature map, the third attention feature map, and the fourth attention feature map. The specific steps include:

[0044] Step 1: The feature map attention layer weights the feature map in each channel with the input first feature map, second feature map, third feature map, and fourth feature map to obtain four sets of features respectively;

[0045] Step 2: Apply a 1x1 convolution kernel and downsample the four sets of features to obtain four sets of output feature maps;

[0046] Step 3: Downsample and connect the first feature map, the second feature map, the third feature map, and the fourth feature map with the four types of output feature maps respectively to obtain four sets of glass feature attention maps, namely: the first feature attention map, the second feature attention map, the third feature attention map, and the fourth feature attention map.

[0047] Furthermore, the first feature attention map, the second feature attention map, the third feature attention map, and the fourth feature attention map are input into the feature enhancement pyramid layer for rapid feature fusion.

[0048] The feature enhancement pyramid layer provides receptive fields for feature maps of different scales by adding a channel enhancement module, making it easier to detect defects with smaller targets.

[0049] The four types of attention maps of the glass features are input into the feature enhancement pyramid layer to obtain four secondary feature images of the glass, namely the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map. The specific steps include:

[0050] Step 1: The first feature attention map, the second feature attention map, the third feature attention map, and the fourth feature attention map are horizontally connected to the channel enhancement module to obtain the first channel enhancement feature map, the second channel enhancement feature map, the third channel enhancement feature map, and the fourth channel enhancement feature map of the glass.

[0051] Step 2: The first channel enhancement feature map, the second channel enhancement feature map, the third channel enhancement feature map, and the fourth channel enhancement feature map of the glass cup are subjected to max pooling operation through the Spatial Pyramid Pooling (SPP) module to obtain four pooling feature maps of different sizes of glass cups, namely the first pooling feature map, the second pooling feature map, the third pooling feature map, and the fourth pooling feature map.

[0052] Step 3: Select the smallest feature map from the pooled feature maps of four different sizes of glass cups and perform multiple upsampling and downsampling operations to obtain the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map.

[0053] The main process of the channel enhancement module is to input four sets of glass feature attention maps, expand the receptive field through a 1x1 convolution kernel and a set of 3x3 convolution blocks, and then adaptively adjust the features through a channel attention mechanism.

[0054] Furthermore, the fifth, sixth, seventh, and eighth feature maps are output as inputs to the candidate region network (RPN), and p anchor boxes are generated in each feature map. These anchor boxes undergo bounding box regression and scoring in the candidate region layer to obtain candidate features.

[0055] The candidate features are input into the region of interest layer, where bilinear interpolation is used to calculate the location information, followed by max pooling normalization to obtain the predicted feature map. Finally, the predicted feature map information is integrated through a fully connected layer, and the classification layer performs classification to determine the defect type of the glass. Bbox regression is then used to adjust the predicted location.

[0056] Furthermore, the step of training the glass detection model using the glass defect training set includes: first, using the labeled training samples for prediction to obtain an initial detection model; then, using the initial detection model to predict the unlabeled training samples and assign pseudo-labels; finally, using the entire training set for model training to obtain the final glass detection model. During training, the model's loss function is defined as:

[0057] Loss = L cls +L CIoU

[0058] Among them, L cls For classifying defects in glass cups, the cross-entropy loss function is used, L. CIoU Loss function for predicting bounding boxes for defects in glass cups;

[0059]

[0060]

[0061] Where α is a positive hyperparameter, v is a penalty term used to measure the high similarity between the predicted glass defect bounding box and the real glass defect bounding box, and w gt h gt w pre and h pre These represent the width and height of the actual and predicted values ​​of the glass defect bounding box, respectively, where ρ is the Euclidean distance. pre_ctr and Box gt_ctr Let be the center coordinates of the predicted and actual values ​​of the glass defect bounding box, and c be the diagonal length of the minimum bounding box between the predicted and actual values ​​of the glass defect bounding box.

[0062] Furthermore, the final glass detection model is used to detect defects in the input glass image;

[0063] The final output includes the defect category and defect bounding box of the glass image.

[0064] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0065] 1. A feature extraction layer is established as the backbone network for extracting glass features. This layer is based on the VovNet model and incorporates a weighted pooling effective compression activation module to learn the correlation between feature maps, thereby filtering out more useful feature information. Global average and global max pooling are used in the weighted pooling effective compression activation module to ensure that the most important feature information is not lost. Furthermore, weight parameters are added to the pooling layer to activate the feature maps, resulting in better glass defect classification accuracy.

[0066] 2. The feature map attention layer weights the feature map output by the feature extraction layer to improve the feature extraction capability of the lightweight feature extraction layer. Since the number of channels in the weighted feature map does not increase, the subsequent calculation remains unchanged, reducing the number of network parameters and improving the detection speed and accuracy of the glass defect model.

[0067] 3. Feature Enhancement Pyramid Layer: To overcome the shortcomings of the original feature pyramid network in determining the effective feature information for detection after multi-scale feature extraction, an improvement module is added. This module selects convolutional kernels of different scales at the lateral connections, thereby providing receptive fields of different scales. By merging feature maps of different scales, it effectively solves the problem of inaccurate detection of small glass defect targets. Attached Figure Description

[0068] Figure 1 A flowchart of a glass defect detection method using semi-supervised learning provided in an embodiment of the present invention;

[0069] Figure 2 A defect diagram of a glass provided in an embodiment of the present invention;

[0070] Figure 3 The T-VovNet flowchart provided in the embodiments of the present invention;

[0071] Figure 4 A model diagram for generating glass features provided in an embodiment of the present invention;

[0072] Figure 5 A comparison chart of T-VovNet classification results provided in this embodiment of the invention;

[0073] Figure 6 This is a schematic diagram of the feature map attention layer provided in an embodiment of the present invention;

[0074] Figure 7 This is a comparison chart of experimental results for the model with added feature map attention layer provided in an embodiment of the present invention;

[0075] Figure 8 A model diagram of the channel enhancement module provided in an embodiment of the present invention. Detailed Implementation

[0076] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0077] Please see Figures 1 to 8 A glass defect detection method using semi-supervised learning is described below:

[0078] A method for detecting defects in glass cups using semi-supervised learning, see [reference]. Figure 1 The process includes: S10. Acquiring images of defective glass cups; S20. Constructing a glass cup image set; S30. Preprocessing the images to construct a glass cup defect training set; S40. Constructing a glass cup detection model; S40. Using the glass cup defect training set to train the model, calculating the loss until convergence, and obtaining the final glass cup detection model; S50. Using the final glass cup detection model to detect defects in the glass cup images; S60. Outputting the defect type and defect bounding box.

[0079] S10. Collect images of defective glasses and construct a set of glass images.

[0080] Further, see Figure 2 Images of glass cups with different types of defects were collected;

[0081] The different types of defects include cracks, cold spots, wrinkles, bubbles, and dirt. This example collected 8,000 images of glass defects.

[0082] Further, S20. The image of the glass is preprocessed;

[0083] The preprocessing includes uniformly fixing the size of the glass image to 512x512 to obtain a scaled glass image, and using contrast-limited adaptive histogram equalization to increase the image contrast, and Gaussian filtering to remove noise such as light and dust from the image.

[0084] Enhancing the scaled glass image using contrast-limited adaptive histogram equalization includes: dividing the image into multiple sub-regions; classifying the histogram of each sub-region; performing histogram equalization on each sub-region separately; and finally obtaining the transformed grayscale value by interpolating each pixel, thereby achieving contrast-limited adaptive histogram equalization image enhancement to obtain an enhanced glass image.

[0085] Furthermore, the process of removing noise using Gaussian filtering includes: a weighted averaging process of the enhanced glass image, wherein the value of each pixel is obtained by weighted averaging of its own value and the values ​​of other pixels in its neighborhood to obtain the defect image of the glass.

[0086] Furthermore, the defect types and locations of the glass defect images are labeled to construct a glass defect training set. This training set includes labeled glass defect samples and unlabeled glass defect samples. The unlabeled glass defect samples are labeled only with their defect locations, not their defect types.

[0087] Furthermore, the glass defect training set is divided into a labeled glass defect training set and an unlabeled glass defect training set in a 2:8 ratio.

[0088] S30. Construct a glass detection model by improving the YOLOv5 model as the base model.

[0089] Furthermore, for ease of description in the following embodiments, the modules of the glass detection model are named using English abbreviations. For example, the feature extraction layer is T-VovNet, short for Tumbler VovNet, which is the glass VovNet network model; WP-esE, short for Weighted Pooling effective compression Excitation; FMA, short for Feature Map Attention; FE-FPN, short for Feature Enhancement Feature Pyramid; and CEM, short for Channel Enhancement Module.

[0090] Further, see Figure 3In the glass detection model, T-VovNet is used as the feature extraction network. VovNet mainly serves as the backbone network for object detection, saving on high memory access costs and energy consumption. The object detection model based on VovNet outperforms models based on DenseNet and ResNet. Furthermore, T-VovNet includes three 3x3 convolutional kernels, four different numbers of single-pass aggregation modules, and a WP-esE module; see [link to documentation]. Figure 4 T-VovNet extracts the defect image features of the glass, and the specific steps include:

[0091] Input the image of the glass defect;

[0092] The basic feature map of the glass image is extracted through the convolutional layers of the T-VovNet;

[0093] Based on the aforementioned basic feature map, four different numbers of the single aggregation modules are used, and the WP-esE module is connected to extract glass features at different scales.

[0094] The output consists of four primary feature images of the glass: the first feature image, the second feature image, the third feature image, and the fourth feature image.

[0095] Furthermore, the glass features include the glass's size, texture, and color characteristics.

[0096] It adds the WP-esE module to the VovNet model to construct the T-VovNet model. The WP-esE module processes data as follows:

[0097]

[0098] Among them, A WP-esE To effectively compress the incentive module function using weighted pooling, U i These represent the four types of primary feature images.

[0099] The WP-esE module can fully learn the correlation between feature maps of defective glasses, thereby filtering out more beneficial feature maps. The WP-esE module specifically includes: average pooling, max pooling, a 1x1 convolutional kernel, a weighting mechanism, and a sigmoid activation function. The following describes how the WP-esE processes the output feature maps of four different numbers of single-pass aggregation layers:

[0100]

[0101]

[0102] Among them, F gapavgPerform global average pooling on the four types of first-level feature images, F gapmax Perform global max pooling on the four types of first-level feature images, a i For U i The activation value of the i-th pixel, w i For U i The i-th weight;

[0103] F gapavg and F gapmax They are defined in the following forms respectively:

[0104]

[0105] F gapmax (U i ) = max(U n,m |n∈W,m∈H)

[0106] Wherein, W and H represent the width and height of the feature map output by the aggregation module, respectively.

[0107] Furthermore, in this step of the embodiment, by comparing with the VovNet model, this method can filter out more useful feature information, thereby improving the accuracy of glass defect detection. To illustrate the effectiveness of the T-VovNet model, it is compared with several basic models, such as the VovNet model, DesNet model, and ResNet101 model, using the same classifier, to demonstrate T-VovNet's excellent performance in feature extraction. Figure 5 The figure shows a comparison of the accuracy (Acc) of the T-VovNet+Softmax model, VovNet+Softmax model, DensNet+Softmax model, and ResNet101+Softmax model on 200–1600 labeled glass defect images in the training dataset. Softmax represents the classifier. The figure clearly shows that the T-VovNet-based model has a significant advantage. As the number of training images increases, the accuracy of the T-VovNet+Softmax model increases faster than other models, demonstrating the importance of T-VovNet as the backbone network in model construction.

[0108] Furthermore, the four first-level feature images output by T-VovNet are input into FMA for weighting to improve the feature extraction capability of T-VovNet.

[0109] Further, see Figure 6After the input feature map is weighted by the FMA, the number of channels in the feature map is downsampled by a 1x1 convolution. Finally, the feature maps of each channel after several downsamplings are concatenated to obtain four feature map attention images: the first feature attention map, the second attention map, the third feature attention map, and the fourth feature attention map.

[0110] In this embodiment, to illustrate the benefits of adding a feature map attention layer (FMA) to the glass detection model, experiments were conducted comparing it with a model without FMA and the original YOLOv5 model. Table 1 shows the comparison of the mean average accuracy (mAP) evaluation index between the glass detection model and the glass detection model without FMA. Table 1 shows that adding the feature map attention layer (FMA) improves the model's detection accuracy. Furthermore, to demonstrate that the feature map attention layer (FMA) can improve feature extraction in the backbone network, FMA was applied to other models. Figure 7 Experimental data shows that adding the Feature Map Attention (FMA) layer improves the performance of each model, but it still falls short compared to the glass detection model. This demonstrates the significant advantage of the glass detection model over other models, highlighting the versatility of the FMA layer and its effective assistance in feature learning.

[0111] Table 1. Comparison of mAP evaluation metrics under different models

[0112]

[0113] Furthermore, the output feature map of the feature attention layer is fed into the feature enhancement pyramid layer. FPN (Feature Pyramid Network) is characterized by different resolutions at different scales, allowing targets of different sizes to have appropriate feature representations at their respective scales. By fusing multi-scale information, it can predict targets of different sizes at different scales, thus significantly improving model performance. The Feature Enhancement Pyramid (FE-FPN) layer adds a channel enhancement module (CEM) to the original Feature Pyramid (FPN) layer. (See [reference needed]). Figure 8 This module provides receptive fields for feature maps at different scales, making it easier to detect defects with smaller targets. The Feature Enhancement Pyramid Layer (FE-FPN) process includes:

[0114] Step 1: Horizontally connect the four feature map attention images to the channel enhancement module (CEM) to obtain the first channel enhancement feature map, the second channel enhancement feature map, the third channel enhancement feature map, and the fourth channel enhancement feature map of the glass.

[0115] Step 2: The first channel enhancement feature map, the second channel enhancement feature map, the third channel enhancement feature map, and the fourth channel enhancement feature map of the glass cup are subjected to max pooling operation through the Spatial Pyramid Pooling (SPP) module to obtain the first pooling feature map, the second pooling feature map, the third pooling feature map, and the fourth pooling feature map of the glass cup.

[0116] Step 3: Select the glass cup pooling feature map with the smallest output size from Step 2 and perform multiple upsampling and downsampling operations to obtain four secondary feature images, namely the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map.

[0117] The main process of the channel enhancement module (CEM) is to input four sets of glass feature attention maps, expand the receptive field through a 1x1 convolution kernel and a set of 3x3 convolution blocks, and then adaptively adjust the features through a channel attention mechanism.

[0118] In this embodiment, to illustrate the beneficial effects of the proposed feature enhancement pyramid layer on the glass detection model, the experimental data in Table 2 are compared.

[0119] Table 2. Comparison of this invention with other models

[0120]

[0121] Table 2 shows experimental comparison charts for six models, including master-slave mAP and AP. S AP M AP L The FPS evaluation metric demonstrates the effectiveness of the proposed method. In the table, the T-VovNet+FMA+FE-FPN model shows the best performance across all evaluation metrics compared to other models, especially in terms of average accuracy (AP) for small target defect detection. S The performance score reached 48.61, significantly higher than other models. Furthermore, to demonstrate that the proposed Feature Enhancement Pyramid Network (FE-FPN) plays a role in improving defect detection accuracy, experiments were conducted comparing it with other feature pyramids while maintaining a unified backbone network. The experiments show that the FE-FPN possesses excellent feature learning capabilities. In addition, a comparison was made between the FPN and FE-FPN using the Faster R-CNN backbone network, demonstrating that FE-FPN exhibits good portability.

[0122] Furthermore, the fifth, sixth, seventh, and eighth feature maps output by the FE-FPN are used as inputs to the candidate region layer, generating p anchor boxes in each feature map. These anchor boxes are then subjected to bounding box regression and scoring by the candidate region network (RPN) to obtain candidate region features.

[0123] The candidate region features generated by the candidate region layer are input into the region of interest layer, where bilinear interpolation is used to calculate the location information, followed by max pooling normalization to obtain the predicted feature map. Finally, the predicted feature map information is integrated through a fully connected layer. The prediction layer uses a Softmax classifier to classify and determine the defect type of the glass, and Bbox regression is used to adjust the predicted position.

[0124] Further, the glass detection model is trained. First, the labeled training samples are used for prediction to obtain an initial detection model; then, the initial detection model is used to predict the unlabeled training samples and assign pseudo-labels; S40. The entire training set is used for model training to obtain the final glass detection model. During training, the model's loss function is defined as:

[0125] Loss = L cls +L CIoU

[0126] Among them, L cls For classifying defects in glass cups, the cross-entropy loss function is used, L. CIoU Loss function for predicting bounding boxes for defects in glass cups;

[0127]

[0128]

[0129] Where α is a positive hyperparameter, v is a penalty term used to measure the high similarity between the predicted glass defect bounding box and the real glass defect bounding box, and w gt h gt w pre and h pre These represent the width and height of the actual and predicted values ​​of the glass defect bounding box, respectively, where ρ is the Euclidean distance. pre_ctr and Box gt_ctr Let be the center coordinates of the predicted and actual values ​​of the glass defect bounding box, and c be the diagonal length of the minimum bounding box between the predicted and actual values ​​of the glass defect bounding box.

[0130] S50. Further, the final glass detection model is used to perform defect detection on the input glass image;

[0131] S60. The final output shows the defect category and defect bounding box of the glass image.

[0132] In summary, this example uses 8000 images of glass defects and divides them into labeled and unlabeled glass defect images in a 2:8 ratio. The main improvements in this glass detection model are as follows: (1) Feature extraction layer: This layer improves upon the VovNet network by adding a weighted pooling effective compression activation module to the single aggregation layer to learn the correlation between feature maps, thereby filtering out more useful feature information. Global average and global max pooling are used in the weighted pooling effective compression activation module to ensure that the most important feature information is not lost. Furthermore, weight parameters are added to the pooling layer to activate the feature map, resulting in better classification accuracy. (2) Feature map attention layer: This layer weights the feature map output by the feature extraction layer to improve the feature extraction capability of the lightweight feature extraction layer and accelerate the model's detection speed. (3) Feature enhancement pyramid layer: To overcome the shortcomings of the original feature pyramid network FPN in determining the effective feature information for detection after multi-scale feature extraction, a channel enhancement module is added for improvement. By merging feature maps of different scales, the problem of small glass defect targets and inaccurate detection is effectively solved.

[0133] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0134] One or more embodiments of this specification are intended to cover all such alternative steel surface defect datasets falling within the broad scope of the appended claims; such datasets, modifications, and variations. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of this disclosure.

Claims

1. A method for detecting defects in glass cups using semi-supervised learning, characterized in that, Including: Training a glass detection model using a glass defect training set; Among them, the glass detection model includes a feature extraction layer, a feature attention layer, a feature enhanced feature pyramid layer, a candidate region layer, a region of interest layer, and a prediction layer; Inputting a glass defect image into the feature extraction layer to obtain four first-level feature images of the glass, namely the first feature map, the second feature map, the third feature map, and the fourth feature map; Inputting the four first-level feature images into the feature attention layer for weighting to obtain four glass feature attention images; Inputting the four glass feature attention maps into the feature enhanced feature pyramid layer to obtain four second-level feature images of the glass, namely the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map; Inputting the four second-level feature images into the candidate region layer to obtain candidate features of the four second-level feature images; Using the region of interest layer to extract the region of interest of the candidate features to obtain prediction features; Predicting the prediction features through the prediction layer; Calculating the training loss value until convergence to obtain the final glass detection model; Using the final glass detection model for glass image detection; Outputting the defect type and the defect bounding box.

2. The glass defect detection method using semi-supervised learning according to claim 1, characterized in that, The defect types include: Cracks, cold spots, wrinkles, bubbles, and dirt.

3. The glass defect detection method using semi-supervised learning according to claim 1, characterized in that, Before the picture is input into the glass detection model, there is also a preprocessing step: Resizing the glass image to a unified size of 512x512 pixels to obtain a scaled glass image; Using limited contrast adaptive histogram equalization on the scaled glass image to increase the contrast of the image to obtain an enhanced glass image; Using Gaussian filtering to denoise the enhanced glass image to obtain a glass defect image; Annotating the defect category and the defect bounding box for the glass defect image.

4. The glass defect detection method using semi-supervised learning according to claim 1, characterized in that, The glass defect training set includes: N labeled glass defect samples and M unlabeled glass defect samples; where N < M, and both N and M are natural numbers greater than  1; the unlabeled glass defect samples are only annotated with defect bounding boxes.

5. The glass defect detection method using semi-supervised learning according to claim 1, characterized in that, The feature extraction layer includes three convolutional modules, four single-aggregation modules with different numbers, and a weighted pooling effective compression excitation module; Among them, the feature extraction layer includes three convolutional modules, four single-aggregation modules with different numbers, and a weighted pooling effective compression excitation module; Among them, the four single-aggregation modules with different numbers are single-aggregation module one, single-aggregation module two, single-aggregation module three, and single-aggregation module four; After each convolutional module of the feature extraction layer, two connections are generated, connecting the current convolutional module to the next convolutional module and connecting to the single-aggregation module. The specific steps include: Step 1: Input a glass defect image; Step 2: After convolution of the glass defect image with a 64-dimensional convolution kernel of 3x3 and a stride of 2, an image of 256x256 is output; Step 3: After convolution of the 256x256 image with a 64-dimensional convolution kernel of 3x3 and a stride of 1, an image of 254x254 is output; Step 4: The 254x254 image is convolved with a 128-dimensional convolution kernel of 3x3 and a stride of 1 to output a 252x252 image; Step 5: The 52x252 image is processed by five 128-dimensional convolutional kernels of the single aggregation module to form a 3x3 image, and then processed by the weighted pooling effective compression activation module to output the first feature map; Step 6: The first feature map is processed by five 160-dimensional convolutional kernels of the single aggregation module 2 to form a 3x3 matrix, and then by the weighted pooling effective compression activation module to output the second feature map; Step 7: The second feature map is processed by the five 192-dimensional convolutional kernels of the single aggregation module three to form a 3x3 matrix, and then by the weighted pooling effective compression activation module to output the third feature map; Step 8: The output of the third feature map is processed by five 224-dimensional convolutional kernels of the single aggregation module four to form a 3x3 matrix, and then by the weighted pooling effective compression activation module to output the fourth feature map; The four primary feature images can be represented as follows: wherein A WP-esE is a weighted pooling effective compression excitation module function, U i denotes four said primary feature images.

6. The glass defect detection method using semi-supervised learning according to claim 5, characterized in that, The weighted pooling effective compression incentive module processes the outputs of the four different numbers of the single aggregation modules, and the processing formula is as follows: wherein, U i is four said primary feature images, F gapavg is a global average pooling operation on four different numbers of said single aggregation module outputs, F gapmax is a global maximum pooling operation on four different numbers of said single aggregation module outputs, a i is the activation value of the i-th pixel point of U i is the i-th weight value of U i is the activation value of the i-th pixel point of U i is the i-th weight value of U F gapavg and F gapmax are defined as follows, respectively: F gapmax (U i )=max(U n,m |n∈W,m∈H) Wherein, W and H represent the width and height of the feature map output by the single aggregation module, respectively.

7. The glass defect detection method using semi-supervised learning according to claim 1, characterized in that, The feature map attention layer step includes: Step 1: The feature map attention layer weights the input first feature map, second feature map, third feature map, and fourth feature map respectively to obtain four sets of features; Step 2: Apply a 1x1 convolution kernel and downsample the four sets of features to obtain four sets of output features; Step 3: Downsample and concatenate the first feature map, the second feature map, the third feature map, and the fourth feature map with the four sets of output features to obtain four types of glass feature attention images, namely the first feature attention map, the second attention map, the third feature attention map, and the fourth feature attention map.

8. The glass defect detection method using semi-supervised learning according to claim 1, characterized in that, A channel enhancement module is added to the feature pyramid layer to expand the receptive field of the feature map. The specific steps include: Step 1: The first feature attention map, the second feature attention map, the third feature attention map, and the fourth feature attention map are horizontally connected to the channel enhancement module to obtain the first channel enhancement feature map, the second channel enhancement feature map, the third channel enhancement feature map, and the fourth channel enhancement feature map of the glass. Step 2: The first channel enhancement feature map, the second channel enhancement feature map, the third channel enhancement feature map, and the fourth channel enhancement feature map of the glass cup are subjected to max pooling operation through the Spatial Pyramid Pooling (SPP) module to obtain four pooling feature maps of different sizes of glass cups, namely the first pooling feature map, the second pooling feature map, the third pooling feature map, and the fourth pooling feature map. Step 3: Select the smallest feature map from the pooled feature maps of four different sizes of glass cups and perform multiple upsampling and downsampling operations to obtain the fifth feature map, the sixth feature map, the seventh feature map, and the eighth feature map.

9. A glass defect detection method using semi-supervised learning according to claim 8, characterized in that, The channel enhancement module includes A set of 1x1 and 3x3 convolution kernels; activated by ReLU and Sigmoid functions.

10. A glass defect detection method using semi-supervised learning according to claim 1, characterized in that, The loss value is calculated using a loss function during training. Based on the glass detection model, the loss function is set as follows: Loss=L cls +L CIoU Among them, L cls For the loss function of classifying defects in glass cups, L CIoU Loss function for predicting bounding boxes for defects in glass cups; Where α is a positive hyperparameter, v is a penalty term used to measure the high similarity between the predicted glass defect bounding box and the real glass defect bounding box, and w gt h gt w pre and h pre These represent the width and height of the actual and predicted bounding boxes of the glass defect, respectively, where ρ is the Euclidean distance. pre_ctr and Box gt_ctr Let be the center coordinates of the predicted and actual values ​​of the glass defect bounding box, and c be the diagonal length of the minimum bounding box between the predicted and actual values ​​of the glass defect bounding box.

Images