Glass defect detection system and method based on unsupervised super-resolution

Abstract

The invention relates to the technical field of computer vision and image processing, provides a glass defect detection system and method based on unsupervised super-resolution, solves the technical problems that defect samples are difficult to collect and the detection precision of small defects is insufficient in industrial defect detection, and realizes the capability of amplifying the defects for detection. The system comprises a data acquisition module used for collecting original pictures containing defects and real pictures and forming a training data set, the real pictures serving as high-resolution pictures are used for guiding down-sampling pictures to recover texture details, and the original pictures are used for down-sampling to obtain low-resolution pictures with lost texture details; the super-division module is used for constructing a super-division model and training the super-division model by using the obtained training data set to obtain a super-division picture, namely a required high-resolution defect picture; and the defect detection module is used for marking defects of the obtained super-division pictures, then carrying out defect detection, and judging the quality and accuracy of the defect pictures.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a glass defect detection system and method based on unsupervised super-resolution. Background Art

[0002] With the rapid development of computer vision and deep learning technology, defect detection technology is playing an increasingly critical role in various industries, especially in the field of industrial manufacturing. The core goal of defect detection is to quickly and accurately identify various defects in materials or products, such as cracks, holes, deformation, uneven surface roughness, color differences, and internal structural abnormalities, through automated or semi-automated means. These defects may seriously affect the performance, service life or appearance of the product. Therefore, timely detection and correction of these problems are key steps to ensure product quality and improve production efficiency. In the glass manufacturing industry, automated production technology has become a trend. However, in this production environment, the generation of defective products is an unavoidable problem, such as broken edges, cracks, scratches, etc. of glass. In order to solve these problems, the manufacturing industry has begun to adopt advanced defect detection models such as YOLO and Faster-RCNN.

[0003] However, traditional defect detection methods often rely on manual labeling and then use machine learning models to locate and classify defects. Although such methods have been successful to a certain extent, their performance and generalization capabilities are gradually challenged with the rapid expansion of industrial production data, the diversification of defect types, and the increase in the accuracy of small defect detection. Especially in the face of complex environments and changing industrial scenarios, traditional methods are difficult to meet the needs of accurate detection and efficient processing. In addition, the cost of equipment to obtain high-resolution images is high, and in actual industrial environments, it is usually difficult to collect enough high-resolution defect images for training, resulting in unstable detection results. Therefore, we need to be able to accurately detect tiny defects without paired data sets. This unsupervised approach helps reduce costs and accelerate the collection of data sets. Accurate detection of small defects can avoid the model's false detection and omission of existing small defects.

[0004] At present, although there are a large number of defect detection models on the market, there is still no good solution for these problems. For example, the generative adversarial model and diffusion model can only be applied to situations with paired data. The iconic models are SRGAN model, ESRGAN model, latent diffusion model, etc. The conditional control type generative adversarial model generally requires detailed condition information, and the generation effect depends entirely on the level of detail of the given conditions. However, detailed condition information requires a lot of time to produce, such as the MaskGAN model. Vector quantization models cannot efficiently generate usable images. Often, only a few of the 100 images generated are usable, and the randomness is too large, such as the VQ-GAN model. Summary of the invention

[0005] The present invention aims to solve the technical problems of difficulty in collecting defect samples in industrial defect detection and insufficient detection accuracy of small defects, and to provide a glass defect detection system and method based on unsupervised super-resolution, so as to improve the resolution of images and detect small defects without a paired data set.

[0006] The above technical objectives of the present invention are achieved through the following technical solutions:

[0007] A technical solution of the present invention provides a glass defect detection system based on unsupervised super-resolution, comprising:

[0008] A data acquisition module is used to collect original pictures containing defects and real pictures to form a training data set. The real pictures are used as high-resolution pictures to guide the downsampled pictures to restore texture details, and the original pictures are used for downsampling to obtain low-resolution pictures with lost texture details.

[0009] The super-resolution module is used to build a super-resolution model and train the super-resolution model with the acquired training data set to obtain super-resolution images, that is, to obtain the required high-resolution defect images;

[0010] The defect detection module is used to perform defect detection after marking defects in the acquired super-resolution images, and to determine the quality and accuracy of the defective images.

[0011] According to a further technical solution, the super-resolution model includes: an image texture detail recovery module for restoring rich texture details from a downsampled image that has lost texture details;

[0012] The super-resolution generator module uses a window attention mechanism to super-resolve the downsampled image with restored texture details to obtain its super-resolution image, and adds the total loss after calculating the loss with the original image. The super-resolution model parameters are updated based on the total loss, which is the sum of domain gap loss, LR reconstruction loss, style loss, feature loss, HR reconstruction loss, and cycle loss.

[0013] The discriminator module uses a self-attention mechanism to calculate the reconstruction loss between the real image and the reconstructed image of the real image, and calculate the style loss between the downsampled image with restored texture details and the real image to determine whether the image is real or fake;

[0014] The scoring network module uses an image feature extraction and classification model based on the VGG network architecture to extract features from the reconstructed image and perform average opinion scoring. Based on the score, it helps the discriminator update parameters by adding decoder loss.

[0015] A further technical solution, the image texture detail recovery module comprises:

[0016] The encoder is used to extract features from the input image. It has several convolutional layers and uses a window attention mechanism.

[0017] The decoder is used to restore the encoded feature map tensor to a picture that conforms to the original data distribution. It adopts a window attention mechanism and includes a shallow feature extraction module, a deep feature extraction module and a high-quality image reconstruction module. The deep feature extraction module is provided with multiple residual blocks and a convolution layer.

[0018] According to a further technical solution, the scoring network module is provided with a content branch and an information branch, both branches are composed of four VGG blocks, and image features are extracted through the content branch and the information branch respectively, and then the output of the content branch is subtracted from the output of the information branch to extract the noise details in the input image.

[0019] Each VGG block includes convolution, normalization and activation functions to extract deep features of the image layer by layer. After feature extraction, the global average pooling GAP is used to compress the high-dimensional feature map into a 1x1 size to reduce the feature dimension. Finally, the classification result is generated through the combination of the fully connected layer and the activation function.

[0020] According to a further technical solution, the discriminator module adopts a discriminator network based on a U-Net structure combined with an attention mechanism, and the discriminator includes a downsampling encoding layer, an upsampling decoding layer and several attention blocks.

[0021] The output of the discriminator module uses a sigmoid function to normalize the output, and uses binary cross entropy loss to calculate the loss, the formula is:

[0022] As a further technical solution, the defect detection module adopts the yolo-v5 detection model.

[0023] Another technical solution of the present invention provides a glass defect detection method based on unsupervised super-resolution, using the above-mentioned defect detection system, comprising the following steps:

[0024] Step S1, collecting original pictures and real pictures containing defects based on the data acquisition module and forming a training data set;

[0025] Step S2, based on the super-resolution module, training the super-resolution model with the acquired training data set to obtain a super-resolution image;

[0026] Step S21: down-sample the original image to obtain a down-sampled image with a size reduced by four times;

[0027] Step S22: input the downsampled image into the encoder to obtain a feature map tensor; input the real image into the encoder to obtain a feature map tensor, and input the obtained two feature tensors into the discriminator and calculate the domain gap;

[0028] Step S23: input the feature map of the downsampled image obtained in step S22 into the super-resolution generator module to obtain a super-resolution image, input the original image and the super-resolution image into the discriminator to determine the authenticity, and calculate the reconstruction loss;

[0029] Step S24: successively inputting the feature map of the real image and the feature map after encoding the downsampled image into the decoder to obtain a reconstructed image of the real image and a restored detail image of the downsampled image, inputting the real image and the reconstructed real image into the discriminator to calculate the LR reconstruction loss; inputting the restored detail downsampled image and the real image into the discriminator to calculate the style loss, wherein the style loss includes perceptual loss, color loss, QA score loss and adversarial loss;

[0030] Step S25: Encode the downsampled image with restored details again to obtain F_s_b, and calculate the LR distance loss with the feature map during the initial encoding to prevent content loss;

[0031] Step S26: Input the feature map of the downsampled image with restored details into the super-resolution generator module to obtain a super-resolution image of the downsampled image, calculate the adversarial loss, perceptual loss, color loss and QA score loss to form a cyclic loss with the original image, and use the loss to update the super-resolution network of the super-resolution module;

[0032] Step S27: repeat steps S21 to S26, and use the optimizer to optimize the total loss until the loss function converges to obtain the trained super-resolution generator module. The total loss is the sum of domain gap loss, LR reconstruction loss, style loss, feature loss, HR reconstruction loss, and cycle loss.

[0033] Step S28: Generate a super-resolution image.

[0034] Step S3: Based on the defect detection module, the acquired super-resolution image is annotated with defects for defect detection, and the quality and accuracy of the defective image are determined.

[0035] In summary, the present invention has the following beneficial effects:

[0036] 1. Obtain high-resolution defect images required by the defect detection model through the super-resolution model, so that the defect part is magnified for easy detection. This method can train the model without paired high-resolution and low-resolution images to magnify the entire image before performing defect detection. This method can not only improve the accuracy of general defect detection, but also can well magnify tiny defects that are difficult to mark manually in the original image, thereby improving the accuracy of glass defect detection as a whole.

[0037] 2. By obtaining a large number of small defect samples, the richness of the data set is expanded, the impact of missed detection and false detection of small defects is reduced, the difficulty in collecting defect samples in industrial defect detection is solved, and the problem of insufficient sample quantity caused by defect samples being too small to be detected is solved, thereby greatly improving the robustness of the detection model and having good practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. It is obvious that the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a block diagram of a glass defect detection system based on unsupervised super-resolution provided by an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a glass defect detection method based on unsupervised super-resolution provided by an embodiment of the present invention;

[0041] Figure 3 is a flowchart of specific steps of step S2 of a glass defect detection method based on unsupervised super-resolution provided in one embodiment of the present invention;

[0042] Figure 4 is a structural diagram of a super-resolution module provided by an embodiment of the present invention;

[0043] Figure 5 is a structural diagram of a scoring network module in a super-resolution module provided by an embodiment of the present invention;

[0044] Figure 6is a structural diagram of an identifier module provided by an embodiment of the present invention;

[0045] Figure 7 is a comparison diagram of the pictures output by each model and the pictures output by the model of this embodiment, wherein: Figure 7 a is the original image; Figure 7 b is a super-resolution image output by an embodiment of the present invention, Figure 7 c is the super-resolution image output by the USR-DA model. Figure 7 d is the super-resolution image output by the DSGAN model; Figure 7 e is the super-resolution image output by the DUSGAN model. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] To make the advantages of the technical solution of the present invention more clear, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0048] Embodiment 1:

[0049] The present invention solves the technical problems of difficulty in collecting defect samples in industrial defect detection and insufficient detection accuracy for small defects, and realizes the ability to magnify defects for detection. The present invention can obtain a large number of small defect samples, expand the richness of the data set, reduce the impact of missed detection and false detection of small defects, and alleviate the problem of insufficient sample quantity caused by samples being too small to be detected, thereby greatly improving the robustness of the detection model and having good practicality.

[0050] This embodiment provides a defect detection system. Taking the detection of glass defects as an example, a glass defect detection system based on unsupervised super-resolution is specifically provided. Figure 1 As shown, the glass sink detection system includes: a data acquisition module, a super-resolution module and a defect detection module.

[0051] The data acquisition module is used to collect original glass images containing defects and real images to form a training data set. The real images are used as high-resolution images to guide the downsampled images to restore texture details, and the original glass images are used for downsampling to obtain low-resolution images with lost texture details.

[0052] The super-resolution module is used to build a super-resolution model and train the super-resolution model with the acquired training data set to obtain super-resolution images, that is, to obtain the required high-resolution defect images. This super-resolution model is an unsupervised super-resolution model that does not require paired LR\HR data. It is difficult to collect paired low-resolution and high-resolution images in industrial production, which greatly saves costs.

[0053] The defect detection module is used to perform defect detection after marking the defects in the acquired super-resolution images, and to determine the quality and accuracy of the defective images.

[0054] like Figure 4 The structural diagram of the super-resolution module shown in the figure shows that the super-resolution model includes: image texture detail recovery module, super-resolution generator module, discriminator module, and QA scoring network module.

[0055] The image texture detail recovery module is used to restore rich texture details from the downsampled image that has lost texture details. The image texture detail recovery module includes an encoder and a decoder arranged in sequence from front to back. The encoder is used to extract features from the input image, and it has several convolutional layers, which adopts a window attention mechanism. The decoder is used to restore the encoded feature map tensor to a picture that conforms to the original data distribution, and it adopts a window attention mechanism. It includes a shallow feature extraction module, a deep feature extraction module and a high-quality image reconstruction module, and the deep feature extraction module is provided with multiple residual blocks and a convolutional layer.

[0056] The super-resolution generator module adopts the window attention mechanism to super-resolution the downsampled image with restored texture details to obtain its super-resolution image, and adds the total loss after calculating the loss with the original image. The super-resolution model parameters are updated based on the total loss, which is the sum of domain gap loss, LR reconstruction loss, style loss, feature loss, HR reconstruction loss, and cycle loss.

[0057] The QA scoring network module uses an image feature extraction and classification model based on the VGG network architecture to extract features from the reconstructed images and perform average opinion scoring. Based on the score, it helps the discriminator update parameters by adding decoder loss.

[0058] like Figure 5As shown, in this embodiment, the scoring network module adopts a QA scoring network, which is an image feature extraction and classification model based on the VGG network architecture, mainly used to process image quality assessment or classification tasks. The network extracts image features through two feature extraction paths, the content branch and the information branch, respectively, and then calculates the difference between the features of the two paths to capture the subtle differences in the image. Each path consists of multiple VGG blocks, each of which contains convolution, normalization, and activation functions to extract deep features of the image layer by layer. After feature extraction, global average pooling (GAP) is used to compress the high-dimensional feature map into a size of 1x1 to reduce the feature dimension. Finally, through the combination of the fully connected layer and the activation function, the model generates a classification result. The scoring network design focuses on detail feature extraction and difference comparison, and is suitable for image quality assessment or complex classification tasks.

[0059] The discriminator module uses a discriminator with a self-attention mechanism to calculate the reconstruction loss between the real image and the reconstructed image of the real image, and calculate the style loss between the downsampled image with restored texture details and the real image to determine whether the image is real or fake.

[0060] like Figure 6 As shown, the discriminator in this embodiment is a discriminator network based on the U-Net structure and combined with the attention mechanism, which is dedicated to the discrimination task in the generative adversarial network. First, the input image passes through multiple convolutional layers conv0, conv1, conv2, and conv3, and the features are gradually extracted and downsampled. The size of the feature map is gradually reduced, but the feature expression ability is enhanced. Then, multiple attention blocks (AttentionBlock, AB) are introduced in the deep layer of the network. By combining the input features with the global control signal, an attention map is generated to highlight the important areas in the image and enhance the discrimination effect. The attention block acts on feature maps of different scales and can capture the relationship between details and global information. In the central part of the network, the deepest features are extracted through additional convolution operations, and then the spatial resolution of the feature map is gradually restored through the cascade block (CatBlock, CB) and the upsampling layer. The cascade block fuses the features of the upsampling path and the downsampling path, so that the network can combine feature information of different scales. Finally, the upsampled features are further processed through additional convolutional layers, and the final discrimination results are output through the classification layer (fully connected layer). The AttentionUNetDiscriminator attention mechanism U-Net discriminator model ensures the fusion of multi-scale features through the up and down sampling structure of U-Net. Combined with the attention mechanism, it effectively enhances the attention of important areas, and ultimately improves the performance of the discriminator in the generative adversarial network.

[0061] The decoder and super-resolution generator modules in this embodiment both have the structure of a window attention mechanism.

[0062] Its working principle is:

[0063] First, the input image is divided into fixed-size image patches (Patch), and each image patch is regarded as a feature vector. These patches are converted into feature representations through the embedding layer and input into subsequent modules. After that, local self-attention is calculated in each window, which reduces the amount of calculation of global self-attention. Within the window, the attention mechanism can capture the interdependence and association between image patches, thereby extracting local details. In this module, self-attention is not calculated over the entire image, but within a local window. Assuming that the image is divided into several non-overlapping windows, each of size M×M, then the window self-attention formula is similar to the above self-attention formula, but is applied within each window:

[0064]

[0065] Where: Q w is the query within the window, K w is the key within the window, V w is the value vector within the window; B is the relative position bias matrix (Relative Position Bias), which is used to model the relative position relationship between different positions within the window.

[0066] The self-attention mechanism within the window can significantly reduce the amount of computation, because the size of the window is usually much smaller than the size of the entire image. In order to preserve the relative spatial relationship of the image blocks within each window, the model adds relative position deviations to the self-attention calculation, which helps the window self-attention mechanism better understand the spatial relationship between pixels. The relative position deviation is used to capture the relative position relationship between different pixels within the window. This deviation is obtained by looking up a relative position deviation table B. The calculation formula for the relative position deviation is:

[0067] B=LookupTable(p i -p j )

[0068] Among them, p i and p j is the position index of two pixels in the window, p i -p j Indicates their relative positions. Each block contains not only a self-attention module, but also an MLP module. The function of the MLP module is to perform nonlinear transformation on the features output by the attention module. The MLP layer usually uses two layers of fully connected networks with an activation function in between:

[0069] MLP(x)=W 2 ·σ(W 1 x)

[0070] Where: W 1 and W 2 is the weight matrix; σ is the activation function, preferably the GELU activation function. The MLP layer helps improve the nonlinear expression ability of the model. In order to extract features at different levels, Swin Transformer uses image block merging operations (similar to the pooling operation in CNN). During the image block merging process, the spatial size of the feature map will be reduced, while the number of channels will increase. The specific operation is to merge adjacent feature blocks together and obtain higher-level features through linear transformation. The formula is:

[0071] X′=Concat(X 2i，2j , X 2i+1，2j , X 2i，2j+1 , X 2i+1，2j+1 )W

[0072] Where: X i,j Represents the features at a specific location; W is the weight matrix. By merging adjacent feature blocks, the model can extract more global information with a larger receptive field.

[0073] The loss of the super-resolution model includes the adversarial loss between the super-resolution generator module and the discriminator, the LR reconstruction loss of the image texture detail recovery module, the QA score loss, the HR reconstruction loss of the super-resolution generator module, the cycle loss, and the style loss. The final loss formula of the entire model is:

[0074] L total =λ adv L adv +λ LR_recon L LR_recon +λ QA L QA +λ HR_recon L HR_recon +λ cycle L cycle +λ style L style

[0075] The QA scoring loss is to use the QA scoring network to perform MOS scoring on the image. The calculation formula of MOS scoring is as follows:

[0076]

[0077] LR reconstruction loss and color loss are both L1 losses, and the formula is:

[0078]

[0079] The adversarial loss between the generator and the discriminator is the adversarial loss, usually using mean squared error

[0080] (MSE) is used as pixel-level loss, and the formula is:

[0081]

[0082] Among them, I pred(i) is the predicted value of the i-th pixel, I gt(i) is the true value of the ith pixel, and N is the total number of pixels in the image.

[0083] The HR reconstruction loss is a combination of L1 loss, perceptual loss, color loss, adversarial loss, and QA score loss;

[0084] L rec_G_SR =loss_L1_rec+λ percept last_per_loss+λ adv ·loss_G_Y_s_s+β·last_col_loss+loss_QA_rec

[0085] Among them, the perceptual loss is calculated by using the high-level features of the pre-trained convolutional network, VGG or ResNet, to calculate the difference between the predicted image and the target image. The formula is:

[0086]

[0087] Among them, φ is the output feature map of a specific layer of the pre-trained network, H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map.

[0088] The sty loss in this example includes perceptual loss, adversarial loss, color loss, and QA score loss.

[0089] L sty_G_t =loss_G_X_s_t+β per last_per_loss_sty+β col ·last_col_loss_sty+loss_QA_sty

[0090] The cycle loss is a combination of QA loss, perceptual loss, color loss, L1 loss, and adversarial loss. The formula is as follows:

[0091] L cycle =λ QA L QA +λ perceptual L perceptual +λ color L color +λ L1 L L1 +λ adv L advThe innovation of the super-resolution model in this embodiment compared with other unsupervised super-resolution methods is:

[0092] 1. Distill the network capability of image reconstruction into the super-resolution model and control the generated images by setting multiple discriminators.

[0093] 2. The window attention mechanism is integrated into the decoder and super-resolution generator to obtain a robust decoder and super-resolution generator. The advantage of this technology is that due to the weak distillation ability, the texture details of the images obtained by the ordinary decoder and super-resolution generator are relatively smooth and lack authenticity. The window attention mechanism divides the image into multiple small blocks for processing, which is conducive to restoring the fine texture of the image.

[0094] 3. In order to have a discriminator with the same capabilities as the decoder, this embodiment designs a discriminator with a self-attention mechanism, which can connect the whole picture and let the model focus on the high-frequency part. In this way, the decoder and the discriminator have equivalent capabilities, avoiding the imbalance phenomenon.

[0095] 4. In order to prevent the decoder from over-learning in the reconstruction module, this embodiment introduces a QA scoring network module, which is trainable and can directly perform an average opinion score on the generated images. What is important is that the score can be added to the total loss as a loss, which can well regulate the ability of the decoder.

[0096] In summary, the technical effects brought about by the above technical solution are as follows:

[0097] The high-resolution defect image required by the defect detection model is obtained through the super-resolution model, so that the defect part is magnified for easy detection. This method can train the model without paired high-resolution and low-resolution images to magnify the entire image before performing defect detection. This method can not only improve the accuracy of general defect detection, but also can well magnify tiny defects that are difficult to mark manually in the original image, thereby improving the accuracy of glass defect detection as a whole.

[0098] By obtaining a large number of small defect samples, the richness of the data set is expanded, the impact of missed detection and false detection of small defects is reduced, the difficulty in collecting defect samples in industrial defect detection is solved, and the problem of insufficient sample number caused by defect samples being too small to be detected is solved, thereby greatly improving the robustness of the detection model and having good practicality.

[0099] The adopted QA scoring network can, on the one hand, effectively suppress the noise of the generated images and prevent the super-resolution network from over-learning and reconstructing unnecessary content in the network. On the other hand, it can be used as the loss of the generator to update the network, making the generated images more realistic and with richer textures.

[0100] The required encoder-decoder and super-resolution modules use the window attention mechanism to improve efficiency and performance in image processing tasks. Window attention greatly improves the training and inference speed of the model by focusing on local areas and reducing global calculations, while maintaining or improving the accuracy of the model.

[0101] Embodiment 2

[0102] like Figure 2 and Figure 3 As shown, this embodiment provides a method for glass defect detection based on unsupervised super-resolution, comprising the following steps:

[0103] Step S1, based on the data acquisition module, collect original pictures containing defects and real pictures to form a training data set.

[0104] Specifically, we collect original images containing defects as a high-resolution HR dataset, copy one as a low-resolution LR dataset, and merge the two to form a training set.

[0105] Step S2, based on the super-resolution module, training the super-resolution model with the acquired training data set to obtain a super-resolution image;

[0106] Step S21: down-sample the original image to obtain a down-sampled image with a size reduced by four times;

[0107] Specifically, the low-resolution LR and the high-resolution HR are input into the data loader, the low-resolution LR is cropped to 64×64, the HR is cropped to 256×256, and the cropped high-resolution HR is downsampled to 64×64 to obtain a picture with lost high-frequency information, wherein the picture with lost high-frequency information is a picture with lost texture details.

[0108] Step S22: input the downsampled image into the encoder to obtain a feature map tensor; input the real image into the encoder to obtain a feature map tensor, and input the obtained two feature tensors into the discriminator and calculate the domain gap;

[0109] Specifically, the downsampled image obtained in step S21 and the cropped LR image are fed into the encoder for encoding to obtain two feature vectors, and the discriminator is used to calculate the score difference between the two feature vectors to obtain the domain loss, forcing the encoder to reduce the distance between the downsampled image and the LR image during the next encoding. The encoded tensor of the downsampled image obtained in step S22 is fed into the super-resolution generator module to obtain a super-resolution image, and the HR reconstruction loss between the original image and the super-resolution image is calculated to optimize the parameters of the super-resolution module.

[0110] Step S23: input the feature map of the downsampled image obtained in step S22 into the super-resolution generator module to obtain a super-resolution image, input the original image and the super-resolution image into the discriminator to determine the authenticity, and calculate the reconstruction loss;

[0111] Step S24: successively inputting the feature map of the real image and the feature map after encoding the downsampled image into the decoder to obtain a reconstructed image of the real image and a restored detail image of the downsampled image, inputting the real image and the reconstructed real image into the discriminator to calculate the LR reconstruction loss; inputting the restored detail downsampled image and the real image into the discriminator to calculate the style loss, wherein the style loss includes perceptual loss, color loss, QA score loss and adversarial loss;

[0112] Specifically, the two feature map tensors obtained in step S2 are input into the decoder to obtain reconstructed images of the two; the reconstructed image of the low-resolution LR image and the low-resolution LR image are input into the discriminator and the LR reconstruction loss is calculated; the reconstruction of the downsampled image and the LR low-resolution image are input into the discriminator and the style loss is calculated.

[0113] Step S25: Encode the downsampled image with restored details again to obtain F_s_b, and calculate the LR distance loss with the feature map during the initial encoding to prevent content loss;

[0114] Specifically, the reconstructed image of the downsampled image is encoded again by the encoder to obtain its feature tensor and the L1 loss is calculated on the encoded tensor of the obtained downsampled image to ensure content consistency.

[0115] Step S26: Input the feature map of the downsampled image with restored details into the super-resolution generator module to obtain a super-resolution image of the downsampled image, calculate the adversarial loss, perceptual loss, color loss and QA score loss to form a cyclic loss with the original image, and use the loss to update the super-resolution network of the super-resolution module;

[0116] Step S27: Repeat steps S21 to S26, and use the optimizer to optimize the total loss until the loss function converges to obtain the trained super-resolution generator module. The total loss is the sum of domain gap loss, LR reconstruction loss, style loss, feature loss, HR reconstruction loss, and cycle loss.

[0117] Specifically, each cycle is counted as an iteration, and the optimizer is used to optimize the total loss until the loss function converges, thus completing the training of the entire super-resolution model.

[0118] Step S28, generating a super-resolution image.

[0119] Specifically, the encoded tensor of the reconstructed image is input into the super-resolution generator module to obtain a clean super-resolution image, and is input into the discriminator together with the original HR for cyclic loss.

[0120] Step S3: Based on the defect detection module, the acquired super-resolution image is annotated with defects for defect detection, and the quality and accuracy of the defective image are determined.

[0121] Specifically, all super-resolved images are annotated with defects and sent to the yolo-v5 defect detection model.

[0122] Experimental verification:

[0123] The present invention selects mobile phone screen glass as a data set for experiment to verify the ability of the super-resolution model. In this experiment, 251 pictures are selected as the training set and 32 pictures are selected as the test set.

[0124] The image is input into the encoder to obtain the extracted feature tensor. The two feature tensors obtained at this time are from the clean domain and the downsampled domain. Our goal is to make the generated image pass through the discriminator, and the discriminator cannot distinguish which domain the image belongs to. Therefore, the domain gap loss is adopted:

[0125] Then the feature tensors of the two are input into the decoder to generate an image. Here, in order to enable the image from the downsampled domain to learn how to reconstruct the texture features during reconstruction, the style loss is calculated for the real LR image and the reconstructed downsampled image. The formula is as follows:

[0126] L sty_G_t =loss_G_X_s_t+β per last_per_loss_sty+β col ·last_col_loss_sty+loss_QA_sty

[0127] Finally, the reconstructed downsampled image is sent to the super-resolution generator module to obtain the super-resolution image. In order to keep it close to the original image, a cyclic loss is used. The formula is as follows:

[0128] L cycle =λ QA L QA +λ perceptual L perceptual +λ color L color +λ L1 L L1 +λ adv L adv

[0129] The final loss formula of the entire model is:

[0130] L=Ladv+L1+Lsty+L QA +Lcyc+L HR

[0131] Repeat the above process and use the adam optimizer to optimize the loss until the model converges to obtain a trained super-resolution model.

[0132] The images generated by the super-resolution model are then labeled and sent to the defect detection model to detect defects.

[0133] like Figure 7 As shown in the figure Figure 7 a is the original image, Figure 7 b is the super-resolution image output by this embodiment, Figure 7 c is the super-resolution image obtained based on the USR-DA model. Figure 7 d is the super-resolution image obtained by the DSGAN model. Figure 7 e is the super-resolution image obtained by the DUSGAN model.

[0134] Using the same experimental conditions, experiments were conducted on USR-DA, DSGAN, and DUSGAN, several annotated generative models, and corresponding experimental results were obtained. It can be seen that the detailed textures of the images produced by USR-DA will have unpleasant shadows; the images produced by DSGAN are too smooth and lack authenticity; and the images produced by DUSGAN will have slight noise; while the images produced by our model will not have the above problems.

[0135] Table 1 is a comparison of experimental data of each model

[0136] original 0.748 USR-DA 0.747 DSGAN 0687 DUSGAN 0.712 This model 0.824

[0137] In order to verify the effectiveness of the present invention for defect detection, as shown in Table 1, 251 defect images were collected as training data, where the defect types were divided into three categories: scratches, friction, and spots. 32 defect images were used as tests to train a yolo model for verification. At the same time, we also used the 4 models for comparison, each of which was super-resolved with the training set and test set, and used the same experimental conditions to train a detection model for each, for comparison. As can be seen from the data in Table 1, the generated image effect of the implementation model is the worst, and the trained detection model has the strongest ability.

[0138] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the above-mentioned embodiments of the methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0139] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A glass defect detection system based on unsupervised super-resolution, characterized in that: include: A data acquisition module is used to collect original pictures containing defects and real pictures to form a training data set. The real pictures are used as high-resolution pictures to guide the downsampled pictures to restore texture details, and the original pictures are used for downsampling to obtain low-resolution pictures with lost texture details. The super-resolution module is used to build a super-resolution model and train the super-resolution model with the acquired training data set to obtain super-resolution images, that is, to obtain the required high-resolution defect images; The defect detection module is used to perform defect detection after marking defects in the acquired super-resolution images, and to determine the quality and accuracy of the defective images.

2. The defect detection system according to claim 1, characterized in that: The super-resolution model includes: Image texture detail restoration module, used to restore rich texture details from downsampled images that have lost texture details; The super-resolution generator module uses a window attention mechanism to super-resolve the downsampled image with restored texture details to obtain the feature map of its super-resolved image, and adds the total loss after calculating the loss with the original image. The super-resolved model parameters are updated based on the total loss, which is the sum of domain gap loss, LR reconstruction loss, style loss, feature loss, HR reconstruction loss, and cycle loss. The discriminator module uses a self-attention mechanism to calculate the reconstruction loss between the real image and the reconstructed image of the real image, and calculate the style loss between the downsampled image with restored texture details and the real image to determine whether the image is real or fake; The scoring network module uses an image feature extraction and classification model based on the VGG network architecture to extract features from the reconstructed image and perform average opinion scoring. Based on the score, it helps the discriminator update parameters by adding decoder loss.

3. The defect detection system according to claim 2, characterized in that: The image texture detail restoration module comprises: The encoder is used to extract features from the input image. It has several convolutional layers and uses a window attention mechanism. The decoder is used to restore the encoded feature map tensor to a picture that conforms to the original data distribution. It adopts a window attention mechanism and includes a shallow feature extraction module, a deep feature extraction module and a high-quality image reconstruction module. The deep feature extraction module is provided with multiple residual blocks and a convolution layer.

4. The defect detection system according to claim 2, characterized in that: The scoring network module is provided with a content branch and an information branch, both branches are composed of four VGG blocks, and image features are extracted through the content branch and the information branch respectively, and then the output of the content branch is subtracted from the output of the information branch to extract the noise details in the input image.

5. The defect detection system according to claim 4, characterized in that: Each VGG block includes convolution, normalization and activation functions to extract deep features of the image layer by layer. After feature extraction, the global average pooling GAP is used to compress the high-dimensional feature map into a 1x1 size to reduce the feature dimension. Finally, the classification result is generated through the combination of the fully connected layer and the activation function.

6. The defect detection system according to claim 2, characterized in that: The discriminator used in the discriminator module is a discriminator network based on a U-Net structure combined with an attention mechanism, and the discriminator includes a downsampling encoding layer, an upsampling decoding layer and several attention blocks.

7. The defect detection system according to claim 6, characterized in that: The output of the discriminator module uses a sigmoid function to normalize the output, and uses binary cross entropy loss to calculate the loss, the formula is:

8. The defect detection system according to any one of claims 1 to 5, characterized in that: The defect detection module adopts the yolo-v5 detection model.

9. A glass defect detection method based on unsupervised super-resolution, using the defect detection system of claim 1, characterized in that: The following steps are involved: Step S1, collecting original pictures and real pictures containing defects based on the data acquisition module and forming a training data set; Step S2, based on the super-resolution module, training the super-resolution model with the acquired training data set to obtain a super-resolution image; Step S3: Based on the defect detection module, the acquired super-resolution image is annotated with defects for defect detection, and the quality and accuracy of the defective image are determined.

10. The defect detection method according to claim 9, wherein step S2 specifically comprises the following steps: Step S21: down-sample the original image to obtain a down-sampled image with a size reduced by four times; Step S22: input the downsampled image into the encoder to obtain a feature map tensor; input the real image into the encoder to obtain a feature map tensor, and input the obtained two feature tensors into the discriminator and calculate the domain gap; Step S23: input the feature map of the downsampled image obtained in step S22 into the super-resolution generator module to obtain a super-resolution image, input the original image and the super-resolution image into the discriminator to determine the authenticity, and calculate the reconstruction loss; Step S24: successively inputting the feature map of the real image and the feature map after encoding the downsampled image into the decoder to obtain a reconstructed image of the real image and a restored detail image of the downsampled image, and inputting the real image and the reconstructed real image into the discriminator to calculate the LR reconstruction loss; Input the downsampled image with restored details and the real image into the discriminator to calculate the style loss, which includes perceptual loss, color loss, QA score loss and adversarial loss; Step S25: Encode the downsampled image with restored details again to obtain F_s_b, and calculate the LR distance loss with the feature map during the initial encoding to prevent content loss; Step S26: Input the feature map of the downsampled image with restored details into the super-resolution generator module to obtain a super-resolution image of the downsampled image, calculate the adversarial loss, perceptual loss, color loss and QA score loss to form a cyclic loss with the original image, and use the loss to update the super-resolution network of the super-resolution module; Step S27: repeat steps S21 to S26, and use the optimizer to optimize the total loss until the loss function converges to obtain the trained super-resolution generator module. The total loss is the sum of domain gap loss, LR reconstruction loss, style loss, feature loss, HR reconstruction loss, and cycle loss. Step S28: Generate a super-resolution image.

Images