Glass surface defect intelligent detection method and system

Abstract

The invention relates to the technical field of glass quality detection, in particular to an intelligent glass surface defect detection method and system.The method comprises the steps that a glass surface image is obtained and subjected to data preprocessing, a GAN-based network model is constructed after data preprocessing is completed, and an existing glass surface defect image data set serves as input of the model; generating a new glass surface defect image data set; and constructing a to-be-detected glass surface image defect detection model based on deep learning, training the model by adopting the generated new glass surface defect image data set, carrying out intelligent detection on the to-be-detected glass surface image defect after training is completed, outputting a detection result, and carrying out visual output on the detection result. According to the method, image processing is combined with deep learning, so that the defect characteristics of the glass surface are effectively learned, the surface defects of the to-be-detected glass are accurately detected, the defect types are classified, and the detection precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of glass quality inspection, and particularly relates to an intelligent detection method and system for glass surface defects. Background Art

[0002] Glass has a wide range of applications in modern industry and daily life, such as in industries like construction, automotive, and electronics. However, during the glass production process, due to factors such as raw materials and processes, various defects may appear on the glass surface, such as scratches, bubbles, inclusions, etc. These defects not only affect the appearance quality of the glass but also reduce its strength and optical properties. Therefore, accurate and efficient detection of glass surface defects is of great significance.

[0003] Traditional glass surface defect detection methods mainly rely on manual visual inspection. This method has many drawbacks, such as low detection efficiency, poor accuracy, and being easily affected by human factors. With the development of computer technology and image processing technology, artificial intelligence detection methods have gradually been applied to the field of glass surface defect detection. However, most of the existing artificial intelligence detection methods are based on traditional image processing algorithms, and the detection effect for tiny defects under complex backgrounds is not ideal, making it difficult to accurately classify different types of defects. Summary of the Invention

[0004] In view of the above-mentioned technical deficiencies, the present invention provides an intelligent detection method and system for glass surface defects to improve the efficiency and accuracy of glass surface defect detection.

[0005] The present invention is achieved through the following technical solutions:

[0006] An intelligent detection method for glass surface defects is provided, and the method includes the following steps:

[0007] Step S10: Build a closed dark box type glass surface image acquisition platform and set a ring-shaped uniform diffuse light source, an industrial camera, and a stage therein. Place the glass to be tested on the stage, adjust the position and parameters of the industrial camera, and make the lens located above the glass to be tested for shooting to obtain the surface image of the glass to be tested;

[0008] Step S20: Perform data preprocessing on the obtained surface image of the glass to be tested, including filtering and denoising and image enhancement;

[0009] Step S30: Construct a GAN-based network model, use the existing glass surface defect image dataset as the input of the model, and generate a new glass surface defect image dataset;

[0010] Step S40: Build a defect detection model for the surface image of the glass to be tested based on deep learning. Use the generated new glass surface defect image dataset to train the model. After training is completed, perform intelligent defect detection on the surface of the glass to be tested and output the detection results;

[0011] Among them, in step S30, build a network model based on GAN, including a generator and a discriminator. By setting the network layer structure and learning parameters in the generator, and the network layer structure and network parameters in the discriminator, make the network model based on GAN adapt to the learning and generation of glass surface defect images.

[0012] Preferably, when the industrial camera in step S10 takes the surface image of the glass to be tested, it automatically adjusts the focal length and aperture of the camera according to the clarity of the presented image, and takes pictures from three angles of 0°, 45°, and 90°. The surface image of the glass to be tested taken by the industrial camera is a lossless image with a 4k resolution.

[0013] Preferably, the steps of data preprocessing for the obtained surface image data of the glass to be tested in step S20 include:

[0014] Filtering and denoising: Use Gaussian filtering to remove the noise in the surface image of the glass to be tested. Taking the pixel point (x, y) in the surface image of the glass to be tested as the center, select a certain neighborhood range for weighted calculation. The central pixel point of the selected neighborhood range is (x 0 , y 0 ), and the calculation formula is shown in Equation (1):

[0015]

[0016] Among them, g(x, y) is the pixel value of a certain point (x, y) on the surface image of the glass to be tested after Gaussian filtering, σ is the standard deviation of the Gaussian distribution, (x - x 0 ) is the distance deviation of the pixel point (x, y) relative to the central pixel point (x 0 , y 0 ) in the horizontal direction, and (y - y 0 ) is the distance deviation of the pixel point (x, y) relative to the central pixel point (x 0 , y 0 ) in the vertical direction;

[0017] Image enhancement: After filtering and denoising, perform image enhancement operations by changing the brightness and contrast of the surface image of the glass to be tested. The calculation formula is shown in Equation (2):

[0018] I new = a×I old + b (2)

[0019] Among them, I oldis the pixel value of the image of the glass surface to be measured before image enhancement, I new is the pixel value of the image of the glass surface to be measured after image enhancement. a is the contrast adjustment coefficient, and b is the brightness adjustment coefficient. When a > 1, the contrast of the image of the glass surface to be measured is increased; when a < 1, the contrast of the image of the glass surface to be measured is decreased; when a = 1, the contrast of the image of the glass surface to be measured remains unchanged. The brightness of the image of the glass surface to be measured is adjusted by adjusting the value of b. When the value of b increases, the brightness of the image of the glass surface to be measured increases; when the value of b decreases, the brightness of the image of the glass surface to be measured decreases.

[0020] Preferably, in the step S30, the steps of constructing a GAN-based network model to generate a new glass surface defect image dataset include:

[0021] Data preparation: Collect glass surface defect images including various defect types, such as cracks, bubbles, and scratches, etc., and perform normalization processing. After the normalization processing is completed, scale and rotate the glass surface defect images to obtain glass surface defect images after normalization, glass surface defect images after scaling and normalization, and glass surface defect images after rotation and normalization;

[0022] Construct a generator: Based on CNN, construct a generator. Remove the convolutional layer in the CNN and add a transposed convolutional layer. The input of the generator is a random noise vector of a glass surface defect image. Through the transposed convolutional layer and the activation function ReLU, gradually convert the random noise vector of the glass surface defect image into an image similar to the real glass surface defect image. For example, use multiple transposed convolutional layers, and each layer gradually increases the resolution and the number of channels of the image, and finally generates an RGB image;

[0023] Construct a discriminator: Based on CNN, construct a discriminator. In the discriminator, use a convolutional layer with multi-scale convolutional kernels. The input of the discriminator is the glass surface defect image output by the generator. Through the convolutional layer, pooling layer, and fully connected layer, using the activation function LeakyReLU and the activation function Sigmoid, the discriminator outputs a probability value, indicating the possibility that the glass surface defect image output by the generator is a real glass surface defect image;

[0024] Training process: Initialize the parameters in the generator and the discriminator, including hyperparameters such as the learning rate and batch size. The learning rate is generally set between 0.0001 and 0.001, and the batch size is set according to the computing resources and the size of the glass surface defect image dataset, such as set to 32 or 64. After setting, alternately train in the order of training the discriminator first and then the generator, and repeat in a cycle;

[0025] Evaluation and screening: After the training is completed, the generator is used to generate glass surface defect images. The quality of the generated images is evaluated by the peak signal-to-noise ratio and the structural similarity index. The glass surface defect images generated by the generator that pass the evaluation and the collected real glass surface defect images are combined to form a new glass surface defect image dataset, which is used for the training, validation, and testing of the deep learning-based glass surface image defect detection model for the glass to be tested in step S40.

[0026] Preferably, in step S40, a convolutional neural network combined with an improved support vector machine is used to construct a deep learning-based glass surface image defect detection model for the glass to be tested. By introducing an adaptive kernel function, the parameters are dynamically adjusted according to the distribution of the input glass surface image features to be detected, enhancing the classification ability of the model for different types of defects and improving the accuracy and generalization of classification. After the training is completed, intelligent defect detection of the glass surface to be tested is performed, and the output detection result is that there are defects or no defects on the glass surface to be tested. When the output result is that there are defects on the glass surface, the detected glass surface defect category is output at the same time, and the position where the defect appears is accurately marked in the glass surface image where the defect is detected.

[0027] Among them, the training steps of the deep learning-based glass surface image defect detection model for the glass to be tested include:

[0028] Dataset preparation: The new glass surface defect image dataset obtained in step S30 is divided into a training set, a validation set, and a test set according to the ratio of 7:2:1. The corresponding glass surface defect types are marked in the training set image data for the model to learn the defect characteristics of different types of glass surface images to be tested.

[0029] Determine the loss function and optimizer: The cross-entropy loss function is used as the loss function of the model. The smaller the value of the loss function, the more accurate the detection result of the model. The Adam optimization algorithm is used as the optimizer to adaptively adjust the learning rate and dynamically update the model parameters. The initial learning rate is set to 0.001 and is attenuated and adjusted according to the model training results and the number of training epochs.

[0030] Model construction: The entire network model includes an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The input to the input layer is the training set with corresponding defect types labeled in the new glass surface defect image dataset. In the convolutional layers, ReLU is used as the activation function, and different scales of convolutional kernels are adopted in each convolutional layer. The pooling layer is set after the last convolutional layer and uses the max pooling method. The fully connected layer is set after the pooling layer and is used to connect the pooling layer and the output layer. The fully connected layer converts the feature map output by the pooling layer into a one-dimensional vector form and selects ReLU as the activation function. The output layer is set as the last layer of the entire glass surface image defect detection model to be tested. According to the number of corresponding defect categories in the glass surface image to be tested, after Softmax processing, it outputs the probability distribution of each defect category. Each probability distribution corresponds to the confidence level of the defect category. The confidence level threshold is set to 0.4. When the confidence level is less than 0.4, the output is no defect. When the confidence level is greater than or equal to 0.4, the output is defective and the defect category is also output. The defect category is the defect category with the highest probability value in the probability distributions of each output defect category. The confidence level threshold is dynamically adjusted according to the detection requirements of the model. When the detection requirements for the glass surface to be tested are strict, the confidence level threshold can be adjusted to between 0 and 0.4. When the detection requirements for the glass surface to be tested are loose, the confidence level threshold can be adjusted to between 0.4 and 1.

[0031] Model training and verification: After the model is constructed, adjust the model parameters according to the above settings. Use the training set with corresponding defect types labeled in the new glass surface defect image dataset allocated above as the input to train the glass surface image defect detection model to be tested. After each round of training is completed, use the allocated validation set to verify the trained model.

[0032] Model evaluation: Set the cross-entropy loss function threshold. When verifying the trained model with the allocated validation set after each round of training is completed, when the value of the loss function is less than or equal to the set threshold, end the training and verification of the glass surface image defect detection model to be tested, and use the allocated test set for model evaluation.

[0033] Model optimization: According to the model evaluation results, when the model detection accuracy is less than 90%, adjust the model parameters and retrain until the model detection accuracy is greater than or equal to 90% to obtain the optimal combination of model parameters and determine the corresponding version of the glass surface image defect detection model to be tested.

[0034] Among them, the defect categories output by the glass surface image defect detection model to be tested based on deep learning include cracks, bubbles, and scratches. Different defect types have different corresponding features. The model determines the defect types existing in the input glass surface defect image to be tested by learning the features of different glass surface image defect types.

[0035] In addition, to achieve the above object, the present invention also provides an intelligent glass surface defect detection system, which includes:

[0036] Image acquisition module for the glass surface to be measured: It is used to build a closed dark box type glass surface image acquisition platform and set a circular uniform diffuse reflection light source, an industrial camera and a stage therein. Place the glass to be measured on the stage, adjust the position and parameters of the industrial camera, and make the lens located above the glass to be measured for shooting to obtain the image of the glass surface to be measured;

[0037] Data preprocessing module for the image of the glass surface to be measured: It is used to perform data preprocessing on the obtained image of the glass surface to be measured, including filtering and denoising and image enhancement;

[0038] Glass surface defect image dataset generation module: It is used to construct a network model based on GAN, take the existing glass surface defect image dataset as the input of the model, and generate a new glass surface defect image dataset;

[0039] Intelligent defect detection module for the glass surface to be measured: It is used to construct a defect detection model for the image of the glass surface to be measured based on deep learning, train the model with the generated new glass surface defect image dataset, perform intelligent defect detection on the glass surface to be measured after training is completed, and output the detection result;

[0040] In the glass surface defect image dataset generation module, the construction of the network model based on GAN includes a generator and a discriminator. By setting the network layer structure and learning parameters in the generator and the network layer structure and network parameters in the discriminator, the network model based on GAN is adapted to the learning and generation of glass surface defect images.

[0041] In addition, to achieve the above object, the present invention also provides an intelligent glass surface defect detection device, which includes: a memory, a processor, and programs such as a defect detection algorithm for the image of the glass surface to be measured based on deep learning stored in the memory and executable on the processor. The programs such as the defect detection algorithm for the image of the glass surface to be measured based on deep learning are used to implement the steps of an intelligent glass surface defect detection method as described above.

[0042] In addition, to achieve the above object, the present invention also provides a computer program product, which includes programs such as a defect detection algorithm for the image of the glass surface to be measured based on deep learning. When the programs such as the defect detection algorithm for the image of the glass surface to be measured based on deep learning are executed by a processor, they implement an intelligent glass surface defect detection method as described above.

[0043] The advantages and effects of the present invention are:

[0044] The present invention proposes an intelligent glass surface defect detection method and system, which adopts a method of combining image processing with deep learning, and generates a larger number of glass surface defect image data sets through a GAN network, so as to solve the problem of too little training data in the traditional glass surface defect detection method based on deep learning during model training. The model can effectively learn the defect characteristics of the glass surface, accurately detect the surface defects of the glass to be tested and classify the defect types, thereby improving the detection accuracy and efficiency; at the same time, through non-contact optical acquisition and processing, mechanical damage to the surface of the glass to be tested is avoided, the integrity of the glass to be tested is guaranteed, and automatic, efficient and accurate image defect detection of the glass surface to be tested is realized, manual operation links are reduced, and labor costs are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 The present invention is a flow chart of a glass surface defect intelligent detection method.

[0047] Figure 2 The figure is a schematic diagram of the structure of a glass surface defect intelligent detection system of the present invention.

[0048] Figure 3 The present invention is a schematic block diagram of the structure of an electronic device for intelligently detecting defects on glass surfaces. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In one embodiment of the present invention, a glass surface defect intelligent detection method is provided. Figure 1 As shown, the following steps are included:

[0051] Step S10: Build a closed dark box glass surface image acquisition platform and set a ring-shaped uniform diffuse reflection light source, an industrial camera and a stage in it, place the glass to be tested on the stage, adjust the position and parameters of the industrial camera, and make the lens be located above the glass to be tested to shoot, so as to obtain the surface image of the glass to be tested.

[0052] Specifically, when the industrial camera captures the image of the glass surface to be measured in step S10, it automatically adjusts the focal length and aperture of the camera according to the clarity of the presented image, and captures from three angles of 0°, 45°, and 90°; in addition, the image of the glass surface to be measured captured by the industrial camera is a lossless image with a resolution of 4k, such as using image formats such as RAW, to ensure that the details of the glass surface to be measured are completely recorded.

[0053] Step S20: Perform data preprocessing on the obtained image of the glass surface to be measured, including filtering denoising and image enhancement.

[0054] Specifically, the steps of performing data preprocessing on the obtained image data of the glass surface to be measured in step S20 include:

[0055] Filtering denoising: Use Gaussian filtering to remove the noise in the image of the glass surface to be measured. Taking the pixel point (x, y) in the image of the glass surface to be measured as the center, select a certain neighborhood range for weighted calculation. The central pixel point of the selected neighborhood range is (x 0 , y 0 ), and the calculation formula is as shown in formula (1):

[0056]

[0057] where g(x, y) is the pixel value of a certain point (x, y) on the image of the glass surface to be measured after Gaussian filtering, σ is the standard deviation of the Gaussian distribution, (x - x 0 ) is the distance deviation of the pixel point (x, y) relative to the central pixel point (x 0 , y 0 ) in the horizontal direction, and (y - y 0 ) is the distance deviation of the pixel point (x, y) relative to the central pixel point (x 0 , y 0 ) in the vertical direction;

[0058] Image enhancement: After filtering denoising, perform image enhancement operations by changing the brightness and contrast of the image of the glass surface to be measured. The calculation formula is as shown in formula (2):

[0059] I new = a × I old + b (2)

[0060] where I old is the pixel value of the image of the glass surface to be measured before image enhancement, and I newis the pixel value of the image of the glass surface to be measured after image enhancement, a is the contrast adjustment coefficient, and b is the brightness adjustment coefficient. When a > 1, the contrast of the image of the glass surface to be measured is increased; when a < 1, the contrast of the image of the glass surface to be measured is decreased; when a = 1, the contrast of the image of the glass surface to be measured remains unchanged. The brightness of the image of the glass surface to be measured is adjusted by adjusting the value of b. When the value of b increases, the brightness of the image of the glass surface to be measured increases; when the value of b decreases, the brightness of the image of the glass surface to be measured decreases.

[0061] Step S30: Construct a GAN-based network model, and use the existing glass surface defect image dataset as the input of the model to generate a new glass surface defect image dataset.

[0062] Among them, constructing the GAN-based network model in step S30 includes a generator and a discriminator. By setting the network layer structure and learning parameters in the generator, and the network layer structure and network parameters in the discriminator, the GAN-based network model is adapted to the learning and generation of glass surface defect images.

[0063] Specifically, the steps of constructing the GAN-based network model in step S30 to generate a new glass surface defect image dataset include:

[0064] Data preparation: Collect glass surface defect images including various defect types, such as cracks, bubbles, and scratches, etc., and perform normalization processing. After the normalization processing is completed, scale and rotate the glass surface defect images to obtain glass surface defect images after normalization, glass surface defect images after scaling and normalization, and glass surface defect images after rotation and normalization.

[0065] Construct a generator: Construct a generator based on CNN. Remove the convolutional layer in CNN and add a transposed convolutional layer. The input of the generator is a random noise vector of a glass surface defect image. Through the transposed convolutional layer and the activation function ReLU, gradually convert the random noise vector of the glass surface defect image into an image similar to the real glass surface defect image. For example, use multiple transposed convolutional layers, and each layer gradually increases the resolution and the number of channels of the image, and finally generates an RGB image.

[0066] Construct a discriminator: Construct a discriminator based on CNN. In the discriminator, use a convolutional layer with multi-scale convolutional kernels. The input of the discriminator is the glass surface defect image output by the generator. Through the convolutional layer, pooling layer, and fully connected layer, and using the activation function LeakyReLU and the activation function Sigmoid, the discriminator outputs a probability value indicating the possibility that the glass surface defect image output by the generator is a real glass surface defect image.

[0067] Training process: Initialize the parameters in the generator and discriminator, including hyperparameters such as the learning rate and batch size. The learning rate is generally set between 0.0001 and 0.001, and the batch size is set according to the computing resources and the size of the glass surface defect image dataset, such as 32 or 64. After setting, alternate training is carried out in the order of training the discriminator first and then the generator, and repeat the cycle;

[0068] Evaluation and screening: After training, use the generator to generate glass surface defect images, evaluate the quality of the generated images through the peak signal-to-noise ratio and the structural similarity index, and form a new glass surface defect image dataset with the glass surface defect images generated by the generator that pass the evaluation and the collected real glass surface defect images, which is used for the training, validation, and testing of the deep learning-based glass surface image defect detection model for the glass to be tested in step S40.

[0069] Step S40: Build a deep learning-based glass surface image defect detection model for the glass to be tested, train the model using the generated new glass surface defect image dataset, and perform intelligent defect detection on the glass surface to be tested after training and output the detection results.

[0070] Specifically, in step S40, the deep learning-based glass surface image defect detection model for the glass to be tested is constructed by combining a convolutional neural network with an improved support vector machine. By introducing an adaptive kernel function, the parameters are dynamically adjusted according to the distribution of the input glass surface image features to be detected, enhancing the classification ability of the model for different types of defects and improving the accuracy and generalization of classification; after training, intelligent defect detection is performed on the glass surface to be tested, and the output detection results are that there are defects and no defects on the glass surface to be tested; when the output result is that there are defects on the glass surface, the detected glass surface defect category is output at the same time and the position where the defect appears is accurately marked in the glass surface image where the defect is detected.

[0071] Among them, the training steps of the deep learning-based glass surface image defect detection model for the glass to be tested include:

[0072] Dataset preparation: Divide the new glass surface defect image dataset obtained in step S30 into a training set, a validation set, and a test set according to the ratio of 7:2:1, and label the corresponding glass surface defect types in the training set image data for the model to learn the defect characteristics of different types of glass surface images to be tested;

[0073] Determine the loss function and optimizer: Use the cross-entropy loss function as the loss function of the model. The smaller the value of the loss function, the more accurate the detection result of the model. The optimizer uses the Adam optimization algorithm to adaptively adjust the learning rate and dynamically update the model parameters. The initial learning rate is set to 0.001 and is attenuated and adjusted according to the model training results and the number of training epochs;

[0074] Model construction: The entire network model includes an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. The input to the input layer is the training set with corresponding defect types labeled in the new glass surface defect image dataset. In the convolutional layers, ReLU is used as the activation function, and convolutional kernels of different scales are adopted in each convolutional layer. The pooling layer is set after the last convolutional layer and uses the max-pooling method. The fully connected layer is set after the pooling layer and is used to connect the pooling layer and the output layer. The fully connected layer transforms the feature map output by the pooling layer into a one-dimensional vector form and selects ReLU as the activation function. The output layer is set as the last layer of the entire glass surface image defect detection model to be tested. According to the number of corresponding defect categories of the glass surface image to be tested, after Softmax processing, it outputs the probability distribution of each defect category. Each probability distribution corresponds to the confidence level of the defect category. The confidence level threshold is set to 0.4. When the confidence level is less than 0.4, the output is no defect. When the confidence level is greater than or equal to 0.4, the output is defective and the defect category is also output. The defect category is the defect category with the highest probability value among the probability distributions of each defect category output. The confidence level threshold is dynamically adjusted according to the detection requirements of the model. When the detection requirements for the glass surface to be tested are strict, the confidence level threshold can be adjusted to between 0 and 0.4. When the detection requirements for the glass surface to be tested are loose, the confidence level threshold can be adjusted to between 0.4 and 1.

[0075] Model training and validation: After the model is constructed, adjust the model parameters according to the above settings. Use the training set with corresponding defect types labeled in the new glass surface defect image dataset allocated above as the input to train the glass surface image defect detection model to be tested. After each round of training, use the allocated validation set to validate the trained model.

[0076] Model evaluation: Set the cross-entropy loss function threshold. When validating the trained model with the allocated validation set after each round of training, when the value of the loss function is less than or equal to the set threshold, end the training and validation of the glass surface image defect detection model to be tested, and use the allocated test set for model evaluation.

[0077] Model optimization: According to the model evaluation results, when the model detection accuracy is less than 90%, adjust the model parameters and retrain until the model detection accuracy is greater than or equal to 90% to obtain the optimal model parameter combination and determine the corresponding version of the glass surface image defect detection model to be tested.

[0078] Among them, the defect categories output by the glass surface image defect detection model to be tested based on deep learning include cracks, bubbles, and scratches. Different defect types have different characteristics. The model determines the defect types existing in the input glass surface defect image to be tested by learning the characteristics of different glass surface image defect types.

[0079] Cracks are linear lines or gaps presented on the glass surface, generally showing straight lines, curves or irregular shapes. The directions and lengths of the cracks vary. There are significant differences in pixel values on both sides of the cracks, contrasting with the smoothness of the surrounding normal glass surface area. Whether there are crack defects is judged through the contrast feature of the image;

[0080] Bubbles are circular or approximately circular enclosed areas formed inside or on the surface of the glass, and are usually hollow or filled with gas inside. The areas with bubble defects usually appear as bright spots or dark spots, and there is usually an obvious bright-dark contrast at the edges of the bubbles, forming an annular contour. Whether there are bubble defects is judged through the brightness feature of the image;

[0081] Scratches are also linear lines or gaps presented on the glass surface. Different from cracks, the lines of scratches are more linear, with clear edges, having a certain width and length. The contrast and brightness with the surrounding normal glass surface are different. There is a clear bright-dark contrast at the scratches and the edges of the scratches, forming a bright or dark light band in the shape of a line. Whether there are scratch defects is judged through the contrast feature and brightness feature of the image.

[0082] In addition, the present invention also proposes an intelligent detection system for glass surface defects. Please refer to Figure 2 , and the intelligent detection system for glass surface defects includes:

[0083] Image acquisition module for the glass surface to be measured: It is used to build an enclosed dark-box type image acquisition platform for the glass surface and set an annular uniform diffuse light source, an industrial camera and a stage in it. Place the glass to be measured on the stage, adjust the position and parameters of the industrial camera, and make the lens located above the glass to be measured for shooting to obtain the image of the glass surface to be measured;

[0084] Data preprocessing module for the image of the glass surface to be measured: It is used to perform data preprocessing on the obtained image of the glass surface to be measured, including filtering and denoising and image enhancement;

[0085] Image dataset generation module for glass surface defects: It is used to build a network model based on GAN, take the existing glass surface defect image dataset as the input of the model, and generate a new glass surface defect image dataset;

[0086] Intelligent defect detection module for the glass surface to be measured: It is used to build a defect detection model for the image of the glass surface to be measured based on deep learning, train the model with the generated new glass surface defect image dataset, perform intelligent defect detection on the glass surface to be measured after training is completed, and output the detection result;

[0087] In the glass surface defect image dataset generation module, a GAN-based network model is constructed, including a generator and a discriminator. By setting the network layer structure and learning parameters in the generator, and the network layer structure and network parameters in the discriminator, the GAN-based network model is adapted to the learning and generation of glass surface defect images.

[0088] An intelligent glass surface defect detection system provided by the present application adopts an intelligent glass surface defect detection method in the above embodiment, and can solve the technical problems of insufficient training data, low detection accuracy and efficiency in the traditional glass surface image defect detection method. Compared with the prior art, the beneficial effects of the intelligent glass surface defect detection system provided by the present application are the same as those of the intelligent glass surface defect detection method provided by the above embodiment, and other technical features in the intelligent glass surface defect detection system are the same as those disclosed in the above embodiment method, which will not be elaborated herein.

[0089] The present application provides an intelligent glass surface defect detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute an intelligent glass surface defect detection method in Embodiment 1 above.

[0090] Next, refer to Figure 3 , which shows a schematic structural diagram of an intelligent glass surface defect detection device suitable for implementing the embodiments of the present application. An intelligent glass surface defect detection device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The intelligent glass surface defect detection device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present application.

[0091] Figure 3An intelligent glass surface defect detection device shown can include a processing system 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage system 1003 into a random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of an intelligent glass surface defect detection device are also stored. The processing system 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input system 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output system 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage system 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication system 1009. The communication system 1009 can allow an intelligent glass surface defect detection device to communicate with other devices wirelessly or wiredly to exchange data. Although an intelligent glass surface defect detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be alternatively implemented or had.

[0092] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication system, or installed from the storage system 1003, or installed from the ROM 1002. When the computer program is executed by the processing system 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0093] An intelligent glass surface defect detection device provided by the present application adopts an intelligent glass surface defect detection method in the above embodiments, and can solve the technical problems of insufficient training data, low detection accuracy and efficiency in traditional glass surface image defect detection methods. Compared with the prior art, the beneficial effects of an intelligent glass surface defect detection device provided by the present application are the same as those of an intelligent glass surface defect detection method provided by the above embodiments, and other technical features in the intelligent glass surface defect detection device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0094] Each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0095] This application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of an intelligent glass surface defect detection method as described above.

[0096] The computer program product provided by this application can solve the technical problems of insufficient training data, low detection accuracy and efficiency in traditional glass surface image defect detection methods. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the intelligent glass surface defect detection method provided by the above embodiments, and will not be elaborated here.

[0097] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. A glass surface defect intelligent detection method, characterized in that: The method comprises the following steps: Step S10: Building a closed dark box glass surface image acquisition platform and setting a ring-shaped uniform diffuse reflection light source, an industrial camera and a stage therein, placing the glass to be tested on the stage, adjusting the position and parameters of the industrial camera, and making the lens be located above the glass to be tested to take pictures, and obtaining an image of the glass surface to be tested; Step S20: performing data preprocessing on the obtained image of the glass surface to be tested, including filtering, denoising and image enhancement; Step S30: constructing a GAN-based network model, taking the existing glass surface defect image dataset as the input of the model, and generating a new glass surface defect image dataset; Step S40: constructing a glass surface image defect detection model based on deep learning, using the generated new glass surface defect image data set to train the model, performing intelligent defect detection on the glass surface to be tested after the training is completed, and outputting the detection results; In step S30, a GAN-based network model is constructed, including a generator and a discriminator. By setting the network layer structure and learning parameters in the generator and the network layer structure and network parameters in the discriminator, the GAN-based network model is adapted to the learning and generation of glass surface defect images.

2. The glass surface defect intelligent detection method according to claim 1, characterized in that: In step S10, when the industrial camera captures the image of the glass surface to be tested, the camera's focal length and aperture are automatically adjusted according to the clarity of the presented image, and the image is captured from three angles: 0°, 45°, and 90°. The image of the glass surface to be tested captured by the industrial camera is a lossless image with a resolution of 4k.

3. The glass surface defect intelligent detection method according to claim 1, characterized in that: The step of performing data preprocessing on the obtained image data of the glass surface to be tested in step S20 includes: Filtering denoising: Gaussian filtering is used to remove noise in the glass surface image to be tested. Taking the pixel point (x, y) in the glass surface image to be tested as the center, a certain neighborhood range is selected for weighted calculation. The central pixel point of the selected neighborhood range is (x0, y0). The calculation formula is shown in formula (1): Among them, g(x,y) is the pixel value of a certain point (x,y) on the image of the glass surface to be tested after Gaussian filtering, σ is the standard deviation of the Gaussian distribution, (x-x0) is the distance deviation of the pixel point (x,y) relative to the central pixel point (x0,y0) of the neighborhood in the horizontal direction, and (y-y0) is the distance deviation of the pixel point (x,y) relative to the central pixel point (x0,y0) of the neighborhood in the vertical direction; Image enhancement: After filtering and denoising, the image enhancement operation is performed by changing the brightness and contrast of the image of the glass surface to be tested. The calculation formula is shown in formula (2): I new =a×I old +b (2) Among them, I old is the pixel value of the glass surface image before image enhancement, I new is the pixel value of the glass surface image to be tested after image enhancement, a is the contrast adjustment coefficient, b is the brightness adjustment coefficient, when a>1, the contrast of the glass surface image to be tested is increased, when a<1, the contrast of the glass surface image to be tested is reduced, and when a=1, the contrast of the glass surface image to be tested remains unchanged; the brightness of the glass surface image to be tested is adjusted by adjusting the size of b, when the value of b increases, the brightness of the glass surface image to be tested increases, and when the value of b decreases, the brightness of the glass surface image to be tested decreases.

4. The glass surface defect intelligent detection method according to claim 1, characterized in that: The step of constructing a GAN-based network model in step S30 to generate a new glass surface defect image dataset includes: Data preparation: glass surface defect images of various defect types are collected and normalized. After the normalization, the glass surface defect images are scaled and rotated to obtain normalized glass surface defect images, normalized and scaled glass surface defect images, and normalized and rotated glass surface defect images; Constructing a generator: The generator is constructed based on CNN. The convolutional layer in CNN is removed and a deconvolutional layer is added. The input of the generator is a random noise vector of a glass surface defect image. Through the deconvolutional layer and the activation function ReLU, the random noise vector of the glass surface defect image is gradually converted into an image similar to the real glass surface defect image. Construct the discriminator: Construct the discriminator based on CNN. The discriminator uses a convolution layer with multi-scale convolution kernels. The input of the discriminator is the glass surface defect image output by the generator. Through the convolution layer, pooling layer and fully connected layer, using the activation function LeakyReLU and the activation function Sigmoid, the discriminator outputs a probability value, indicating the possibility that the glass surface defect image output by the generator is a real glass surface defect image; Training process: Initialize the parameters in the generator and discriminator, including the learning rate and batch size. The learning rate is set between 0.0001 and 0.

001. The batch size is set to 32 or 64 according to the computing resources and the size of the glass surface defect image dataset. After the setting is completed, the training is performed alternately in the order of training the discriminator first and then the generator, and the cycle is repeated; Evaluation and screening: After the training is completed, the generator is used to generate glass surface defect images, and the quality of the generated images is evaluated by the peak signal-to-noise ratio and the structural similarity index. The glass surface defect images generated by the generator that pass the evaluation and the collected real glass surface defect images are combined into a new glass surface defect image dataset, which is used for the training, verification and testing of the glass surface image defect detection model to be tested based on deep learning in step S40.

5. The glass surface defect intelligent detection method according to claim 1, characterized in that: In the step S40, the glass surface image defect detection model to be tested based on deep learning is constructed by using a convolutional neural network combined with an improved support vector machine, and by introducing an adaptive kernel function, the parameters are dynamically adjusted according to the distribution of the input glass surface image features to be tested; After the training is completed, intelligent defect detection is performed on the glass surface to be tested, and the output detection results are whether the glass surface to be tested has defects or not. When the output result is that the glass surface has defects, the detected glass surface defect category is output at the same time, and the location of the defect is accurately marked in the glass surface image where the defect is detected.

6. The glass surface defect intelligent detection method according to claim 5, characterized in that: The training steps of the deep learning-based glass surface image defect detection model include: Dataset preparation: The new glass surface defect image dataset obtained in step S30 is divided into a training set, a validation set, and a test set in a ratio of 7:2:1, and the corresponding glass surface defect types are marked in the training set image data for the model to learn different types of glass surface image defect features to be tested; Determine the loss function and optimizer: Use the cross entropy loss function as the loss function of the model. The optimizer uses the Adam optimization algorithm to adaptively adjust the learning rate and dynamically update the model parameters. The initial learning rate is set to 0.001 and is adjusted according to the model training results and the number of training rounds. Model construction: The entire network model includes input layer, convolution layer, pooling layer, fully connected layer and output layer; the input of the input layer is the training set with the corresponding defect type annotated in the new glass surface defect image dataset; ReLU is used as the activation function in the convolution layer, and each convolution layer uses convolution kernels of different scales; the pooling layer is set after the last convolution layer, and the maximum pooling method is used; the fully connected layer is set after the pooling layer to connect the pooling layer and the output layer. The fully connected layer converts the feature map output by the pooling layer into a one-dimensional vector form, and ReLU is used as the activation function; the output layer is set It is placed in the last layer of the entire defect detection model of the glass surface image to be tested. According to the corresponding number of defect categories of the glass surface image to be tested, the probability distribution of each defect category is output after Softmax processing. Each probability distribution corresponds to the confidence of the defect category. The confidence threshold is set to 0.

4. When the confidence is less than 0.4, the output is no defect. When the confidence is greater than or equal to 0.4, the output is defective and the defect category is output at the same time. The defect category is the defect category with the highest probability value in the probability distribution of each defect category output; the confidence threshold is dynamically adjusted according to the detection requirements of the model; Model training and verification: After the model is built, adjust the model parameters according to the above settings, use the training set assigned above and annotated with the corresponding defect types in the new glass surface defect image dataset as input to train the glass surface image defect detection model to be tested, and use the assigned verification set to verify the trained model after each training round; Model evaluation: Set the cross entropy loss function threshold. When the value of the loss function is less than or equal to the set threshold after each training round, the training and verification of the glass surface image defect detection model to be tested is terminated, and the model is evaluated using the assigned test set. Model optimization: According to the model evaluation results, when the model detection accuracy is less than 90%, adjust the model parameters and retrain until the model detection accuracy is greater than or equal to 90%. Obtain the optimal model parameter combination and determine the corresponding glass surface image defect detection model version to be tested.

7. The glass surface defect intelligent detection method according to claim 5, characterized in that: The categories of defects output in the deep learning-based glass surface image defect detection model include cracks, bubbles and scratches. Different defect types have different corresponding features. The model determines the defect type existing in the input glass surface defect image to be tested by learning the features of different glass surface image defect types.

8. A glass surface defect intelligent detection system, characterized in that: The glass surface defect intelligent detection system comprises: The image acquisition module of the glass surface to be tested is used to build a closed dark box glass surface image acquisition platform and set a ring-shaped uniform diffuse reflection light source, an industrial camera and a stage in it. The glass to be tested is placed on the stage, and the position and parameters of the industrial camera are adjusted so that the lens is located above the glass to be tested to take pictures and obtain the image of the glass surface to be tested. The data preprocessing module of the image of the glass surface to be tested is used to perform data preprocessing on the obtained image of the glass surface to be tested, including filtering, denoising and image enhancement; Glass surface defect image dataset generation module: used to build a GAN-based network model, taking the existing glass surface defect image dataset as the input of the model to generate a new glass surface defect image dataset; Intelligent defect detection module for the glass surface to be tested: used to build an image defect detection model for the glass surface to be tested based on deep learning, use the generated new glass surface defect image data set to train the model, perform intelligent defect detection on the glass surface to be tested after training, and output the detection results; A GAN-based network model is constructed in the glass surface defect image dataset generation module, including a generator and a discriminator. By setting the network layer structure and learning parameters in the generator and the network layer structure and network parameters in the discriminator, the GAN-based network model is adapted to the learning and generation of glass surface defect images.

9. An intelligent glass surface defect detection device, characterized in that: The glass surface defect intelligent detection device comprises: A memory, a processor, and a glass surface defect intelligent detection program stored in the memory and executable on the processor, wherein the glass surface defect intelligent detection program, when executed by the processor, implements a glass surface defect intelligent detection method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a glass surface defect intelligent detection program, and when the glass surface defect intelligent detection program is executed by a processor, an intelligent glass surface defect detection method as described in any one of claims 1 to 7 is implemented.

Images