Glass surface defect classification method

Abstract

The invention discloses a glass surface defect classification method. The method comprises the following steps: acquiring and processing a glass surface image; reconstructing the image data of the glass surface; extracting features; training an initial model; an active learning strategy; a mixed learning strategy; retraining the model; classifying defects; iteratively circulating; the deep learning technology is introduced into the glass surface defect detection model provided by the invention, so that different types of defects can be identified more accurately, the area where the defects appear can be accurately positioned, compared with traditional manual detection, the efficiency and accuracy are higher, and by introducing an active learning algorithm to assist sample labeling, the detection accuracy is higher. According to the method, the most valuable sample can be effectively selected, the dependence of the model on data is reduced, the performance of the model is improved, meanwhile, the labeling cost is reduced, after the technology is successfully implemented, intelligent detection of the glass surface defects is realized, the labor consumption in the detection stage is reduced, the detection efficiency is improved, and the dependence of the model on the data is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and machine learning, and in particular to a method for classifying defects on a glass surface. Background Art

[0002] The glass industry is an important basic material industry and an important part of the building materials industry. It plays an irreplaceable role in the process of national economic and social development. It is mainly used in the fields of construction, transportation, new energy and electronic information. In the process of glass production, due to the influence of various factors, defects such as bubbles, stones, tin ash, coating scratches and pinholes are prone to appear on the glass surface, which will seriously affect the quality and performance of glass products. The traditional glass detection method relies on the human eye to judge various problems on the glass surface, which has great limitations. For example, the human eye is not sensitive to tiny defects, and there are risks of false detection and missed detection. The development of deep learning technology provides a new solution for glass surface defect detection. However, since the training of deep learning models often requires a large number of labeled data sets, and the industrial production process strictly controls the defective rate, defects are rare in products. Therefore, compared with normal samples that are easier to obtain, the number of defective samples is very small, resulting in the deep learning model facing problems such as insufficient labeled data and difficulty in coping with the changing types of glass surface defects, making the detection accuracy and efficiency unable to meet the actual production needs.

[0003] The on-line detection device for defects on the coated surface of glass with publication number CN114858805A comprises a line light source module, an imaging module and a terminal data processing module connected to the output end of the imaging module, wherein the width of the light emitted by the line light source module is not greater than the thickness of the glass, the light emitted by the line light source module forms reflected light after being reflected by the glass, the imaging module can receive the reflected light reflected by the coated surface on the upper surface of the glass, the reflected light on the lower surface is set outside the receiving range of the imaging module, and the output end of the imaging module determines whether there is a defect on the coated surface of the glass based on whether the imaging module can receive the reflected light of the coated surface.

[0004] Disadvantages of prior art 1: 1. Low sensitivity: Since the width of the light emitted by the line light source module is not greater than the thickness of the glass, the detection sensitivity may be low. The smaller light source width may limit the ability to detect subtle defects, especially very small defects may not be accurately detected.

[0005] 2. Limited basis for defect judgment: Judging whether there are defects on the coated surface of the glass is based on whether the imaging module can receive the reflected light from the coated surface. This judgment method may be relatively simple and limited. It may not be possible to accurately classify and quantitatively evaluate different types of defects, limiting the application ability of the device in complex defect detection. Summary of the invention

[0006] The purpose of the present invention is to provide a glass surface defect classification method, aiming to solve the problem of limited number of samples in glass surface defect classification. The method uses deep learning technology to replace manual defect detection to improve detection speed and accuracy, and at the same time uses active learning technology to screen out the most valuable samples for annotation, thereby improving the classification accuracy of the deep learning model under limited data, thereby achieving the expected effect with a small amount of data and reducing the annotation cost.

[0007] To achieve the above object, the present invention provides the following technical solutions: A method for classifying glass surface defects comprises the following steps: S1: glass surface image acquisition and processing; S2: reconstructing the image data of the glass surface; S3: Feature extraction; S4: initial model training; S5: Active learning strategies; S6: Blended learning strategies; S7: Model retraining; S8: Defect classification; S9: Iteration loop.

[0008] Preferably, a high-precision industrial camera is used in S1 to capture images of the glass surface. By adjusting the height and focal length of the industrial camera, clear and accurate images of glass surface defects can be obtained in different states.

[0009] Preferably, in S2, an algorithm is used to reduce and eliminate noise in the image to make the glass image clearer, and a color image is converted into a grayscale image through grayscale processing, thereby reducing the amount of data while retaining important structural information of the image. Through these image preprocessing methods, the deep learning model can accurately identify and classify defects on the glass surface.

[0010] Preferably, S3 combines a deep convolutional neural network and a self-attention mechanism as a feature extractor to extract fine features of the defects from the repaired image.

[0011] Preferably, S4 randomly selects a small number of samples from the collected images for annotation, establishes an initial training set, and trains an initial model with the annotated images.

[0012] Preferably, the S5 introduces an active learning algorithm, which evaluates the uncertainty of the image by mixing the labeled images and the unlabeled images to find the unlabeled samples containing new features, and evaluates the similarity between the unlabeled images and the labeled images by calculating the distance between them. The images with lower similarity are more likely to be selected, and high-value images are selected by comprehensively considering uncertainty and diversity.

[0013] Preferably, in the glass surface defect classification in S6, the hybrid learning strategy can be applied in the model training and classification process, combining the results of different feature extraction methods and the prediction results of multiple classifiers to improve the accuracy and robustness of classification.

[0014] Preferably, S7 adds the newly annotated image to the training set and retrains the deep learning model. The model can learn new features from the re-added images, thereby improving the ability to classify defects.

[0015] Preferably, the S8 module constructs an EfficientFormer classifier, which, through its innovative MetaBlock structure, combined with deep convolution and self-attention mechanisms, extracts fine feature representations from images and can quickly and accurately identify various defects on the glass surface.

[0016] Preferably, the step S9 is repeated, and the model continuously learns new features from the re-added images to improve the classification capability of the model until the preset performance index is reached or the demand is met.

[0017] In summary, due to the adoption of the above technology, the beneficial effects of the present invention are: The glass surface defect detection model proposed in the present invention introduces deep learning technology, which can more accurately identify different types of defects and accurately locate the areas where defects occur. Compared with traditional manual inspection, it has higher efficiency and accuracy.

[0018] 2. The present invention proposes a glass surface defect classification method based on feature hybrid active learning. By introducing an active learning algorithm to assist sample labeling, the most valuable samples can be effectively selected, the model's dependence on data can be reduced, the model's performance can be improved, and the labeling cost can be reduced.

[0019] 3. After the successful implementation of this technology, glass surface defects will be intelligently detected, reducing labor consumption in the detection stage, improving detection efficiency, reducing the model's dependence on data, reducing the data required for deep learning training models, and reducing labeling costs.

[0020] 4. The detection process and methods adopted by this technology can be applied to defect detection of various industrial products, such as metal, plastic and other industrial manufacturing industries, bringing more efficient and accurate defect detection solutions to these industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention, so that other features, purposes and advantages of the present invention become more obvious. The accompanying drawings of the exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings: Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0023] In the description of the present invention, it is necessary to understand that the terms indicating orientation or positional relationship are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0024] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances of the specification.

[0025] The present invention provides Figure 1 A glass surface defect classification method shown includes the following steps: Step 1: Glass surface image acquisition and processing: Use a high-precision industrial camera to capture images of the glass surface. By adjusting the height and focal length of the industrial camera, clear and accurate images of glass surface defects can be obtained under different conditions.

[0026] Image acquisition: Image acquisition is the first step in glass surface quality inspection, and its quality has a direct impact on subsequent processing results. In order to ensure the consistency and reliability of image quality, glass manufacturers usually use high-resolution cameras to shoot glass and strictly control the shooting environment, such as lighting conditions, shooting angles, etc.

[0027] Image preprocessing: Since there may be various defects on the glass surface, such as bubbles, scratches, flaws, etc., these defects will affect the subsequent feature extraction and defect recognition. Therefore, after image acquisition, image preprocessing is required to eliminate the impact of these defects on subsequent processing. Common image preprocessing methods include noise reduction, filtering, and image enhancement, which can convert images into grayscale or binary images that are more suitable for subsequent processing.

[0028] Step 2: Reconstruct the image data of the glass surface: reduce or eliminate the noise in the image through algorithms to make the glass image clearer; convert the color image into a grayscale image through grayscale processing, reduce the amount of data while retaining important structural information of the image. Through these image preprocessing methods, the deep learning model can more accurately identify and classify defects on the glass surface.

[0029] 1. It is necessary to ensure that the image data quality of the glass surface is high enough. This includes image clarity, contrast, and resolution. If the original image data quality is poor, it may affect the subsequent reconstruction effect. Therefore, when collecting image data, high-quality camera equipment and appropriate lighting conditions should be used as much as possible.

[0030] 2. You can choose an appropriate image reconstruction method. Common image reconstruction methods include interpolation, Fourier transform reconstruction, compressed sensing reconstruction, and deep learning reconstruction. These methods have their own advantages and disadvantages, and the appropriate method should be selected according to the specific application scenario and needs. For example, interpolation is simple and fast, but may not be able to restore high-quality details; while deep learning reconstruction can learn and restore more complex image structures, but requires a lot of training data and computing resources.

[0031] 3. During the reconstruction process, the characteristics of the glass surface also need to be considered. The reflection, refraction and transparency of the glass surface may have certain effects on image reconstruction. Therefore, when selecting reconstruction methods and parameters, these characteristics should be fully considered, and special treatment methods for glass surfaces should be tried. For example, advanced coating technology can be used to reduce the effects of reflection and refraction, or the degree of image deformation can be reduced by optimizing parameters such as glass thickness and spacing.

[0032] 4. Some advanced image processing and computer vision technologies can also be used to assist in image reconstruction. For example, stereo vision reconstruction technology can be used to restore the three-dimensional shape of the glass surface, or image enhancement and restoration technology can be used to improve the quality and clarity of the image.

[0033] It is also very important to evaluate and verify the quality of the reconstructed image. The reconstruction effect can be evaluated by comparing it with the original image and observing whether the details and structure of the reconstructed image are clear and reasonable. If problems or deficiencies are found, the reconstruction method and parameters should be adjusted in time to obtain better reconstruction results.

[0034] Step 3. Feature extraction: Combining deep convolutional neural networks and self-attention mechanisms as feature extractors Feature extraction in glass surface defect classification is a key step, which involves effectively identifying and extracting the features of defects from glass images for subsequent classification and identification. The following is a detailed explanation of the process: 1. Feature extraction of glass surface defects is mainly based on image processing and machine vision technology. These technologies can capture and analyze subtle changes on the glass surface to extract the features of the defects. The basic principle of feature extraction is to select features that can effectively distinguish defects from normal areas based on the characteristics of glass defects.

[0035] 2. Geometric features are an important aspect in the feature extraction process. By measuring the geometric parameters of the defect, such as shape, size, and position, a feature vector describing the defect can be constructed. In addition, the invariant moment feature is also one of the commonly used features. It can reflect the invariance of the image under affine transformation and has a good effect on identifying defects at different angles and scales.

[0036] 3. In addition to geometric features and invariant moment features, other specific features can be extracted according to the specific defect type. For example, for crack defects on the glass surface, the crack direction, length, width and other features can be extracted; for bubble defects, the bubble roundness, area and other features can be extracted. The extraction of these features helps to more accurately identify different types of glass surface defects.

[0037] 4. In the process of feature extraction, the quality and processing methods of the image also need to be considered. Factors such as lighting conditions, camera parameters, and image format will affect the quality of the image, and thus the accuracy of feature extraction. Therefore, when acquiring images, it is necessary to ensure that the lighting conditions are appropriate, the camera parameters are properly adjusted, and the appropriate image format is selected for storage and processing, so as to extract the fine features of the defects from the repaired images.

[0038] Step 3: Initial model training: Randomly select a small number of samples from the collected images for annotation, establish an initial training set, and use these annotated images to train the initial model.

[0039] Initial model training in glass surface defect classification is a key step in building an efficient defect detection system. The following is a detailed explanation of this process: ‌1. Data Preparation‌ ‌Dataset Collection‌: First, image data containing various glass surface defects need to be collected. These data should cover different types of defects, such as pinholes, scratches, dirt, wrinkles, etc., to ensure the comprehensiveness and accuracy of the model. The size and diversity of the dataset are crucial to the performance of the model.

[0040] ‌Data Annotation‌: Annotate each image with its corresponding defect type label, such as "crack", "stain", etc., to facilitate supervised learning during training‌.

[0041] ‌2. Model selection and construction‌ ‌Select a basic network‌: According to the task requirements, select a suitable basic network for model building. For example, in the classification of aluminum profile surface defects, the ResNet network is widely used because of its "simple and practical" characteristics. For glass surface defect classification, you can also consider using this type of pre-trained model as a starting point.

[0042] ‌Model structure design‌: Based on the basic network, design a specific model structure, including input layer, convolution layer, pooling layer, fully connected layer, etc. At the same time, adjust network parameters such as convolution kernel size, step size, padding method, etc. according to the characteristics of the dataset and task requirements.

[0043] ‌3. Model training‌ ‌Data preprocessing‌: Before training, the data is preprocessed, including steps such as image enhancement (such as rotation, flipping, scaling, etc.), normalization (normalizing pixel values ​​to the range of 0-1) and data cleaning (removing invalid or erroneous data) to improve data quality and the generalization ability of the model‌.

[0044] ‌Loss function selection‌: Select an appropriate loss function according to the task type (classification task), such as the cross entropy loss function. The loss function is used to measure the difference between the model prediction result and the true label, and guide the optimization of the model parameters‌.

[0045] ‌Training process‌: Input the preprocessed data into the model, calculate the prediction results through forward propagation, and then update the model parameters according to the loss function through back propagation. Repeat this process until the performance of the model on the training set reaches a satisfactory level.

[0046] ‌4. Model evaluation and optimization‌ ‌Performance evaluation‌: During or after training, use the validation set or test set to evaluate the performance of the model, including indicators such as accuracy, recall, and F1 score. This helps to understand the performance of the model on unknown data and serves as a basis for model optimization.

[0047] ‌Model optimization‌: Optimize the model based on the performance evaluation results. For example, adjust the network structure, increase or decrease the number of network layers, change the learning rate and other strategies to improve the performance and generalization ability of the model.

[0048] Step 4, active learning strategy: Introduce active learning algorithm, by mixing labeled images and unlabeled images to find unlabeled samples with new features, so as to evaluate the uncertainty of the image; by calculating the distance between unlabeled images and labeled images, the similarity between them is evaluated. The images with lower similarity are more likely to be selected, and high-value images are selected by comprehensively considering uncertainty and diversity.

[0049] The active learning strategy in glass surface defect classification is not a straightforward concept, but it can be explained by combining active learning with glass surface defect classification.

[0050] Active learning strategy in glass surface defect classification mainly refers to intelligently selecting the most informative samples for annotation and training to improve the accuracy and efficiency of the classification model. The core of this strategy is to use machine learning algorithms, especially deep learning technology, to automatically identify and classify glass surface defects, and to achieve accurate identification of defects through continuous iteration and optimization of the model.

[0051] Specifically, active learning strategies may include the following steps: 1. Sample collection and preprocessing: First, a large number of glass surface defect image samples are collected and preprocessed, such as image enhancement and normalization, to improve image quality and model training effect.

[0052] 2. Initial model training: Use the preprocessed samples to train an initial deep learning neural network model for preliminary classification of glass surface defects.

[0053] 3. Uncertain sample selection: Based on the preliminary classification, active learning strategies are used to select samples with high model classification uncertainty, that is, samples that are difficult for the model to accurately judge. These samples are usually located near the classification boundary and play a key role in improving the model's classification ability.

[0054] 4. ‌Expert Labeling and Model Update‌: Submit the selected uncertain samples to experts for labeling to obtain accurate label information. Then, use these newly labeled samples to update the deep learning neural network model to further improve the accuracy and generalization ability of the model.

[0055] Step 5: Hybrid learning strategy: There is no specific definition or standardized method for hybrid learning strategy in glass surface defect classification, but it can be inferred from the characteristics of glass defects and existing detection technologies that may be involved. The following is an understanding of hybrid learning strategy in glass surface defect classification: The hybrid learning strategy in glass surface defect classification mainly combines multiple detection technologies and classification algorithms to improve the accuracy and efficiency of defect identification.

[0056] ‌Integration of multiple detection technologies‌: There are many types of glass surface defects, including scratches, cracks, spots, bumps, color anomalies, etc. Different defect types may require different detection technologies to effectively identify them.

[0057] For example, scratches and cracks might be detected through edge detection and morphological processing, while spots and color anomalies might require techniques such as threshold segmentation and color space conversion‌.

[0058] Hybrid learning strategies combine these different detection technologies, select the most appropriate detection method based on the characteristics of the defects, or integrate multiple detection methods to improve the accuracy and comprehensiveness of detection.

[0059] Optimization and combination of classification algorithms: In the defect classification stage, the hybrid learning strategy uses a variety of classification algorithms, such as machine learning algorithms, deep learning algorithms, etc., to accurately classify the detected defects.

[0060] Different classification algorithms may have different advantages and applicable scenarios. For example, deep learning algorithms perform well in processing complex images and pattern recognition, while machine learning algorithms may have more advantages in processing large-scale data sets and feature extraction.

[0061] By optimizing and combining these classification algorithms, their respective advantages can be fully utilized to improve the accuracy and efficiency of defect classification.

[0062] ‌Continuous learning and updating‌: The types and characteristics of glass surface defects may change with changes in production processes, raw materials and other factors.

[0063] Hybrid learning strategies need to include mechanisms for continuous learning and updating to adapt to these changes. For example, new defect samples can be collected regularly to train and update detection techniques and classification algorithms to ensure that they can accurately identify emerging defect types.

[0064] Step 6: Model retraining: Add the newly annotated images to the training set and retrain the deep learning model so that the model can learn new features from the re-added images, thereby improving the ability to classify defects.

[0065] 1. Data preparation: Collect new glass surface defect image data, ensuring that the data covers various defect types (such as cracks, bubbles, stains, pinholes, scratches, etc.)‌.

[0066] The data is labeled, including information such as the location, type, and size of the defects, so that it can be used for model training.

[0067] 2. Model selection: Select the appropriate deep learning model according to the task requirements. For example, in glass surface defect detection, commonly used models include convolutional neural networks (CNNs), such as ResNet18 and ResNet101.

[0068] If you have a previously trained model, you can consider fine-tuning it to save training time and resources.

[0069] 3. Model training: Use the prepared data set to train the model. During the training process, you need to pay attention to evaluation indicators such as the model's accuracy and recall rate to ensure that the model can accurately identify various defects.

[0070] Data enhancement techniques (such as rotation, scaling, flipping, etc.) can be used to increase data diversity and improve the generalization ability of the model.

[0071] 4. Model evaluation and optimization: After training is completed, use the validation set or test set to evaluate the model to check whether the performance of the model meets the requirements.

[0072] If the model performance is poor, it can be improved by adjusting the model structure, optimizing hyperparameters, increasing the amount of data, etc.

[0073] 5. Defective specification control: After the model can accurately identify defects, it is also necessary to control the specifications of the defects. For example, the extracted parameters such as length, width, aspect ratio, contrast, etc. can be used to further classify and evaluate the defects.

[0074] 6. Deployment and Update: Deploy the trained model to the actual glass surface defect detection system for real-time defect detection.

[0075] As the production process progresses, new defect types or defect forms may continue to appear. Therefore, new data needs to be collected regularly and the model needs to be updated and retrained to maintain its accuracy and effectiveness.

[0076] Step 7: Defect classification: Build the EfficientFormer classifier. Through its innovative MetaBlock structure, combined with deep convolution and self-attention mechanisms, it extracts fine feature representations from the image, enabling it to quickly and accurately identify various defects on the glass surface.

[0077] Step 8. Iteration loop: Repeat steps 4, 5, 6, and 7 to allow the model to continuously learn new features from the added images to improve the model classification ability until the preset performance indicators are reached or the requirements are met.

Claims

1. A method for classifying glass surface defects, characterized in that: The following steps are involved: S1: glass surface image acquisition and processing; S2: reconstructing the image data of the glass surface; S3: Feature extraction; S4: initial model training; S5: Active learning strategies; S6: Blended learning strategies; S7: Model retraining; S8: Defect classification; S9: Iteration loop.

2. A glass surface defect classification method according to claim 1, characterized in that: The S1 uses a high-precision industrial camera to capture images of the glass surface. By adjusting the height and focal length of the industrial camera, clear and accurate images of glass surface defects can be obtained in different states.

3. A glass surface defect classification method according to claim 1, characterized in that: In S2, an algorithm is used to reduce and eliminate noise in the image, making the glass image clearer. The color image is converted into a grayscale image through grayscale processing, reducing the amount of data while retaining important structural information of the image. Through these image preprocessing methods, the deep learning model can accurately identify and classify defects on the glass surface.

4. A glass surface defect classification method according to claim 1, characterized in that: The S3 combines a deep convolutional neural network and a self-attention mechanism as a feature extractor to extract fine features of defects from the repaired image.

5. A glass surface defect classification method according to claim 1, characterized in that: The S4 randomly selects a small number of samples from the collected images for annotation, establishes an initial training set, and trains an initial model with the annotated images.

6. A glass surface defect classification method according to claim 1, characterized in that: The S5 introduces an active learning algorithm, which evaluates the uncertainty of the image by mixing labeled images and unlabeled images to find unlabeled samples containing new features. The similarity between unlabeled images and labeled images is evaluated by calculating the distance between them. Images with lower similarity are more likely to be selected, and high-value images are selected by combining uncertainty and diversity.

7. A glass surface defect classification method according to claim 1, characterized in that: In the glass surface defect classification of S6, the hybrid learning strategy can be applied to the model training and classification process, combining the results of different feature extraction methods and the prediction results of multiple classifiers to improve the accuracy and robustness of classification.

8. A glass surface defect classification method according to claim 1, characterized in that: The S7 adds the newly annotated image to the training set and retrains the deep learning model. The model can learn new features from the re-added images, thereby improving the ability to classify defects.

9. A glass surface defect classification method according to claim 1, characterized in that: The S8 module constructs the EfficientFormer classifier, which, through its innovative MetaBlock structure, combined with deep convolution and self-attention mechanisms, extracts fine feature representations from images and can quickly and accurately identify various defects on the glass surface.

10. The method for classifying glass surface defects according to claim 1, characterized in that: The S9 step is repeated, and the model continuously learns new features from the added images to improve the model classification capability until the preset performance index is reached or the demand is met.

Images