Screen flaw classification method and device based on deep learning image classification model, and storage medium

Through the screen defect classification method for deep learning image classification model, multiple binary classification models are used to detect screen defects and perform conflict detection, the uncertainty and classification error problems of deep learning models are solved, and detection accuracy and productivity are improved.

CN119091188APending Publication Date: 2024-12-06SUZHOU GACII OPTOELECTRONICTECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411067429.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The black box characteristics of existing deep learning image classification models lead to uncertainty in detection results and classification errors, especially when the detection accuracy rate is low after product batch replacement, requiring frequent manual re-checking and model retraining, affecting production efficiency.

Method used

The screen defect classification method based on the deep learning image classification model is adopted. By obtaining screen defect sample data, multiple sample data sets are divided, the corresponding binary classification model is trained, and the screen image is detected. If the classification results conflict, it will be sent to manual visual inspection. Otherwise, the result shall prevail, and the conflict ratio shall be determined whether to re-detect or retrain the model.

Benefits of technology

It improves the accuracy and reliability of screen defect category detection, reduces the number of manual visual inspections, enhances the generalization ability and flexibility of the model, and adapts to the stability of classification results when product replacement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091188A_ABST
    Figure CN119091188A_ABST
Patent Text Reader

Abstract

The invention relates to a screen flaw classification method based on a deep learning image classification model, and the method comprises the steps: obtaining screen flaw sample data, and dividing the data into a plurality of sample data sets according to the flaw types; training a dichotomy model corresponding to the defect category by using each sample data set; respectively inputting all the to-be-detected screen images of the current batch into each trained dichotomy model, and obtaining a plurality of classification results of each to-be-detected screen image; for each to-be-detected screen image, if only one of all the classification results is positive, taking the defect category represented by the classification result as the screen defect category of the to-be-detected screen image; and otherwise, judging that the to-be-detected screen image is a conflict image, sending the conflict image to manual visual inspection, and obtaining the screen defect category of the to-be-detected screen image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image classification technology, and in particular to a screen defect classification method, device and computer-readable storage medium based on a deep learning image classification model. Background Art

[0002] A deep learning image classification model is an advanced neural network architecture that usually uses deep learning algorithms, such as convolutional neural networks (CNNs), to capture complex features and patterns in images. Through training, a model that can judge an image as a specific category can be obtained. This model can identify key features in an image and classify the image into one of the predefined categories based on these features. During the training process, the deep learning image classification model uses the back-propagation algorithm to optimize the parameters of the model. At the beginning of training, the model receives a large number of labeled images as input and calculates the predicted probability of each category through forward propagation. Then, the model calculates the error between the predicted result and the true label, and propagates the error back to each layer of the model through the back-propagation algorithm. During the back-propagation process, the model adjusts the weights and biases of each layer according to the error signal to minimize the prediction error. This process is repeated until the performance of the model reaches the preset standard or the termination condition is met.

[0003] Deep learning image classification models usually consist of multiple layers, including an input layer, multiple hidden layers, and an output layer. The input layer receives the original image data, the hidden layer extracts features from the image through operations such as convolution and pooling, and the output layer calculates the predicted probability of each category based on the extracted features. Although deep learning image classification models have shown excellent performance in the field of image recognition, their black box characteristics make it difficult to fully analyze their internal working principles. This model is regarded as a "black box" because it can process input images and output classification results, but the internal decision-making process is not transparent. In addition, when the deep learning image classification model is retrained, the detection results of each category may change significantly, which increases the uncertainty of the results. During the training process, the model optimizes its parameters by learning from a large number of labeled images and tries to capture the complex features and patterns in the image. However, due to the black box characteristics of the model, it is impossible to know exactly how it makes classification decisions based on these features. This means that even if the model performs well on the training set, its accuracy and reliability in practical applications cannot be fully guaranteed.

[0004] In addition, when retraining a deep learning image classification model, the detection results of each category may change significantly due to factors such as differences in training data, randomness of parameter initialization, and the choice of optimization algorithm. This uncertainty makes it possible that the retrained model may not maintain the original performance and may even make unexpected classification errors.

[0005] Therefore, when using the existing deep learning image classification model to classify screen defects, due to the high consistency of features of consecutive batches of products during the production process, but the features of different batches or product changes, the product features will change, but the detection model will always give a classification result, but the reliability of the detection results of this batch of products is unknown, and additional manual re-inspection is required before evaluation. If the evaluation results show that the detection accuracy of a category is too low and does not meet the standard, the model needs to be retrained, and retraining may cause the category that originally met the accuracy standard to fail to meet the standard, which is easy to hinder each other and requires repeated adjustments and tests, delaying timely online production. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem in the prior art that due to the black box characteristics of the classification model, there is negative interference in the detection between different categories, and the detection accuracy is low after the type of object to be detected is changed.

[0007] In order to solve the above technical problems, the present invention provides a screen defect classification method based on a deep learning image classification model, comprising:

[0008] Obtain sample data of screen defects, and divide them into multiple sample data sets according to defect categories; use each sample data set to train a binary classification model corresponding to the defect category;

[0009] Input all the screen images to be detected in the current batch into each trained binary classification model to obtain multiple classification results for each screen image to be detected;

[0010] For each screen image to be detected, if there is only one classification result that is positive among all classification results, the defect category represented by the classification result is used as the screen defect category of the screen image to be detected;

[0011] Otherwise, the screen image to be detected is determined to be a conflict image, and the conflict image is sent to manual visual inspection to obtain the screen defect category of the screen image to be detected.

[0012] Preferably, the using each sample data set to train a binary classification model corresponding to the defect category includes:

[0013] For the sample data set corresponding to each defect category, a training set is constructed with the sample data in the sample data set as positive samples and the sample data in the remaining sample data sets as negative samples, so as to obtain a training set corresponding to each defect category;

[0014] Based on the data set corresponding to each defect category, the binary classification model is trained to obtain the binary classification model corresponding to each defect category;

[0015] The binary classification model outputs a positive classification result when the corresponding defect category exists on the input screen image, otherwise, the output classification result is negative.

[0016] Preferably, it also includes:

[0017] After obtaining the screen defect categories of all screen images to be detected in the current batch, count the number of conflicting images in the current batch, calculate the ratio of the number of conflicting images to the number of all screen images to be detected in the current batch, and use it as the conflict ratio of the current batch, and compare it with the preset conflict threshold:

[0018] If the conflict ratio is not greater than the preset conflict threshold, there is no need to re-detect the screen image to be detected;

[0019] If the conflict ratio is greater than the preset conflict threshold, all screen images to be tested in the current batch are recalled and sent to manual visual inspection to re-obtain the screen defect categories of all screen images to be tested in the current batch.

[0020] Preferably, if the conflict ratio is greater than a preset conflict threshold, it also includes: using the gradient descent method, based on the defect category obtained by manual visual inspection, the conflicting binary classification model is trained again until the accuracy of the conflicting binary classification model reaches a preset accuracy, and a pre-trained binary classification model is obtained.

[0021] Preferably, the binary classification model includes any one of LeNet, AlexNet, VGG and ResNet.

[0022] Preferably, the defect categories include glass chips, no foreign matter, foreign matter with halo, foreign matter without halo, fiber with halo, fiber without halo, and cracks.

[0023] This embodiment also provides a screen defect classification device based on a deep learning image classification model, including:

[0024] A pre-training module is used to obtain sample data of screen defects and divide them into multiple sample data sets according to defect categories; each sample data set is used to train a binary classification model corresponding to the defect category;

[0025] The conflict detection module is used to input all the screen images to be detected in the current batch into each of the trained binary classification models, and obtain multiple classification results for each screen image to be detected; for each screen image to be detected, if there is only one classification result that is positive among all the classification results, the defect category represented by the classification result is used as the screen defect category of the screen image to be detected; otherwise, the screen image to be detected is judged to be a conflict image, and the conflict image is sent to manual visual inspection to obtain the screen defect category of the screen image to be detected.

[0026] Preferably, it also includes a self-assessment module, which is used to obtain the screen defect categories of all screen images to be detected in the current batch, count the number of conflicting images in the current batch, calculate the ratio of the number of conflicting images to the number of all screen images to be detected in the current batch, and use it as the conflict ratio of the current batch, which is compared with a preset conflict threshold: if the conflict ratio is not greater than the preset conflict threshold, there is no need to re-detect the screen images to be detected; if the conflict ratio is greater than the preset conflict threshold, all screen images to be detected in the current batch are recalled and sent for manual visual inspection, and the screen defect categories of all screen images to be detected in the current batch are re-obtained.

[0027] Preferably, a retraining module is also included. If the conflict ratio is greater than a preset conflict threshold, the binary classification model in conflict is trained again based on the defect category obtained by manual visual inspection using the gradient descent method until the accuracy of the binary classification model in conflict reaches a preset accuracy, thereby obtaining a pre-trained binary classification model.

[0028] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the screen defect classification method based on the deep learning image classification model as described above are implemented.

[0029] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0030] The screen defect classification method based on deep learning image classification model described in the present invention trains corresponding binary classification models for different defect categories, uses multiple binary classification models to detect the screen image to be detected, and obtains multiple classification results; determines whether the multiple classification results conflict, if there is a conflict, it is sent for manual visual inspection, if there is no conflict, the current classification result is used as the next screen category of the screen image to be detected. The present invention uses multiple binary classification models trained with different defect categories to detect the screen image to be detected at the same time, ensuring that the binary classification models can accurately identify the defects corresponding to each other. When facing continuous adjacent product batches with high feature consistency, the conflict between the classification results output by different binary classification models can be used, and the conflicting ones are sent to manual visual inspection, which reduces the number of screen images that need manual visual inspection and improves the accuracy of category detection; and for newly added defect categories, new binary classification models can be trained and integrated into existing devices, which are highly flexible and easy to expand.

[0031] When conflicts occur in classification results, the present invention retrains the conflicting binary classification model, collects feedback results from manual visual inspection, continuously optimizes the binary classification model, improves its generalization ability and accuracy, and further improves the accuracy of screen defect category detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0033] Figure 1 It is a flowchart of the steps of the screen defect classification method based on the deep learning image classification model provided by the present invention;

[0034] Figure 2 It is a structural schematic diagram of a screen defect classification device based on a deep learning image classification model provided by the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0036] Reference Figure 1 As shown, the step flow chart of the screen defect classification method based on the deep learning image classification model provided by the present invention includes the following steps:

[0037] S101: obtaining sample data of screen defects, and dividing the data into a plurality of sample data sets according to defect categories; and using each sample data set to train a binary classification model corresponding to the defect category;

[0038] S101-1: For a sample data set corresponding to each defect category, construct a training set using sample data in the sample data set as positive samples and sample data in other sample data sets as negative samples, to obtain a training set corresponding to each defect category;

[0039] S101-2: Based on the data set corresponding to each defect category, a binary classification model is trained to obtain a binary classification model corresponding to each defect category;

[0040] The binary classification model outputs a positive classification result when the corresponding defect category exists on the input screen image, otherwise, the output classification result is negative.

[0041] S102: inputting all the screen images to be detected in the current batch into each trained binary classification model respectively, and obtaining multiple classification results for each screen image to be detected;

[0042] S103: For each screen image to be detected, if there is only one classification result that is positive among all the classification results, the defect category represented by the classification result is used as the screen defect category of the screen image to be detected; otherwise, the screen image to be detected is judged to be a conflict image, and the conflict image is sent to manual visual inspection to obtain the screen defect category of the screen image to be detected.

[0043] The screen defect classification method based on deep learning image classification model described in the present invention trains corresponding binary classification models for different defect categories, uses multiple binary classification models to detect the screen image to be detected, and obtains multiple classification results; determines whether the multiple classification results conflict, if there is a conflict, it is sent for manual visual inspection, if there is no conflict, the current classification result is used as the next screen category of the screen image to be detected. The present invention uses multiple binary classification models trained with different defect categories to detect the screen image to be detected at the same time, ensuring that the binary classification models can accurately identify the defects corresponding to each other. When facing continuous adjacent product batches with high feature consistency, the conflict between the classification results output by different binary classification models can be used, and the conflicting ones are sent to manual visual inspection, which reduces the number of screen images that need manual visual inspection and improves the accuracy of category detection; and for newly added defect categories, new binary classification models can be trained and integrated into existing devices, which are highly flexible and easy to expand.

[0044] Based on the above embodiment, after obtaining the screen defect categories of all screen images to be detected in the current batch, the number of conflicting images in the current batch is counted, and the ratio of the number of conflicting images to the number of all screen images to be detected in the current batch is calculated as the conflict ratio of the current batch, which is compared with the preset conflict threshold:

[0045] If the conflict ratio is not greater than the preset conflict threshold, there is no need to re-detect the screen image to be detected;

[0046] If the conflict ratio is greater than the preset conflict threshold, all screen images to be tested in the current batch are recalled and sent to manual visual inspection to re-obtain the screen defect categories of all screen images to be tested in the current batch.

[0047] Among them, if the conflict ratio is greater than the preset conflict threshold, it also includes: using the gradient descent method, based on the defect category obtained by manual visual inspection, the conflicting binary classification model is trained again until the accuracy of the conflicting binary classification model reaches the preset accuracy, and a pre-trained binary classification model is obtained.

[0048] When conflicting classification results occur, the present invention retrains the conflicting binary classification model, continuously optimizes the binary classification model by collecting feedback results from manual visual inspection, improves its generalization ability and accuracy, and further improves the accuracy of screen defect category detection.

[0049] Specifically, in an embodiment of the present invention, the binary classification model used includes any one of LeNet, AlexNet, VGG and ResNet.

[0050] Based on the above embodiments, in an embodiment of the present invention, the screen defect classification method based on the deep learning image classification model provided by the present invention is used to detect different types of products on the production line, specifically including:

[0051] S201: Dataset processing;

[0052] Assume that the products on the production line are divided into three categories: A, B, and C. The original dataset contains images of products of all categories. The dataset processing module regroups the original dataset into three new datasets, named D_A, D_B, and D_C. In D_A, all images of products in category A are used as positive samples, and images of products in categories B and C are used as negative samples. Similarly, D_B and D_C are also grouped in this way.

[0053] S202: Classification model inference;

[0054] Use deep learning architecture (such as CNN) to build three independent binary classification models, named Model l_A, Model l_B, and Model l_C. Model l_A is trained based on dataset D_A to identify Class A products; Model l_B and Model l_C are trained based on D_B and D_C, respectively, to identify Class B and Class C products.

[0055] S203: Classification result conflict detection:

[0056] The image to be classified is input into Mode 1_A, Mode 1_B, and Mode 1_C for inference. If two or more models give different classification results, it is considered that there is a conflict. In this embodiment, the conflict threshold is set to the different results given by the two models.

[0057] S204: Result output and recording:

[0058] For images without conflict, the classification results and model confidence are directly output;

[0059] For conflicting images, conflict prompts are output and sent to manual visual inspection.

[0060] After processing a batch of images each time, the recorded data (including classification results, confidence, conflict, etc.) is passed to the self-evaluation module.

[0061] S205: Self-Assessment:

[0062] The self-evaluation module counts the proportion of conflicting samples in each batch of images. If the conflict ratio is lower than the preset threshold (such as 10%), it is considered that the current classification model module can effectively identify the batch of products, and the product images of the batch sent for visual inspection are retained in the data set for the next training. If the conflict ratio is higher than the threshold, the inspected products of the batch are recalled for manual visual inspection, and the training module is triggered to retrain the model with serious conflicts.

[0063] S206: Training:

[0064] The classification model is retrained based on the feedback from the self-assessment module.

[0065] If it is detected that the classification result of a certain category is not good, such as the error rate of Model l_A is high when identifying Class A products, the training module is triggered to train Model l_A. The training data can come from the retained data set or the newly collected data.

[0066] Specifically, in the embodiment of the present invention, the defect categories include glass chips, no foreign matter, foreign matter with halo, no foreign matter with halo, fiber with halo, no fiber with halo, and cracks:

[0067] ① When there is only one positive, it is a credible result; for example, the outputs of multiple classifiers are: glass chips-Negative, no foreign matter-Negative, foreign matter with halo-Positive, foreign matter without halo-Negative, fiber with halo-Negative, no fiber with halo-Negative, crack-Negative, then there is no conflict, and the defect category detection result is definitely that there is foreign matter with halo.

[0068] ② If the output results of multiple classifiers are: glass chips - Positive, no foreign matter - Negative, foreign matter with halo - Positive, foreign matter without halo - Negative, fiber with halo - Negative, fiber without halo - Negative, crack - Positive, and there is more than one positive, then there can only be one defect and it is impossible for two defects to appear at the same time. At this time, a conflict occurs and the screen image to be tested needs to be sent to manual visual inspection.

[0069] Based on the above embodiments, in an embodiment of the present invention, a screen defect classification method based on a deep learning image classification model provided by the present invention is used to conduct a comparative experiment on a test set containing 10,000 product sample images; among them, in the classification results of the traditional single classification model, there are 8,625 correct cases and 1,375 incorrect cases; in the classification results of the present invention, there are 9,175 correct cases and 463 incorrect cases, and 362 cases with conflicting conclusions were sent for manual visual inspection.

[0070] The results show that the accuracy of the traditional single classification model is 86.25%, while the accuracy of the system of the present invention is 91.75%, the direct accuracy rate is increased by 5.5%, and the error rate is reduced by 9.12%; at the same time, the present invention also directly submits 3.62% of the difficult-to-detect samples to manual visual inspection through the conflict detection module, further improving the final detection accuracy of the entire batch of products, and improving the efficiency of manual visual inspection. Therefore, in terms of effect, the present invention can not only improve the accuracy of the conclusions given by the model, but also directly submit a part of the difficult-to-detect samples to manual visual inspection through the conflict detection module, further improving the final detection accuracy of the entire batch of products, and also improving the efficiency of manual visual inspection, which can reduce the manpower demand for visual inspection. When a new classification category appears in the product change, it is only necessary to add a new category model to the classification model inference module in the present invention and train it, which will not affect the use of the existing category model.

[0071] Based on the above embodiments, an embodiment of the present invention further provides a screen defect classification device based on a deep learning image classification model, comprising:

[0072] A pre-training module is used to obtain sample data of screen defects and divide them into multiple sample data sets according to defect categories; each sample data set is used to train a binary classification model corresponding to the defect category;

[0073] The conflict detection module is used to input all the screen images to be detected in the current batch into each of the trained binary classification models, and obtain multiple classification results for each screen image to be detected; for each screen image to be detected, if there is only one classification result that is positive among all the classification results, the defect category represented by the classification result is used as the screen defect category of the screen image to be detected; otherwise, the screen image to be detected is judged to be a conflict image, and the conflict image is sent to manual visual inspection to obtain the screen defect category of the screen image to be detected.

[0074] The screen defect classification device based on the deep learning image classification model of this embodiment is used to implement the aforementioned screen defect classification method based on the deep learning image classification model. Therefore, the specific implementation method of the screen defect classification device based on the deep learning image classification model can be seen in the embodiment part of the screen defect classification method based on the deep learning image classification model in the previous text. For example, the pre-training module 100 is used to implement step S101 in the above-mentioned screen defect classification method based on the deep learning image classification model; the conflict detection module 200 is used to implement steps S102 and S103 in the above-mentioned screen defect classification method based on the deep learning image classification model. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part, which will not be repeated here.

[0075] Specifically, the screen defect classification device based on the deep learning image classification model provided by the embodiment of the present invention also includes a self-evaluation module, which is used to obtain the screen defect categories of all screen images to be detected in the current batch, count the number of conflicting images in the current batch, and calculate the ratio of the number of conflicting images to the number of all screen images to be detected in the current batch as the conflict ratio of the current batch, and compare it with the preset conflict threshold: if the conflict ratio is not greater than the preset conflict threshold, there is no need to re-detect the screen image to be detected; if the conflict ratio is greater than the preset conflict threshold, all screen images to be detected in the current batch are recalled and sent for manual visual inspection, and the screen defect categories of all screen images to be detected in the current batch are re-obtained.

[0076] Specifically, the screen defect classification device based on the deep learning image classification model provided by the embodiment of the present invention also includes a retraining module. If the conflict ratio is greater than a preset conflict threshold, the gradient descent method is used to train the conflicting binary classification model again based on the defect category obtained by manual visual inspection until the accuracy of the conflicting binary classification model reaches the preset accuracy, thereby obtaining a pre-trained binary classification model.

[0077] This embodiment implements image classification by introducing multiple binary classification models, and improves classification accuracy through conflict detection and self-evaluation mechanisms. The system is highly flexible and scalable. When new classification categories appear due to product changes, only the new category model needs to be added and trained, which will not affect the use of existing category models.

[0078] The present invention is based on a multi-model integrated classification strategy, and uses a binary classification method with multiple models to cope with the problem of variable product category changes and easy interference between categories in actual production. At the same time, the present invention designs an effective conflict detection algorithm. When multiple models give different classification results for the same image, it can accurately identify and mark it as a conflicting image to ensure the reliability of the classification results. The system also has a self-evaluation function, which can automatically judge the performance of the current model based on the proportion of conflicting classification results, and trigger a retraining process when necessary to achieve continuous optimization of the model.

[0079] Reference Figure 2 As shown, it is a structural schematic diagram of a screen defect classification device based on a deep learning image classification model provided by an embodiment of the present invention; specifically, this embodiment provides a system and method for solving image classification conflicts and self-evaluation based on multiple binary classification models, and the system includes: a data set processing module, a classification model inference module, a classification result conflict detection module, a self-evaluation module, a result output module, and a training module. The modules communicate with each other through a data interface to realize functions such as image data preprocessing, classification, result detection, evaluation, and output. The functions of each module are described as follows:

[0080] Dataset processing module: The dataset processing module regroups the original dataset. Specifically, assuming that the original dataset has N categories, N new datasets are regrouped. The positive samples in each new dataset are samples of that category in the original dataset, and the negative samples are the set of all category samples that are not samples of that category in the original dataset. Each new dataset will be used for model training of each corresponding category.

[0081] Classification model inference module: The classification model inference module contains multiple independent binary image classification models, each of which is built based on a deep learning architecture, such as CNN (convolutional neural network). Each model is responsible for inferring the input image as positive or negative. The model architecture and hyperparameters are optimized and selected through methods such as cross-validation.

[0082] Classification result conflict detection module: The classification result conflict detection module receives the classification results output by the classification model module and performs conflict detection on the results. In this embodiment, conflict is defined as when more than a certain proportion of models give different classification results. The specific conflict threshold can be set according to the application scenario. When a conflict is detected, the image will be marked as a conflict image and passed to the subsequent processing module.

[0083] Result output and recording module: The result output and recording module is responsible for outputting the final classification results in the form of visualization or report. For images without conflict, the classification results and model confidence are directly output. For conflicting images, conflict prompts are output and sent for manual visual inspection. After each batch of image samples is detected, the recorded data is passed to the self-evaluation module.

[0084] Self-evaluation module: The self-evaluation module evaluates the classification results of a batch of products during the production process. For each batch of products, the proportion of samples that conflict in the classification result conflict detection module is counted. If the conflict ratio is lower than the preset threshold, it is considered that the current classification model module can effectively identify the batch of products, and the product images of the batch sent for visual inspection are retained in the data set for the next training. If the conflict ratio is higher than the threshold, the inspected products of the batch are recalled for manual visual inspection, and the model with unsatisfactory accuracy is retrained.

[0085] Training module: Use the initial training data set to train each classification model. Then, based on the feedback from the self-evaluation module, retrain the classification model. When a poor classification result is detected for a certain category, the training module is triggered to train the binary classification model for that category. The training data can come from the retained data set or newly collected data. During the training process, optimization algorithms such as gradient descent can be used to adjust the model parameters to improve the model classification performance.

[0086] The system and method implement image classification by introducing multiple binary classification models, and improve the classification accuracy through conflict detection and self-evaluation mechanisms. In practical applications, the system can effectively solve the conflict problem in image classification and improve the reliability and accuracy of classification results. At the same time, the self-evaluation mechanism can timely discover and repair model problems to ensure the continuous and stable operation of the system.

[0087] Based on the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the screen defect classification method based on the deep learning image classification model as described above are implemented.

[0088] The screen defect classification method based on deep learning image classification model described in the present invention trains corresponding binary classification models for different defect categories, uses multiple binary classification models to detect the screen image to be detected, and obtains multiple classification results; determines whether the multiple classification results conflict, if there is a conflict, it is sent for manual visual inspection, if there is no conflict, the current classification result is used as the next screen category of the screen image to be detected. The present invention uses multiple binary classification models trained with different defect categories to detect the screen image to be detected at the same time, ensuring that the binary classification models can accurately identify the defects corresponding to each other. When facing continuous adjacent product batches with high feature consistency, the conflict between the classification results output by different binary classification models can be used, and the conflicting ones are sent to manual visual inspection, which reduces the number of screen images that need manual visual inspection and improves the accuracy of category detection; and for newly added defect categories, new binary classification models can be trained and integrated into existing devices, which are highly flexible and easy to expand. When conflicts occur in classification results, the present invention retrains the conflicting binary classification model, collects feedback results from manual visual inspection, continuously optimizes the binary classification model, improves its generalization ability and accuracy, and further improves the accuracy of screen defect category detection.

[0089] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A screen defect classification method based on a deep learning image classification model, characterized in that: include: Obtain screen defect sample data, and divide it into multiple sample data sets according to defect categories; Use each sample data set to train a binary classification model corresponding to the defect category; Input all the screen images to be detected in the current batch into each trained binary classification model to obtain multiple classification results for each screen image to be detected; For each screen image to be detected, if there is only one classification result that is positive among all classification results, the defect category represented by the classification result is used as the screen defect category of the screen image to be detected; Otherwise, the screen image to be detected is determined to be a conflict image, and the conflict image is sent to manual visual inspection to obtain the screen defect category of the screen image to be detected.

2. The screen defect classification method based on the deep learning image classification model according to claim 1, characterized in that: The method of using each sample data set to train a binary classification model corresponding to the defect category includes: For the sample data set corresponding to each defect category, a training set is constructed with the sample data in the sample data set as positive samples and the sample data in the remaining sample data sets as negative samples, so as to obtain a training set corresponding to each defect category; Based on the data set corresponding to each defect category, the binary classification model is trained to obtain the binary classification model corresponding to each defect category; The binary classification model outputs a positive classification result when the corresponding defect category exists on the input screen image, otherwise, the output classification result is negative.

3. The screen defect classification method based on the deep learning image classification model according to claim 1, characterized in that: Also includes: After obtaining the screen defect categories of all screen images to be detected in the current batch, count the number of conflicting images in the current batch, calculate the ratio of the number of conflicting images to the number of all screen images to be detected in the current batch, and use it as the conflict ratio of the current batch, and compare it with the preset conflict threshold: If the conflict ratio is not greater than the preset conflict threshold, there is no need to re-detect the screen image to be detected; If the conflict ratio is greater than the preset conflict threshold, all screen images to be tested in the current batch are recalled and sent to manual visual inspection to re-obtain the screen defect categories of all screen images to be tested in the current batch.

4. The screen defect classification method based on the deep learning image classification model according to claim 3 is characterized in that: If the conflict ratio is greater than a preset conflict threshold, the method further includes: using a gradient descent method to retrain the conflicting binary classification model based on defect categories obtained through manual visual inspection until the accuracy of the conflicting binary classification model reaches a preset accuracy, thereby obtaining a pre-trained binary classification model.

5. The screen defect classification method based on deep learning image classification model according to claim 1, characterized in that: The binary classification model includes any one of LeNet, AlexNet, VGG and ResNet.

6. The screen defect classification method based on deep learning image classification model according to claim 1, characterized in that: The defect categories include glass chips, no foreign matter, foreign matter with halo, foreign matter without halo, fiber with halo, fiber without halo, and cracks.

7. A screen defect classification device based on a deep learning image classification model, characterized in that: include: A pre-training module is used to obtain sample data of screen defects and divide them into multiple sample data sets according to defect categories; Use each sample data set to train a binary classification model corresponding to the defect category; The conflict detection module is used to input all the screen images to be detected in the current batch into each of the trained binary classification models, and obtain multiple classification results for each screen image to be detected; for each screen image to be detected, if there is only one classification result that is positive among all the classification results, the defect category represented by the classification result is used as the screen defect category of the screen image to be detected; otherwise, the screen image to be detected is judged to be a conflict image, and the conflict image is sent to manual visual inspection to obtain the screen defect category of the screen image to be detected.

8. The screen defect classification device based on deep learning image classification model according to claim 7, characterized in that: It also includes a self-assessment module, which is used to obtain the screen defect categories of all screen images to be detected in the current batch, count the number of conflicting images in the current batch, calculate the ratio of the number of conflicting images to the number of all screen images to be detected in the current batch as the conflict ratio of the current batch, and compare it with a preset conflict threshold: if the conflict ratio is not greater than the preset conflict threshold, there is no need to re-detect the screen image to be detected; If the conflict ratio is greater than the preset conflict threshold, all screen images to be tested in the current batch are recalled and sent to manual visual inspection to re-obtain the screen defect categories of all screen images to be tested in the current batch.

9. The screen defect classification device based on deep learning image classification model according to claim 8, characterized in that: It also includes a retraining module. If the conflict ratio is greater than a preset conflict threshold, the gradient descent method is used to retrain the conflicting binary classification model based on the defect categories obtained by manual visual inspection until the accuracy of the conflicting binary classification model reaches the preset accuracy, thereby obtaining a pre-trained binary classification model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the screen defect classification method based on a deep learning image classification model as described in any one of claims 1 to 6 are implemented.