Laptop screen defect detection method and medium based on self-learning network model
By using a self-learning network model, combined with high-definition camera video and deep network training, the accuracy and adaptability issues of laptop screen defect detection have been solved, achieving efficient and low-cost defect detection and improving the efficiency and quality of the production line.
Patent Information
- Application Number
- CN202110859010.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-28
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-07-28
AI Technical Summary
Traditional methods for detecting defects in laptop screens suffer from low detection accuracy, limited adaptability to different scenarios, poor transferability, and the need for large amounts of sample data for deep learning methods, making them particularly difficult to detect effectively in complex scenarios and with limited sample sizes.
A self-learning network model is adopted. High-definition camera videos from the laptop production line are acquired, preprocessed, and keyframes are extracted. A deep network model is trained using k-means clustering and cross-entropy loss function. Defect detection is performed using the GoogLeNet backbone network and AL decision layer. The detection results are optimized by manual verification and reverse adjustment of model parameters.
By reducing the time and effort spent on manual sample labeling, the success rate of testing has been improved, the false positive rate and labor costs have been reduced, and the efficiency and quality of the laptop screen production line have been enhanced.
Smart Images

Figure CN113569737B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent manufacturing defect detection, and in particular to a notebook screen defect detection method based on an autonomous learning network model. Background Art
[0002] Traditional appearance defect detection methods rely on manual inspection, which is time-consuming, labor-intensive, and has a high false detection rate. Later, the industry introduced ASI (Automated Surface Inspection) systems, but most of them are based on traditional machine vision technologies, including:
[0003] 1) Methods based on traditional digital image processing plus machine learning. This method first uses human experience to manually create features (feature engineering), then uses machine learning methods (such as support vector machines and template matching) to train samples and then determine whether there are defects in the notebook's appearance.
[0004] 2) Pen appearance defect detection method based on deep learning. Compared with traditional machine learning methods, deep learning generally does not require various manually designed image preprocessing techniques, is more adaptable and generalizable, and can produce more accurate appearance defect detection results when trained on large sample sizes. Therefore, it has been widely used in recent years.
[0005] Of the two methods mentioned above, the former has a low algorithm complexity and can be easily ported to ordinary embedded devices, but its accuracy is not high, and there are problems such as false detection and missed detection. In addition, its adaptation scenarios are relatively single and its portability is poor. The latter has wide adaptability and high accuracy. Although it avoids complex traditional feature engineering, its algorithm complexity is high and requires a large number of training samples. In actual applications, there may be no or few samples available for training. At the same time, if manual labeling of data is chosen, it will also require a lot of time and financial costs, and it is difficult to implement. Summary of the Invention
[0006] The present invention proposes a notebook screen defect detection method and medium based on an autonomous learning network model, which is used to overcome the problems of traditional machine learning such as low detection accuracy, single scene adaptability, poor migration, and the need for a large amount of sample data for deep learning-based defect detection in recent years. That is, the present invention is a notebook surface defect detection and classification algorithm with an autonomous learning strategy suitable for complex scenes and small sample situations.
[0007] To achieve the above object, the present invention adopts the following technical solutions:
[0008] A method for detecting defects in notebook screens based on an autonomous learning network model comprises the following steps: obtaining high-definition camera videos from a notebook production line, pre-processing the videos, and then inputting them into a notebook screen detection model; detecting the videos to be detected from the high-definition cameras of the notebook screen production line, and obtaining detection results;
[0009] The steps for building the laptop screen detection model are as follows:
[0010] Step 1: Extract keyframes from pre-obtained HD camera videos from a laptop production line containing the detection target, obtain raw sample images, and perform preprocessing to obtain a certain number of raw dataset images. Manually calibrate the pixel areas containing the specified production defects in the raw dataset images to obtain the coordinate information of the production defects as input for subsequent training models.
[0011] Step 2: Build a defect detection network model. Use the k-means clustering method to obtain the location of the prior boxes containing defect images in the dataset. Use the cross-entropy loss function to train the constructed deep network model and adjust the network model parameters to obtain the target detection model.
[0012] Step 3: Obtain the video to be inspected from the high-definition camera of the production line, feed it into the pre-trained target detection model, and directly output the detection results. For video frames that are difficult for the network model to distinguish, sort them according to the predicted probability of defective targets. After that, they are verified by humans and used as training sample sets. The model then reversely adjusts the model parameters. After reaching the set number of times, a stable network model is obtained.
[0013] Furthermore, the step 1 includes the following steps:
[0014] S11: Collect inspection videos from a high-definition camera on a notebook appearance production line, extract original samples of key frames containing appearance defects, and perform mean normalization image preprocessing on the original samples to construct a training sample set.
[0015] S12: Annotate the pixel areas containing appearance defects in the dataset image with rectangular boxes, where a dataset image contains multiple rectangular boxes, that is, multiple appearance defects, but each rectangular box only contains one target to be detected, and the annotated label results are saved in an XML file according to the VOC dataset format; where each image corresponds to an XML file with the same name, which contains the center point coordinate information (x, y) of each rectangular box in the dataset image, the width w of the rectangular box, and the height h of the rectangular box; here, the label in each batch image, that is, (x, y, w, h) is normalized in size, that is, it is mapped to a sub-area under 256*256 coordinates in the original image coordinate system according to the unchanged aspect ratio of the original image, and the transformed coordinate information is stored in the XML file as the input of the subsequent training model.
[0016] Furthermore, the defect detection network model in step 2 includes an input layer, a GoogLeNet backbone network, an AL decision layer, and an output layer. In addition, since appearance defects are mostly point-like or linear structures, the two softmax layers in GoogLeNet are removed to accelerate network training.
[0017] The GoogLeNet backbone network is a 22-layer deep network with three groups of Inception modules separated by pooling layers.
[0018] The Inception module uses dense components to approximate the optimal local sparse structure. This structure stacks commonly used convolution kernels (1*1, 3*3, 5*5) and pooling operations, which not only increases the network depth but also improves the network's adaptability to scale.
[0019] The input image is resized to 224*224*3 and input to the first layer. It then passes through 64 convolution kernels of 7*7*3 with a stride of 2 to output 112*112*64 data. The output is then input to the ReLU activation, and then the network parameters are reduced through the maximum pooling operation as the input of the Inception module.
[0020] Furthermore, the training steps of step 2 are as follows:
[0021] S21: Pre-train the backbone network GoogLeNet using the public dataset to obtain the original parameters;
[0022] S22: For the rectangular boxes marked in the data and pictures, cluster them to obtain anchor boxes. Here, first extract the coordinate information of the rectangular boxes of all training pictures, and then obtain the width and height information of the rectangular boxes from the two coordinate information. Use the k-means clustering method to process the width and height data of the bounding boxes of all training data. The processing process is as follows:
[0023] First, initialize k anchor boxes by randomly selecting k values from all rectangular bounding boxes as the initial values of k anchor boxes;
[0024] Then, calculate the IoU value of each bounding box and each anchor box to generate k anchor boxes with different widths and heights. Here, IoU is the ratio of the intersection and union of the "predicted border" and the "real border".
[0025] Then, the distance d = 1-IoU (labeled box, cluster center) between each annotation box and each cluster center is calculated. When calculating, the center point of each annotation box coincides with the cluster center, so that the IOU value can be calculated, that is,
[0026] d=1-IoU[(x j ,y j ,w j ,h j ),(x j ,y j ,W j ,H j )]
[0027] j∈{1,2,…,N},i∈{1,2,…,k}
[0028] Assign the label box to the cluster center with the closest "distance";
[0029] After all the annotation boxes are assigned, the cluster center point is recalculated for each cluster and the width and height values of the anchorbox are updated;
[0030] The calculation method is N i is the number of labeled boxes in the i-th cluster, and the average width and height of all labeled boxes in the cluster are obtained;
[0031] Repeat the above steps until the cluster center changes to the set value, and obtain the anchor box obtained by clustering the bounding box of the appearance defect dataset;
[0032] S23, inputting the training set into the backbone network GoogLeNet with the initial parameters, and extracting the corresponding feature map;
[0033] S24, using the anchorbox obtained by clustering in S22, predict the appearance defect feature map, and obtain the center coordinates of the predicted appearance defect area prediction box in the map ((b x ,b y ), the width and height of the prediction box, the probability of containing appearance defects and the probability of object conditional category;
[0034] S25. After a prescribed number of times and adjustment of the learning rate parameters, a notebook appearance defect detection model with classification results that meet the requirements is obtained after training.
[0035] Furthermore, for step S3: the framed area suspected of containing appearance defects is input into the trained target detection model for detection to determine whether it exists in the suspected area. The predictable detection results are directly output. For video frames that are difficult for the network model to distinguish, they are sorted according to the predicted defect target probability. After that, they are verified by humans as a training sample set, and then input into the model to reversely adjust the model parameters.
[0036] On the other hand, the present invention also discloses a notebook screen defect detection device based on an autonomous learning network model, comprising the following units:
[0037] A data acquisition unit, used to acquire high-definition camera videos from the notebook production line;
[0038] A data processing unit, used for pre-processing the acquired high-definition camera video;
[0039] The detection unit is used to input the processed video data into the notebook screen detection model, detect the video to be detected from the high-definition camera of the notebook screen production line, and obtain the detection results;
[0040] The construction unit of the screen detection model includes the following steps:
[0041] Step 1: Extract keyframes from pre-obtained HD camera videos from a laptop production line containing the detection target, obtain raw sample images, and perform preprocessing to obtain a certain number of raw dataset images. Manually calibrate the pixel areas containing the specified production defects in the raw dataset images to obtain the coordinate information of the production defects as input for subsequent training models.
[0042] Step 2: Build a defect detection network model. Use the k-means clustering method to obtain the location of the prior boxes containing defect images in the dataset. Use the cross-entropy loss function to train the constructed deep network model and adjust the network model parameters to obtain the target detection model.
[0043] Step 3: Obtain the video to be inspected from the high-definition camera of the production line, feed it into the pre-trained target detection model, and directly output the detection results. For video frames that are difficult for the network model to distinguish, sort them according to the predicted probability of defective targets. After that, they are verified by humans and used as training sample sets. The model then reversely adjusts the model parameters. After reaching the set number of times, a stable network model is obtained.
[0044] In a third aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0045] It can be seen from the above technical solution that the notebook screen defect detection method based on the autonomous learning network model of the present invention, that is, the notebook screen defect detection and classification method based on autonomous learning, can reduce the time and financial costs of traditional notebook production lines, and at the same time introduce intelligent detection and classification, which is of great significance to industrial transformation and upgrading, improving product quality, and enhancing competitiveness.
[0046] It should be noted that the manually labeled training sample set required in the present invention is small, and the network model trained for the first time is only a target network model that can better detect the input video frame. Its purpose is to reduce the time and effort spent on manually labeling samples. Later, in actual production practice, manual factors will be added for the second time, and new training samples will be generated semi-automatically to readjust the network model parameters and optimize the model parameters.
[0047] Specifically, the advantages of the present invention are as follows:
[0048] (1) Reduce the time and effort spent on manually annotating notebook screen defect samples;
[0049] (2) It has good performance in training of small-sample network models and can detect defects generated during the production process of notebook screens with a high success rate, reducing the false detection rate and labor costs of traditional manual inspection and improving the efficiency of the notebook screen production line. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 It is a schematic diagram of the method principle of the present invention;
[0051] Figure 2 It is a network structure diagram of the present invention;
[0052] Figure 3 This is the basic structure of the inception module of the present invention;
[0053] Figure 4 This is a reference diagram of the test results of an embodiment of the present invention. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.
[0055] like Figure 1 As shown, the notebook screen defect detection method based on the autonomous learning network model described in this embodiment includes:
[0056] Step 1: Obtain high-definition camera video from a notebook production line, extract keyframes from the video containing the detection target, and obtain original sample images. Then, use image enhancement and other preprocessing methods on the original sample images to obtain a suitable number of original dataset images. Finally, manually calibrate the pixel areas containing production defects (scratches, dents, etc.) in the original dataset images to obtain the coordinate information of the production defects as input for subsequent training models.
[0057] Step 2: Establish a defect detection network model. Use the k-means clustering method to obtain the prior box locations of defect images in the dataset. Use the cross-entropy loss function to train the constructed deep network model and adjust the network model parameters to obtain a model that can better predict the results.
[0058] Step 3: The video to be inspected from the high-definition camera of the production line is fed into a pre-trained target detection model. The predictable detection results are directly output. The video frames that are difficult for the network model to distinguish are sorted according to the predicted probability of defective targets. They are then verified by humans and used as training sample sets. The model then reversely adjusts the model parameters. After repeated cycles, a stable network model is obtained.
[0059] Furthermore, the specific steps in step 1 are subdivided into:
[0060] S11: Collect inspection videos from a high-definition camera on a notebook appearance production line, extract original samples of video key frames containing appearance defects, and use mean normalization image preprocessing on the original samples to construct a training sample set.
[0061] S12: Annotate the pixel areas containing appearance defects in the dataset image with rectangular boxes, where a dataset image can contain multiple rectangular boxes, i.e., multiple appearance defects, but each rectangular box only contains one target to be detected. The annotated label results are saved in an xml file according to the VOC dataset format; where each image corresponds to an xml file with the same name, which contains the center point coordinate information (x, y) of each rectangular box in the dataset image, the width w of the rectangular box, and the height h of the rectangular box. It should be noted that the labels in each batch image, i.e., (x, y, w, h), are normalized in size here, i.e., they are mapped to a sub-area under 256*256 coordinates in the original image coordinate system according to the unchanged aspect ratio of the original image, and the transformed coordinate information is stored in the xml file as the input of the subsequent training model.
[0062] For the target monitoring model structure constructed in step S2 above, refer to Figure 2, including the input layer, the GoogLeNet main backbone network, the AL decision layer and the output layer. In addition, since the appearance defects are mostly point-like or linear structures, the two softmax layers in GoogLeNet are removed to accelerate network training.
[0063] like Figure 2 and Figure 3 As shown, the GoogLeNet backbone network of the embodiment of the present invention is a 22-layer deep network, and the network has three groups of Inception modules:
[0064] inception(3a) / inception(3b), inception(4a) / inception(4b) / inception(4c) / inception(4d) / inception(4e), inception(5a) / inception(5b). The three groups of Inception modules are separated by pooling layers. The main idea of the Inception structure is to use dense components to approximate the optimal local sparse structure. This structure stacks commonly used convolution kernels (1*1, 3*3, 5*5) and pooling operations together, which not only increases the depth of the network but also increases the adaptability of the network to scale. The input image is resized to 224*224*3 and input to the first layer. After that, it passes through 64 7*7*3 convolution kernels with a stride of 2 to output 112*112*64 data. It is then input to ReLU activation and then the network parameters are reduced through the maximum pooling operation as the input of the Inception module. Taking Inception3a as an example, the input data is convolved with 64 1*1 convolution kernels, then activated by ReLU, and the output is 28*28*64; 2) convolution with 96 1*1 convolution kernels, and then dimensionality reduction with 3*3 convolution kernels, and then convolution with 128 3*3 convolution kernels, and the output is 28*28*128; 3) convolution with 16 1*1 convolution kernels, and then dimensionality reduction with 5*5 convolution kernels, and the output is 28*28*16 , then activated by ReLU, and then 32 5*5 convolutions are performed to output 28*28*32; ④ Pooling layer pooling operation, through the 3*3 convolution kernel, output 28*28*192, and then use 32 1*1 convolution kernels to output 28*28*32, finally connect the four results, and the output results of these four parts are connected in parallel in the third dimension, that is: 64+128+32+32=256, to determine the final output 28*28*256.
[0065] Images pre-processed with S1 are fed into the GoogLeNet network model as raw data for feature extraction and detection and classification. It's worth noting that GoogLeNet introduces the Inception layer, which uses readily available dense components to overlay an approximate optimal sparse structure for convolutional vision networks. This approach, based on fast convolutions, finds the optimal local structure. Currently, Inception layers typically use combinations of kernel sizes of 1*1, 3*3, and 5*5, with the filter groups concatenated into a single output vector that serves as input for the next stage. In areas where computational overhead is prohibitive, a 1*1 kernel is used to effectively reduce dimensionality. Furthermore, because appearance defects are often point-like or linear, the two softmax layers in GoogLeNet were removed to accelerate network training. An AL decision layer (which categorizes the classifier's detection results by probability for manual decision-making) was introduced at the final output. Images that the classifier fails to detect are used as training samples after manual decision-making.
[0066] In step S2, we build the target network model and train it, which mainly includes the following steps:
[0067] S21: Pre-train the backbone network GoogLeNet using the public dataset to obtain the original parameters.
[0068] S22: Cluster the rectangular boxes marked in the data and images to obtain anchor boxes. Here, we first extract the coordinate information of the rectangular boxes of all training images, and then obtain the width and height information of the rectangular boxes from the two coordinate information. Use the k-means clustering method to process the width and height data of the bounding boxes of all training data. The processing process is as follows:
[0069] First, initialize k anchor boxes by randomly selecting k values from all rectangular bounding boxes as the initial values of k anchor boxes;
[0070] Then, calculate the IoU value of each bounding box and each anchor box to generate k anchor boxes with different widths and heights. Here, IoU is the ratio of the intersection and union of the "predicted border" and the "real border".
[0071] Then, the distance d = 1-IoU (annotated box, cluster center) between each labeled box and each cluster center is calculated, where the labeled box is the box containing the detection result when the data set is manually annotated; the cluster center is the center point of the random box generated by the computer using the clustering algorithm;
[0072] When calculating, the center point of each annotation box coincides with the cluster center, so that the IOU value can be calculated, that is,
[0073] d=1-IoU[(x j ,y j ,w j ,h j ),(x j ,y j ,W j ,Hj)]
[0074] j∈{1,2,…,N},i∈{1,2,…,k}
[0075] Assign the label box to the cluster center with the closest "distance";
[0076] After all the annotation boxes are assigned, the cluster center point is recalculated for each cluster and the width and height of the anchorbox are updated. The calculation method is N i is the number of labeled boxes in the i-th cluster, and the average width and height of all labeled boxes in the cluster are obtained;
[0077] Repeat the above steps until the change in cluster center is small, and get the anchor box obtained by clustering the bounding box of the appearance defect dataset.
[0078] S23. Input the training set into the backbone network GoogLeNet with initial parameters and extract the corresponding feature map.
[0079] S24, using the anchorbox obtained by clustering in S22, predict the appearance defect feature map, and obtain the center coordinates of the predicted appearance defect area prediction box in the map ((b x ,b y ), the width and height of the prediction box, the probability of containing appearance defects and the object conditional category probability.
[0080] S25. After multiple adjustments to parameters such as the learning rate, a notebook appearance defect detection model with good classification results was obtained. Furthermore, to prevent overfitting, a mechanism was proposed to regularize the classifier by pre-estimating the marginalization effect of label dropout during training. For each training sample x, the model predicts the probability of label k∈{1…K}:
[0081]
[0082] Where Zi is the unnormalized logical probability value. Considering that q(k|x) is actually used instead of p(k|x), the cross entropy loss function is defined as:
[0083]
[0084] Minimizing the loss function is equivalent to the maximum likelihood function, where Z k Differentiable:
[0085]
[0086] However, the above method uses predicted probabilities to fit true probabilities, which may lead to two problems: (1) model overfitting; (2) it encourages predicted values to be much larger than their label values, and combined with the boundedness of the gradient, it weakens the model's generality. Therefore, a mechanism is proposed to encourage the model to reduce predictions, called label smoothing. We assume a label distribution u(k), introduce a smoothing parameter for a training sample x, and calculate q(k|x) instead of the actual sample x. The formula is as follows:
[0087] q′(k|x)=(1-θ)δ k,y +θu(k)
[0088] For step S3: the framed area suspected of containing appearance defects is input into the trained target detection model for detection to determine whether it exists in the suspected area. The predictable detection results are directly output. For video frames that are difficult for the network model to distinguish, they are sorted according to the predicted defect target probability. They are then verified by humans and used as training sample sets. They are then input into the model and the model parameters are adjusted in reverse.
[0089] Figure 4 This is a reference diagram of the detection results of an embodiment of the present invention. It can be seen that the notebook screen defect detection method based on the autonomous learning network model of the present invention can reduce the time and effort spent on manually labeling notebook screen defect samples, has good performance in training of small-sample network models, and can detect defects generated in the notebook screen production process with a high success rate, thereby reducing the false detection rate and labor cost of traditional manual detection and improving the efficiency of the notebook screen production line.
[0090] On the other hand, the present invention also discloses a notebook screen defect detection device based on an autonomous learning network model, comprising the following units:
[0091] A data acquisition unit, used to acquire high-definition camera videos from the notebook production line;
[0092] A data processing unit, used for pre-processing the acquired high-definition camera video;
[0093] The detection unit is used to input the processed video data into the notebook screen detection model, detect the video to be detected from the high-definition camera of the notebook screen production line, and obtain the detection results;
[0094] The construction unit of the screen detection model includes the following steps:
[0095] Step 1: Extract keyframes from pre-obtained HD camera videos from a laptop production line containing the detection target, obtain raw sample images, and perform preprocessing to obtain a certain number of raw dataset images. Manually calibrate the pixel areas containing the specified production defects in the raw dataset images to obtain the coordinate information of the production defects as input for subsequent training models.
[0096] Step 2: Build a defect detection network model. Use the k-means clustering method to obtain the location of the prior boxes containing defect images in the dataset. Use the cross-entropy loss function to train the constructed deep network model and adjust the network model parameters to obtain the target detection model.
[0097] Step 3: Obtain the video to be inspected from the high-definition camera of the production line, feed it into the pre-trained target detection model, and directly output the detection results. For video frames that are difficult for the network model to distinguish, sort them according to the predicted probability of defective targets. After that, they are verified by humans and used as training sample sets. The model then reversely adjusts the model parameters. After reaching the set number of times, a stable network model is obtained.
[0098] In a third aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0099] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts of the above method.
[0100] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0101] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0102] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A notebook screen defect detection method based on an autonomous learning network model, characterized in that: It includes building a laptop screen detection model, obtaining high-definition camera videos from the laptop production line, pre-processing them and inputting them into the laptop screen detection model, detecting the video to be detected from the high-definition camera of the laptop screen production line, and obtaining the detection results; The steps for building the laptop screen detection model are as follows: Step 1: For the high-definition camera video of the notebook production line obtained in advance, extract the key frames of the video containing the detection target, obtain the original sample images and perform preprocessing to obtain a certain number of original data set images; then manually calibrate the pixel area containing the specified production defects in the original data set images, and obtain the coordinate information of the production defects as the input of the subsequent training model; Step 2: Establish a defect detection network model, use the k-means clustering method to obtain the prior box position of the defect image in the data set, use the cross entropy loss function to train the constructed deep network model, adjust the network model parameters, and obtain the target detection model; Step 3: Obtain the video to be tested from the high-definition camera of the production line, feed it into the pre-trained target detection model, directly output the detection results, sort the video frames that are difficult for the network model to distinguish according to the predicted defect target probability, and then have a human verify it as a training sample set. After that, the model will reversely adjust the model parameters. After reaching the set number of times, a stable network model is obtained; The step 1 comprises the following steps: S11: Collect inspection videos of notebook appearance production lines under high-definition cameras, extract original samples of video key frames containing appearance defects, use "mean normalization" image preprocessing on the original samples, and construct a training sample set; S12: Annotate the pixel areas containing appearance defects in the dataset image with rectangular boxes, wherein one dataset image contains multiple rectangular boxes, i.e., multiple appearance defects, but each rectangular box only contains one target to be detected, and the annotated label results are saved in an xml file according to the VOC dataset format; wherein each image corresponds to an xml file with the same name, which contains the center point coordinate information (x, y) of each rectangular box in the dataset image, the width w of the rectangular box, and the height h of the rectangular box; here, the size of the label in each batch image, i.e., (x, y, w, h), is normalized, i.e., it is mapped to a sub-area under 256*256 coordinates in the original image coordinate system according to the aspect ratio of the original image, and the transformed coordinate information is stored in the xml file as the input of the subsequent training model; The defect detection network model in step 2 includes an input layer, a GoogLeNet main backbone network, an AL decision layer and an output layer. In addition, since most appearance defects are point-shaped or linear structures, the two softmax layers in GoogLeNet are removed to accelerate network training. The GoogLeNet backbone network is a 22-layer deep network with three groups of Inception modules separated by pooling layers. The Inception module uses dense components to approximate the optimal local sparse structure. This structure stacks commonly used convolution kernels (1*1, 3*3, 5*5) and pooling operations together, which not only increases the network depth, but also increases the network's adaptability to scale. The input image is normalized to 224*224*3 and input to the first layer. Then, it passes through 64 convolution kernels of 7*7*3 with a step size of 2 to output 112*112*64 data. Then, it is input to ReLU activation, and then the network parameters are reduced through the maximum pooling operation as the input of the Inception module. The convolution algorithm of the Inception module is as follows: The input data is convolved with ①64 1*1 convolution kernels, then activated by ReLU, and the output is 28*28*64; ②96 1*1 convolution kernels are convolved, and then the dimensionality is reduced by 3*3 convolution kernels, and then 128 3*3 convolution kernels are convolved, and the output is 28*28*128; ③16 1*1 convolution kernels are convolved, and then the dimensionality is reduced by 5*5 convolution kernels, and the output is 28*28*16, and then ReLU Activate, and then perform 32 5*5 convolutions to output 28*28*32; ④ Pooling layer pooling operation, through the 3*3 convolution kernel, output 28*28*192, then use 32 1*1 convolution kernels to output 28*28*32, and finally connect the four results, and connect the output results of these four parts in parallel in the third dimension, that is: 64+128+32+32=256, to determine the final output 28*28*256; The training steps of step 2 are as follows: S21: Pre-train the backbone network GoogLeNet using a public dataset to obtain original parameters; S22: For the rectangular boxes marked in the data and pictures, cluster them to obtain anchor boxes. Here, first extract the coordinate information of the rectangular boxes of all training pictures, and then obtain the width and height information of the rectangular boxes from the two coordinate information. Use the k-means clustering method to process the width and height data of the bounding boxes of all training data. The processing process is as follows: First, initialize K anchor boxes by randomly selecting K values from all rectangular bounding boxes as the initial values of K anchor boxes; Then, calculate the IoU value of each bounding box and each anchor box to generate k anchorboxes with different widths and heights. Here, IoU is the ratio of the intersection and union of the "predicted bounding box" and the "real bounding box". Then, the distance between each annotation box and each cluster center point is calculated as d = 1-IoU (annotation box, cluster center). During the calculation, the center point of each annotation box coincides with the cluster center, so that the IOU value can be calculated, that is, d=1-IoU[(x j ,y j ,w j ,h j ),(x j ,y j ,W i ,H i )] j∈{1,2,…,N},i∈{1,2,…,K} Assign the label box to the cluster center with the closest "distance"; After all the annotation boxes are assigned, the cluster center point is recalculated for each cluster and the width and height of the anchor box are updated; The calculation method is N i is the number of labeled boxes in the ith cluster, and the average width and height of all labeled boxes in the cluster are obtained; Repeat the above steps until the cluster center changes to the set value, and obtain the anchor box obtained by clustering the bounding box of the appearance defect dataset; S23, inputting the training set into the backbone network GoogLeNet with initial parameters, and extracting the corresponding feature map; S24, using the anchor box obtained by clustering in S22, predict the appearance defect feature map to obtain the center coordinates of the predicted appearance defect area prediction box in the map (b x ,b y ), the width and height of the prediction box, the probability of containing appearance defects and the probability of object conditional category; S25. After a prescribed number of times and adjustment of learning rate parameters, a notebook appearance defect detection model with classification results that meet the requirements is obtained after training.
2. The method for detecting laptop screen defects based on the autonomous learning network model according to claim 1 is characterized in that: The step 2 also includes regularizing the classifier by pre-estimating the marginalization effect of label discarding during the training process, specifically including: Assume that each training sample The model predicts the likelihood of a label k∈{1…K}: Where Z i is the unnormalized logical probability value, considering the actual use Rather than Its cross entropy loss function is defined as: Minimizing the loss function is equivalent to the maximum likelihood function, where Z k Differentiable:
3. The notebook screen defect detection method based on the autonomous learning network model according to claim 2 is characterized in that: The step 2 also includes assuming a label distribution u(k), for a training sample Introducing the smoothing parameter θ, calculate Instead of actual samples The formula is as follows:
4. The notebook screen defect detection method based on the autonomous learning network model according to claim 3 is characterized in that: For step S3: input the framed suspected appearance defects into the trained target detection model for detection to determine whether the suspected area exists. The predicted detection results are directly output. For video frames that are difficult for the network model to distinguish, they are sorted according to the predicted defect target probability. They are then verified by humans as a training sample set, and then input into the model to adjust the model parameters in reverse.
5. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Method and device for intelligently adjusting screen brightness, storage medium and mobile terminal
CN109951595A
Defect detection method based on target detection
CN110175982A
Inception model-based aluminum material defect identification device of convolutional neural network
CN110263630A
Aerial power component image classification method based on knowledge transfer learning
CN110472545A
Neural network model training method and device and terminal equipment
CN110766153A
