Display panel defect detection model training method, defect detection method and device

Through the full convolutional cross-scale flow model, defect detection of the display panel is solved, and the problem of low detection efficiency and accuracy in the prior art is achieved, efficient and accurate defect recognition is achieved, and manual detection costs are reduced.

CN118609127BActive Publication Date: 2024-11-12QINGDAO HAIGAO DESIGN & MANUFACTURING CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411026311.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-12
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The existing display panel defect detection methods rely on manual visual inspection, resulting in low detection efficiency and accuracy.

Method used

The display panel defect detection model based on full convolutional cross-scale flow is adopted, and the training model is adjusted through the training data set and the test data set, the feature map of the display panel sample image at different scales is obtained, and the cross-scale flow is used for processing. Output likelihood values ​​to indicate the presence of defects, improving detection accuracy and efficiency.

Benefits of technology

It realizes efficient analysis and defect identification of display panel images, improves the accuracy and efficiency of defect detection, reduces manual intervention and reduces costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118609127B_ABST
    Figure CN118609127B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a display panel defect detection model training method, a defect detection method and a device, and relates to the field of display panel detection technology. The above method includes: obtaining a data set, which includes a training data set and a test data set; training a model to be trained based on the training data set to obtain a display panel defect detection model; using the test data set to test the display panel defect detection model, and adjusting the model to be trained according to the test results, and continuing to train the adjusted model to be trained based on the training data set until the trained display panel defect detection model meets the preset standard. Obtain an image of the display panel to be detected; input the image into the above display panel defect detection model, and determine whether there is a defect on the surface of the display panel to be detected according to the output result of the display panel defect detection model; the present application can improve the accuracy and efficiency of display panel defect detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of display panels, and in particular to a display panel defect detection model training method, a defect detection method and a device. Background Art

[0002] In the manufacturing industry, the quality of display panels is one of the important factors affecting product appearance and user experience. High-quality display panels can not only enhance the visual appeal of the product, but also bring users a more comfortable and pleasant experience.

[0003] According to the existing manufacturing technology, some defects may occur in the display panel during the manufacturing process, which not only affects the aesthetics of the product, but also reduces the quality and market competitiveness of the product. Therefore, it is very necessary to perform defect detection on the display panel during the manufacturing process of the display panel.

[0004] However, existing display panel defect detection methods mainly rely on manual visual inspection, and the detection efficiency and accuracy are low. Summary of the invention

[0005] The present application provides a display panel defect detection model training method, a defect detection method and a device, which can solve the technical problems of low detection efficiency and accuracy in existing display panel defect detection methods.

[0006] In a first aspect, an embodiment of the present application provides a display panel defect detection model training method, the method comprising:

[0007] Acquire a data set, the data set comprising a training data set and a test data set; the training data set comprises a plurality of first display panel sample images with no surface defects, the test data set comprises a plurality of second display panel sample images with no surface defects, a plurality of third display panel sample images with surface defects, and defect annotation data corresponding to each of the third display panel sample images;

[0008] The model to be trained is trained based on the training data set to obtain a display panel defect detection model; the model to be trained is used to obtain feature maps of the first display panel sample image at different scales, and after processing the feature map using a cross-scale flow, output a likelihood value corresponding to the feature map, wherein the likelihood value is used to indicate whether there is a defect on the surface of the first display panel sample image at a scale corresponding to the feature map;

[0009] The display panel defect detection model is tested using the test data set, and the model to be trained is adjusted according to the test result, and the adjusted model to be trained is continuously trained based on the training data set until the trained display panel defect detection model meets the preset standard.

[0010] In some implementations, the model to be trained includes a convolutional neural network and a cross-scale flow, and the training of the model to be trained based on the training data set includes:

[0011] Using the convolutional neural network to obtain feature maps of the first display panel sample image at different scales;

[0012] Inputting each of the feature maps into the cross-scale stream; the cross-scale stream comprises a plurality of coupling blocks, each of the coupling blocks comprises a plurality of cross-convolution layers, and different cross-convolution layers are used to perform reversible transformation on the feature maps at different scales;

[0013] Calculating the likelihood value of the transformed feature map output by the cross-scale flow in the latent space;

[0014] The parameters of the model to be trained are adjusted according to the likelihood value.

[0015] In some embodiments, the using the convolutional neural network to obtain feature maps of the first display panel sample image at different scales includes:

[0016] Detecting the type of the first display panel sample and / or the resolution or size of the first display panel sample image;

[0017] Determine a plurality of different target scales corresponding to the first display panel sample image according to the type of the first display panel sample and / or the resolution or size of the first display panel sample image;

[0018] The convolutional neural network is used to obtain a feature map of the first display panel sample image at each of the target scales.

[0019] In some embodiments, the using the convolutional neural network to obtain feature maps of the first display panel sample image at different scales includes:

[0020] Acquire scale indication information, where the scale indication information includes a plurality of bits, and different bits correspond to different scales;

[0021] Determining a plurality of different target scales corresponding to the first display panel sample image according to values ​​of respective bits in the scale indication information;

[0022] The convolutional neural network is used to obtain a feature map of the first display panel sample image at each of the target scales.

[0023] In some implementations, the transformed feature map retains spatial information; the method further comprises:

[0024] The spatial information is used to visualize feature values ​​in the latent space to locate abnormal regions in the first display panel sample image.

[0025] In some embodiments, the method further comprises:

[0026] Collecting images of a plurality of display panels and detecting whether there are defects on the surfaces of the images;

[0027] When there is no defect in the surface of the image, adding the image to the training data set or the test data set;

[0028] When there are defects in the surface of the image, the defects on the surface of the image are marked to obtain defect marking data corresponding to the image, and the image and the defect marking data corresponding to the image are added to the test data set; the defect marking data includes at least one of the position, size, shape or type of the defects existing in the image.

[0029] In some embodiments, the method further comprises:

[0030] Extracting texture features from the collected image;

[0031] The texture features are added to the test dataset.

[0032] In a second aspect, an embodiment of the present application provides a display panel defect detection method, the method comprising:

[0033] Acquire an image of the display panel to be inspected;

[0034] Inputting the image into a display panel defect detection model, and determining whether there is a defect on the surface of the display panel to be detected according to an output result of the display panel defect detection model;

[0035] The display panel defect detection model is trained by the display panel defect detection model training method provided in the first aspect.

[0036] In some embodiments, the method further comprises:

[0037] When it is determined that there is a defect on the surface of the display panel to be inspected, obtaining positioning information output by the display panel defect detection model, the positioning information including an abnormal area where there is a defect on the surface of the display panel to be inspected;

[0038] The position, size or type information of the abnormal area is marked on the image and recorded in a database.

[0039] In some embodiments, the method further comprises:

[0040] When it is determined that there are defects on the surface of the display panel to be inspected, classifying the defects according to information output by the display panel defect detection model and a preset classification algorithm;

[0041] Performing a quality assessment on the display panel to be inspected according to the severity of the defect and assigning a corresponding quality grade to it;

[0042] According to the type of the defect and / or the quality grade of the display panel to be detected, a processing strategy corresponding to the display panel to be detected is selected; the processing strategy includes: automatically repairing the display panel to be detected, or triggering a quality alarm signal.

[0043] In a third aspect, an embodiment of the present application provides a display panel defect detection model training device, the device comprising:

[0044] an acquisition module, used to acquire a data set, wherein the data set includes a training data set and a test data set; the training data set includes a plurality of first display panel sample images with no surface defects, and the test data set includes a plurality of second display panel sample images with no surface defects, a plurality of third display panel sample images with surface defects, and defect annotation data corresponding to each of the third display panel sample images;

[0045] a training module, configured to train a to-be-trained model based on the training data set to obtain a display panel defect detection model; the to-be-trained model is configured to obtain feature maps of the first display panel sample image at different scales, and after processing the feature map using a cross-scale flow, output a likelihood value corresponding to the feature map, wherein the likelihood value is configured to indicate whether there is a defect on the surface of the first display panel sample image at a scale corresponding to the feature map;

[0046] A testing module is used to test the display panel defect detection model using the test data set, and adjust the model to be trained according to the test result, and continue to train the adjusted model to be trained based on the training data set until the trained display panel defect detection model meets the preset standard.

[0047] In a fourth aspect, an embodiment of the present application provides a display panel defect detection device, the device comprising:

[0048] An acquisition module, used for acquiring an image of a display panel to be inspected;

[0049] A detection module, used for inputting the image into a display panel defect detection model, and determining whether there is a defect on the surface of the display panel to be detected according to an output result of the display panel defect detection model;

[0050] Among them, the display panel defect detection model is trained by the display panel defect detection model training device provided by the third aspect.

[0051] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0052] The memory is used to store computer-executable instructions;

[0053] The processor is used to execute the computer-executable instructions stored in the memory to implement the display panel defect detection model training method provided in the first aspect; or to implement the display panel defect detection method provided in the second aspect.

[0054] In a sixth aspect, a computer-readable storage medium is provided in an embodiment of the present application, wherein the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the display panel defect detection model training method provided in the first aspect; or to implement the display panel defect detection method provided in the second aspect.

[0055] In the seventh aspect, a computer program product is provided in an embodiment of the present application, which includes a computer program, and when the computer program is executed by a processor, it implements the display panel defect detection model training method provided in the first aspect above; or implements the display panel defect detection method provided in the second aspect.

[0056] The display panel defect detection model training method, defect detection method and device provided in the embodiments of the present application can efficiently analyze and identify defects in display panel images by training a display panel defect detection model based on a fully convolutional cross-scale flow, thereby improving the accuracy and efficiency of display panel defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0058] Figure 1 A schematic diagram of a flow chart of a display panel defect detection model training method provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of the architecture of a cross-scale flow provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of a flow chart of a display panel defect detection method provided in an embodiment of the present application;

[0061] Figure 4A schematic diagram of a program module of a display panel defect detection model training device provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of a program module of a display panel defect detection device provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0064] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0066] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0067] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc.

[0068] The following is an explanation of some of the terms involved in the embodiments of the present application:

[0069] Cross-Scale Flow (CS-Flow): Aims to improve the performance and robustness of the model by jointly processing feature maps of different scales. It allows the model to consider both global information and local detail information when processing images, thereby more accurately identifying objects or defects in images.

[0070] In the manufacturing industry, the quality of display panels is one of the important factors affecting product appearance and user experience. High-quality display panels can not only enhance the visual appeal of products, but also bring users a more comfortable and pleasant experience.

[0071] According to the existing manufacturing technology, some defects may inevitably appear in the display panel during the manufacturing process, such as scratches, cracks, texture errors, displacement, etc. These defects not only affect the aesthetics of the product, but also reduce the quality and market competitiveness of the product. Therefore, it is very necessary to perform defect detection on the display panel during the manufacturing process of the display panel.

[0072] With the continuous development of intelligent manufacturing technology, the manufacturing industry is gradually shifting towards intelligence. Intelligent manufacturing technology not only has traditional manufacturing capabilities, but also has the capabilities of intelligent identification and automatic control. Therefore, combining advanced display panel defect detection technology with intelligent manufacturing technology can achieve real-time monitoring and automatic control of display panel quality, which in turn helps to improve product quality and manufacturing efficiency, and better meet consumers' needs for product quality and user experience.

[0073] However, the existing display panel defect detection method mainly relies on manual visual inspection, which is inefficient and also has the technical problem of low detection accuracy due to the influence of human subjective factors.

[0074] In addition, with the development of computer vision and deep learning technology, automated inspection methods based on image processing and machine learning have gradually begun to be applied. In particular, through the synthesis of multiple photos, high dynamic range imaging (HDR) and super-resolution reconstruction technology, ultra-high resolution images can be generated, which can more accurately reveal tiny scratches, subtle surface unevenness and other detailed defects.

[0075] For example, in some technical solutions, the following defect detection methods can be used to detect various defects:

[0076] Color difference: The deep convolutional neural network (DCNN) based on transfer learning can learn color distribution from large-scale image datasets and effectively distinguish normal from abnormal color differences.

[0077] Texture Errors: Use texture recognition algorithms such as Local Binary Patterns (LBP) combined with neural networks for reinforcement learning to identify irregular patterns and anomalies in textures.

[0078] Surface deformation: Use 3D reconstruction-based technology combined with time series analysis to track and evaluate surface changes during the production process, effectively detecting deformation caused by improper molding.

[0079] Missing parts: Applying image segmentation technology in deep learning, such as Mask R-CNN, can not only locate the specific position of parts, but also identify missing or damaged parts.

[0080] However, the above display panel defect detection method still has technical problems of low detection efficiency and detection accuracy.

[0081] In view of the above-mentioned technical problems, an embodiment of the present application provides a display panel defect detection model training method and a defect detection method. By training a display panel defect detection model based on fully convolutional cross-scale flow (CS-Flow), the display panel image can be efficiently analyzed and defects can be identified, thereby improving the accuracy and efficiency of display panel defect detection.

[0082] It should be noted that the method provided in the embodiments of the present application can be applied to the field of display panels, and can also be applied to other fields besides the field of display panels, including but not limited to image processing, medical image diagnosis, etc. It can help researchers to more accurately detect targets or defects in images and improve recognition accuracy and efficiency. This is not limited in the embodiments of the present application.

[0083] The technical solution shown in the present application is described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be described repeatedly in different embodiments.

[0084] Reference Figure 1 , Figure 1 1 is a flow chart of a display panel defect detection model training method provided in an embodiment of the present application. In some embodiments of the present application, the display panel defect detection model training method includes:

[0085] S101. Obtain a data set, which includes a training data set and a test data set.

[0086] In some embodiments, the training data set includes a plurality of sample images of the first display panel with no surface defects. Optionally, these images may cover display panels of different models, different batches, and different production conditions, so as to ensure that the display panel defect detection model can learn the characteristics of various defect-free display panels.

[0087] In some embodiments, the test data set includes a plurality of sample images of the second display panel with no surface defects, and these images can be used to verify the performance of the display panel defect detection model when detecting defect-free display panels.

[0088] In some embodiments, the test data set further includes a plurality of sample images of the third display panel with surface defects and defect annotation data corresponding to each sample image of the third display panel. These images may contain various types of defects, such as scratches, stains, bright spots, dark spots, etc., and there are sufficient samples for each defect.

[0089] Optionally, the defect annotation data may include at least one of the position, size, shape or type of the defect existing in the third display panel sample image.

[0090] In some embodiments, the above-mentioned multiple sample images of the third display panel with surface defects can be used to verify the ability of the display panel defect detection model in detecting defects, and the defect annotation data corresponding to each third display panel sample image can be used to evaluate the accuracy of the model in defect location and classification.

[0091] S102: Train the model to be trained based on the training data set to obtain a display panel defect detection model.

[0092] In some embodiments, the above-mentioned model to be trained is used to obtain feature maps of the first display panel sample image at different scales, and after processing these feature maps using a cross-scale flow (CS-Flow), output a likelihood value corresponding to the feature map, which is used to indicate whether there are defects on the surface of the first display panel sample image at the scale corresponding to the feature map.

[0093] In some embodiments, a convolutional neural network may be selected as a feature extractor to extract features from the input first display panel sample image. At the same time, a multi-scale feature extraction mechanism is introduced to obtain feature maps of the input first display panel sample image at different scales. This is because defects of the display panel may have different sizes and shapes, and cross-scale feature maps help the model to be trained to better identify these defects.

[0094] In some implementations, a cross-scale flow may be used to process the feature maps of different scales. Optionally, one or more fully connected layers (or convolutional layers) may be added at the output of the cross-scale flow to map the features to the final likelihood value output.

[0095] The likelihood value can be used to indicate whether there is a defect on the surface of the first display panel sample image at the scale corresponding to the feature map. During the training process, the model to be trained will continuously adjust its parameters to minimize the difference between the predicted likelihood value and the actual label (no defect / defective).

[0096] Among them, cross-scale flow utilizes the neural network model and regularized flow method in deep learning technology to realize the extraction and density estimation of image features, so as to efficiently analyze and identify defects in display panel images and improve the accuracy and efficiency of defect detection.

[0097] In deep learning, convolutional neural networks are a commonly used model architecture. Fully convolutional cross-scale flows can be implemented in this architecture, gradually extracting features from images through multiple layers of convolution and pooling operations. Features at different levels correspond to information at different scales, and cross-scale fusion can be achieved by fusing features at different levels. This can effectively process multi-scale feature maps to estimate their density, thereby performing defect detection or anomaly detection. Compared with traditional density estimation-based methods, fully convolutional cross-scale flows can process multi-scale feature maps, reduce information loss, and improve the flexibility of density estimation.

[0098] S103: Use a test data set to test the display panel defect detection model.

[0099] In some embodiments, the trained display panel defect detection model can be tested using a test data set. Optionally, the test process includes evaluating the false alarm rate of the display panel defect detection model on a non-defective display panel, as well as the detection rate, positioning accuracy, and classification accuracy on a defective display panel.

[0100] In some embodiments, the images in the test data set can be input into a trained display panel defect detection model, and the display panel defect detection model will output a likelihood value for each image, where the likelihood value indicates the possibility that a defect exists in the input image.

[0101] In some embodiments, the performance index of the display panel defect detection model can be evaluated based on the likelihood value output by the display panel defect detection model and the true label of the input image (whether it is defective).

[0102] Optionally, the above performance indicators may include accuracy, precision, recall and F1 score, etc.

[0103] S104, determining whether the display panel defect detection model meets the preset standard. If not, executing S105; if yes, executing S106.

[0104] S105. Adjust the above-mentioned model to be trained according to the test result, and return to execute S102 to continue training the adjusted model to be trained based on the training data set.

[0105] In some implementations, the above-mentioned model to be trained may be adjusted according to the test results, including modifying the structure of the model, adjusting hyperparameters, using more complex feature fusion strategies, etc.

[0106] After adjusting the above-mentioned model to be trained, continue to use the training data set to train the adjusted model to be trained, and repeat the above-mentioned testing and adjustment process until the trained display panel defect detection model meets the preset standard.

[0107] S106, outputting the above display panel defect detection model.

[0108] In some embodiments, once the trained display panel defect detection model meets the preset standard, it can be deployed in an actual production environment to detect in real time whether there are defects on the surface of the display panel.

[0109] The display panel defect detection model training method provided in the embodiment of the present application can efficiently analyze and identify defects in display panel images by training a display panel defect detection model based on a fully convolutional cross-scale flow, thereby improving the accuracy and efficiency of display panel defect detection.

[0110] In some embodiments, the above dataset may be prepared in advance in the following manner:

[0111] Capture images of multiple display panels and detect whether there are defects on the surface of the image; when there are no defects on the surface of the image, add the image to the above-mentioned training data set or test data set; when there are defects on the surface of the image, mark the defects on the surface of the image to obtain defect annotation data corresponding to the image, and add the image and the defect annotation data corresponding to the image to the test data set.

[0112] Optionally, the defect annotation data includes at least one of the position, size, shape or type of the defect in the image.

[0113] In some embodiments, the collected images may be preprocessed, including cropping, scaling, normalization, etc., to ensure consistency and processability of the input data. In addition, the images may be converted to grayscale or color images, and the size and resolution of the images may be adjusted according to the input requirements of the model.

[0114] In some embodiments, a large amount of display panel image data can also be collected and annotated based on existing public data sets, including defects of different sizes, shapes, and types such as cracks, scratches, and displacements, and a Monte Carlo-based method can be used to predict important shape features of the texture to construct an anomaly detection data set with multi-target and multi-defect data.

[0115] In some implementations, multiple photo synthesis technology may be used to create an ultra-high resolution display panel texture image to improve the accuracy of defect recognition.

[0116] In some implementations, texture features may also be extracted from the captured image; and the texture features may be added to the test data set.

[0117] In some embodiments, after the extracted texture features are added to the test data set as additional information, these texture features can be used together with the original image or defect annotation data to test the performance of the model.

[0118] Among them, the above texture features usually include directionality, roughness, periodicity, contrast, etc. When predicting important shape features, it is necessary to clarify which texture features are critical for shape recognition or anomaly detection.

[0119] In some embodiments, an image dataset containing various texture and shape features may be collected first, and then image processing techniques (such as Fourier transform, wavelet transform, gray level co-occurrence matrix, etc.) may be used to extract texture features from each image.

[0120] Optionally, during the simulation, randomness can be introduced to simulate the diversity of different textures.

[0121] In addition, during the simulation process, texture parameters related to important shape features can be adjusted to generate texture images with different shape features.

[0122] In some embodiments, in order to construct an anomaly detection dataset with multiple targets and multiple defects, different types of targets and defects can be introduced in the simulation process. For example, objects of different shapes, sizes, colors, and directions are randomly placed in the texture image as targets; or different types of defects (such as scratches, stains, damage, etc.) are simulated by changing certain parameters of the texture.

[0123] In some embodiments, a Monte Carlo algorithm can be used to simulate and generate a large number of images with different texture features, targets, and defects. Optionally, these images can be divided into two categories: normal samples and abnormal samples. Normal samples usually have common texture features and targets, while abnormal samples contain different types of defects or abnormal targets.

[0124] In some embodiments, the model to be trained includes a convolutional neural network and a cross-scale flow, and training the model to be trained based on the above training data set may include:

[0125] The convolutional neural network is used to obtain feature maps of the first display panel sample image at different scales; each feature map is input into the cross-scale stream; the cross-scale stream includes a plurality of coupling blocks, each coupling block includes a plurality of cross-convolution layers, and different cross-convolution layers are used to perform reversible transformation on feature maps at different scales; the likelihood value of the transformed feature map output by the cross-scale stream in the latent space is calculated.

[0126] In some embodiments, the type of the first display panel sample and / or the resolution or size of the first display panel sample image can be detected; based on the type of the first display panel sample and / or the resolution or size of the first display panel sample image, multiple different target scales corresponding to the first display panel sample image can be determined; and the above-mentioned convolutional neural network can be used to obtain a feature map of the first display panel sample image at each target scale.

[0127] Among them, when using a convolutional neural network to process the first display panel sample image, obtaining feature maps at different scales is crucial for subsequent image analysis, defect detection and other tasks. In some embodiments, the following steps can be used to obtain these feature maps:

[0128] S1.1. Detect the type of the first display panel sample and / or the resolution / size of the image.

[0129] Type detection: It is necessary to determine the type of the first display panel sample, such as LCD (liquid crystal display), OLED (organic light emitting diode display) or other types of display panels. Different types of display panels may have different physical properties and image features, which may need to be considered in subsequent scale selection and feature extraction.

[0130] Resolution / Size Detection: The resolution or size of the image is also key information. High-resolution images contain more details, while low-resolution images may lose some details. Based on the resolution or size of the image, it can be determined at which scales it makes sense to extract features.

[0131] S1.2. Determine multiple different target scales.

[0132] Depending on the type of the first display panel sample detected in step S1.1 and / or the resolution / size of the image, a range of different target scales can be determined. These scales should be able to cover the various scales of features that may exist in the image, from larger global features to smaller local features.

[0133] The choice of target scale can be based on experience, experimental data, or the requirements of a specific task. For example, for a defect detection task, it may be necessary to choose a scale that can capture the characteristics of the defect size.

[0134] S1.3. Use convolutional neural network to obtain feature maps.

[0135] Build or select a convolutional neural network model: Select a convolutional neural network model that is appropriate for the task. This model should have sufficient depth and complexity to capture multi-scale features in the image.

[0136] Multi-scale feature extraction: In convolutional neural network models, multi-scale feature extraction can be achieved in different ways. One possible approach is to use multiple parallel convolution paths, each of which processes images at different scales. Another approach is to use pooling layers or dilated convolutions in a single path to change the size of the receptive field to capture features of different scales.

[0137] Obtaining feature maps: At each target scale, the convolutional neural network model generates one or more feature maps. These feature maps encode information about the image at different scales and can be used for subsequent tasks such as classification, localization, or segmentation.

[0138] Post-processing: The extracted feature maps can be further processed and analyzed according to the specific task. For example, in the defect detection task, these feature maps can be used to train a classifier or regressor to identify defects in the image and estimate their size, location and other properties.

[0139] In other embodiments, scale indication information can be obtained, the scale indication information including multiple bits, and different bits correspond to different scales; based on the values ​​on each bit in the above scale indication information, multiple different target scales corresponding to the first display panel sample image are determined; and a convolutional neural network is used to obtain a feature map of the first display panel sample image at each target scale.

[0140] In some implementations, the following steps may be used to obtain these feature maps:

[0141] S2.1. Obtain scale indication information.

[0142] The scale indication information is a set of bits, where each bit represents a different scale. For example, if there are 4 bits, they may represent 4 different scales (such as small, medium, large, and extra large). These scale indication information can be pre-defined and determined based on the characteristics of the sample image of the display panel (such as size, resolution, possible defect size, etc.).

[0143] Optionally, the scale indication information may be manually set or automatically learned through a certain algorithm or model.

[0144] S2.2. Determine the target scale according to the scale indication information.

[0145] Traverse each bit in the scale indication information. If the value of a bit is 1 (or other value indicating an active state), the corresponding scale is used as one of the target scales. If the value of a bit is 0 (or other value indicating an inactive state), ignore the scale. Finally, a list of target scales corresponding to the activated bits will be obtained.

[0146] S2.3. Use convolutional neural network to obtain feature maps.

[0147] Prepare the convolutional neural network model and make sure it is suitable for processing the display panel sample images.

[0148] For each determined target scale, do the following: If the convolutional neural network model has a built-in multi-scale feature extraction mechanism (such as multi-scale input, multi-scale pooling, or dilated convolution), then directly input the image into the model and specify the current target scale.

[0149] If the convolutional neural network model does not have a built-in multi-scale feature extraction mechanism, it is necessary to modify the model structure or use specific techniques (such as feature pyramid networks) to capture features of different scales.

[0150] Input the image into the modified convolutional neural network model (or the original model) and obtain the feature map at the current target scale. Repeat the above process until the feature maps at all target scales are obtained.

[0151] Exemplarily, in some embodiments, a pre-trained convolutional neural network EfficientNet-B5 can be used as a feature extractor to extract features from an input image. Optionally, the input image can be first scaled to different scales (such as 768×768, 384×384, and 192×192), and then the 36th layer output of EfficientNet-B5 is used as a feature extractor to extract feature maps corresponding to the input image of each scale. Different feature maps can have different spatial resolutions (such as 24×24, 12×12, and 6×6).

[0152] In some embodiments, feature maps of different scales may be combined to form a feature list y=[y(1), y(2), y(3)], where y(i) represents the feature map at the i-th scale. The feature list y is then transformed using a cross-scale flow to obtain a latent space representation z.

[0153] In some implementations, the cross-scale flow may include four coupling blocks, each coupling block including a plurality of cross convolutional layers for performing reversible transformation on feature maps of different scales.

[0154] In some embodiments, in each coupling block, the scale and translation parameters of the feature map are estimated through the internal network to achieve reversible transformation of the feature map, and the information interaction and influence between feature maps of different scales are achieved through the cross convolution layer, so that a richer feature distribution can be learned. After multiple coupling blocks, the transformed feature maps are obtained, and these feature maps still retain spatial information.

[0155] In some embodiments, by calculating the likelihood value of the transformed feature map in the latent space, it can be determined whether the corresponding input image has an abnormality, such as a high likelihood value indicates that the image is normal, and a low likelihood value indicates that the image is abnormal.

[0156] In some implementations, since the feature map after the cross-scale flow transformation retains spatial information, the spatial information can be used to visualize the feature values ​​in the latent space to locate the abnormal area in the input image. Finally, the abnormality detection result and the positioning result are output.

[0157] For example, in the latent space, an anomaly detection mechanism can be designed to identify which feature values ​​or feature combinations are abnormal. Once the abnormal feature values ​​are identified, these abnormal values ​​can be mapped back to the spatial location of the input image. This can be achieved by upsampling the feature map (or processed feature map) to the resolution of the input image and superimposing or fusing them onto the original image. When superimposing, different colors or brightness can be used to indicate the degree of abnormality, thereby helping users to intuitively identify abnormal areas.

[0158] In some embodiments, during the model training process, the Adam optimizer can be used, and the learning rate is set to 2e -4 , weight decay is 1e -5 , and perform iterative training.

[0159] In some embodiments, during the model training process, a likelihood value may be assigned to the input image x by using the density estimate of the image features y, assuming that a low likelihood value indicates the presence of a defect. The density estimate is obtained by using the unknown distribution P of the feature space Y Y to have a Gaussian distribution P Z The latent space Z is learned by bijective mapping. Therefore, the process can be divided into feature extraction X→Y and density estimation Y→Z.

[0160] Specifically, the above model training process involves a bijective mapping that transforms the feature space Y (features extracted from image x) into a feature space with a known Gaussian distribution P ZA low likelihood value in the latent space Z indicates that the representation of the input image x in the feature space Y deviates from the normal data distribution, which may mean that there are defects in the image.

[0161] Among them, feature extraction X→Y includes: using a pre-trained model (such as a convolutional neural network) to forward propagate the input image x to extract the feature vector y.

[0162] Density estimation Y→Z involves learning a bijective mapping from feature space Y to latent space Z using a cross-scale flow.

[0163] Once the model is trained, it can be used to extract features y for a new input image x and calculate its corresponding representation z in the latent space Z. Then, the likelihood of z in the latent space can be calculated, and a low likelihood value indicates that z deviates from the distribution of normal data, which means that there is a defect in the input image x.

[0164] Among them, in order to convert the likelihood value into a binary anomaly detection result (i.e., whether the image contains defects), a threshold θ needs to be set. Optionally, the selection of the threshold θ can be determined by analyzing the distribution of likelihood values ​​of normal and abnormal images, and a value that can maximize the detection accuracy and recall rate can usually be selected.

[0165] For example, after extracting feature y from the input image x, the tensor can be reshaped by the us2Squeeze layer, for example, by aggregating the channels of 4 adjacent pixels into a pixel with 4 times the number of channels. Using the likelihood value of PZ(z), it is possible to determine whether the input image x is abnormal based on the threshold θ.

[0166] In the embodiment of the present application, the traditional normalization flow is extended with a new cross-scale flow to allow effective defect detection on the image. It processes the interacting feature maps of different sizes so that the information between the different scale communications is shared to obtain a compound. In addition, it is designed to be fully convolutional and maintain the spatial dimension so that the abnormal position in Z can be determined.

[0167] In some implementations, the cross-scale flow may be a chain of coupled blocks, each of which performs an affine transformation. Taking the number of scale divisions s=3 as an example, the specific structure is as follows: Figure 2 As shown, Figure 2 A schematic diagram of the architecture of a cross-scale flow provided in an embodiment of the present application.

[0168] exist Figure 2 In the “fixed” module, the input feature map (For example ) are randomly arranged, and the “uniform” module is used to divide the feature map evenly according to the channel dimension. (For example )and (For example ) parts. These parts manipulate each other by regressing element scale and displacement parameters, which are applied in turn to their respective counterparts to obtain the output [ ].For example, include include The "channel connection" module is used to merge features from different channels and output ,include .

[0169] In some embodiments, the scale and shift parameters are estimated by coupling block-independent subnetworks r1 and r2, whose outputs are split into [s1; t1] and [s2; t2].

[0170] In some embodiments:

[0171]

[0172]

[0173] The embodiment of the present application adopts a display panel defect detection model based on the CS-Flow algorithm, which can achieve efficient and accurate identification of surface defects of the display panel image, and can effectively improve the accuracy and efficiency of defect detection.

[0174] Based on the contents described in the above embodiments, a display panel defect detection method is also provided in the embodiments of the present application. Figure 3 , Figure 3 The following is a flow chart of a display panel defect detection method provided in an embodiment of the present application. In some embodiments of the present application, the display panel defect detection method includes:

[0175] S301 , collecting an image of a display panel to be inspected.

[0176] S302 , inputting the above image into a display panel defect detection model, and determining whether there are defects on the surface of the display panel to be detected according to an output result of the display panel defect detection model.

[0177] Among them, the above-mentioned display panel defect detection model can be trained by the display panel defect detection model training method described in the above-mentioned embodiment, and will not be repeated in the embodiments of the present application.

[0178] In some implementations, if a defect is found in the display panel to be inspected, an alarm may be issued in a timely manner or automatic processing may be performed, thereby ensuring product quality and production efficiency.

[0179] The display panel defect detection method provided in the embodiment of the present application includes the following beneficial results compared with the prior art:

[0180] By adopting a display panel defect detection model based on fully convolutional cross-scale flow, real-time automated detection of display panels can be achieved. There is no need to invest a large amount of manpower in manual visual inspection, which not only reduces labor costs and production costs, but also helps to improve the accuracy and efficiency of defect detection.

[0181] Based on the contents described in the above embodiments, in some embodiments of the present application, when it is determined that there are defects on the surface of the display panel to be inspected, the positioning information output by the display panel defect detection model is obtained, and the positioning information includes the abnormal area where there are defects on the surface of the display panel to be inspected; the position, size or type information of the above abnormal area is marked on the above image and recorded in the database.

[0182] In some embodiments, when the display panel defect detection model detects that there is a defect on the surface of the display panel to be detected, the model will output corresponding positioning information. The positioning information generally describes the position of the defect in the image.

[0183] Optionally, the positioning information may be bounding box coordinates (i.e., upper left corner and lower right corner coordinates of the defect area), center point and radius, polygon vertex coordinates, etc., depending on the design and output format of the model.

[0184] In some implementations, the abnormal area where the defect is located can be marked on the original image according to the positioning information output by the model, such as by drawing a rectangular box, polygon or other shapes on the image.

[0185] Optionally, visual attributes such as the color and line width of the annotation can be configured as needed to ensure that the annotation is clear and easy to identify.

[0186] In some embodiments, key information such as the location, size or type of the abnormal area is recorded in a database. Such information can be helpful for subsequent quality assessment, defect analysis and production process improvement.

[0187] Optionally, the records in the database may include fields such as image file name, defect type, defect location (such as coordinates), defect size (such as area or diameter), detection time, etc.

[0188] Based on the contents described in the above embodiments, in some embodiments of the present application, when it is determined that there are defects on the surface of the display panel to be inspected, the defects are classified according to the information output by the display panel defect detection model and the preset classification algorithm; the quality of the display panel to be inspected is evaluated according to the severity of the defects, and a corresponding quality grade is assigned to it; according to the type of defects and / or the quality grade of the display panel to be inspected, a processing strategy corresponding to the display panel to be inspected is selected; the processing strategy includes: automatically repairing the display panel to be inspected, or triggering a quality alarm signal.

[0189] In some embodiments, when the display panel defect detection model detects a defect, detailed information about the defect, such as location, size, shape, etc., may be output. The defect may be classified using a preset classification algorithm based on this information and possible defect features (such as color, texture, brightness, etc.). Common defect types may include bright spots, dark spots, scratches, stains, etc.

[0190] Optionally, the above classification algorithm can be rule-based (such as threshold judgment) or machine learning-based (such as using support vector machines, random forests, neural networks, etc.).

[0191] The display panels are evaluated for quality based on the severity of the defects (e.g., quantity, size, type, etc.). For example, a clear quality evaluation standard can be developed to classify the display panels into different quality levels, such as excellent, good, fair, poor, etc. Then, a preset evaluation algorithm or model is used to assign the corresponding quality level to the display panels to be inspected in combination with the defect information.

[0192] Depending on the type of defect and / or the quality level of the display panel to be inspected, the corresponding processing strategy is selected. For example, if the defect type can be repaired by automatic repair technology (such as certain types of stains or scratches), the display panel can be automatically repaired. If the defect is serious or cannot be repaired, or the quality level of the display panel is low, a quality alarm signal can be triggered for manual inspection or further processing.

[0193] In some implementations, defect information, classification results, quality levels, and processing strategies may also be recorded in a database for subsequent analysis and statistics.

[0194] Based on the contents described in the above embodiments, the present application also provides a display panel defect detection model training device, referring to Figure 4 , Figure 4 Schematic diagram of a program module of a display panel defect detection model training device provided in an embodiment of the present application. In some embodiments, the display panel defect detection model training device 40 includes:

[0195] The acquisition module 401 is used to acquire a data set, wherein the data set includes a training data set and a test data set; the training data set includes a plurality of first display panel sample images with no surface defects, and the test data set includes a plurality of second display panel sample images with no surface defects, a plurality of third display panel sample images with surface defects, and defect annotation data corresponding to each of the third display panel sample images.

[0196] The training module 402 is used to train the model to be trained based on the training data set to obtain a display panel defect detection model; the model to be trained is used to obtain the feature map of the first display panel sample image at different scales, and after processing the feature map using a cross-scale flow, output a likelihood value corresponding to the feature map, and the likelihood value is used to indicate whether there is a defect on the surface of the first display panel sample image at a scale corresponding to the feature map.

[0197] The testing module 403 is used to test the display panel defect detection model using the test data set, adjust the model to be trained according to the test results, and continue to train the adjusted model to be trained based on the training data set until the trained display panel defect detection model meets the preset standard.

[0198] In some embodiments, the model to be trained includes a convolutional neural network and a cross-scale flow, and the training module 402 is used to:

[0199] Using the convolutional neural network to obtain feature maps of the first display panel sample image at different scales;

[0200] Inputting each of the feature maps into the cross-scale stream; the cross-scale stream comprises a plurality of coupling blocks, each of the coupling blocks comprises a plurality of cross-convolution layers, and different cross-convolution layers are used to perform reversible transformation on the feature maps at different scales;

[0201] Calculating the likelihood value of the transformed feature map output by the cross-scale flow in the latent space;

[0202] The parameters of the model to be trained are adjusted according to the likelihood value.

[0203] In some embodiments, the step of acquiring feature maps of the first display panel sample image at different scales using the convolutional neural network includes:

[0204] Detecting the type of the first display panel sample and / or the resolution or size of the first display panel sample image;

[0205] Determine a plurality of different target scales corresponding to the first display panel sample image according to the type of the first display panel sample and / or the resolution or size of the first display panel sample image;

[0206] The convolutional neural network is used to obtain a feature map of the first display panel sample image at each of the target scales.

[0207] In some embodiments, the step of acquiring feature maps of the first display panel sample image at different scales using the convolutional neural network includes:

[0208] Acquire scale indication information, where the scale indication information includes a plurality of bits, and different bits correspond to different scales;

[0209] Determining a plurality of different target scales corresponding to the first display panel sample image according to values ​​of respective bits in the scale indication information;

[0210] The convolutional neural network is used to obtain a feature map of the first display panel sample image at each of the target scales.

[0211] In some embodiments, the transformed feature map retains spatial information; the training module 402 is further used to:

[0212] The spatial information is used to visualize feature values ​​in the latent space to locate abnormal regions in the first display panel sample image.

[0213] In some embodiments, the above device further comprises a pre-processing module, which is used to:

[0214] Collecting images of a plurality of display panels and detecting whether there are defects on the surfaces of the images;

[0215] When there is no defect in the surface of the image, adding the image to the training data set or the test data set;

[0216] When there are defects in the surface of the image, the defects on the surface of the image are marked to obtain defect marking data corresponding to the image, and the image and the defect marking data corresponding to the image are added to the test data set; the defect marking data includes at least one of the position, size, shape or type of the defects existing in the image.

[0217] In some embodiments, the preprocessing module is further used to:

[0218] Extracting texture features from the collected image;

[0219] The texture features are added to the test dataset.

[0220] It should be noted that the specific execution contents and beneficial effects of the acquisition module 401, the training module 402 and the testing module 403 in the embodiment of the present application can be referred to the various steps in the display panel defect detection model training method described in the above embodiment, which will not be repeated here.

[0221] Based on the contents described in the above embodiments, a display panel defect detection device is also provided in the embodiments of the present application. Figure 5 , Figure 5 Schematic diagram of a program module of a display panel defect detection device provided in an embodiment of the present application. In some embodiments, the display panel defect detection device 50 includes:

[0222] The acquisition module 501 is used to acquire the image of the display panel to be detected.

[0223] A detection module 502, configured to input the image into a display panel defect detection model, and determine whether there is a defect on the surface of the display panel to be detected according to an output result of the display panel defect detection model;

[0224] The display panel defect detection model is obtained by training the display panel defect detection model training device provided in the above embodiment.

[0225] In some embodiments, the display panel defect detection device 50 further includes a processing module, which is used to:

[0226] When it is determined that there is a defect on the surface of the display panel to be inspected, obtaining positioning information output by the display panel defect detection model, the positioning information including an abnormal area where there is a defect on the surface of the display panel to be inspected;

[0227] The position, size or type information of the abnormal area is marked on the image and recorded in a database.

[0228] In some embodiments, the processing module is further used to:

[0229] When it is determined that there are defects on the surface of the display panel to be inspected, classifying the defects according to information output by the display panel defect detection model and a preset classification algorithm;

[0230] Performing a quality assessment on the display panel to be inspected according to the severity of the defect and assigning a corresponding quality grade to it;

[0231] According to the type of the defect and / or the quality grade of the display panel to be detected, a processing strategy corresponding to the display panel to be detected is selected; the processing strategy includes: automatically repairing the display panel to be detected, or triggering a quality alarm signal.

[0232] It should be noted that the specific execution contents and beneficial effects of the acquisition module 501 and the detection module 502 in the embodiment of the present application can be referred to the various steps in the display panel defect detection method described in the above embodiment, and will not be repeated here.

[0233] Furthermore, based on the contents described in the above embodiments, an electronic device is also provided in an embodiment of the present application, which includes at least one processor and a memory; wherein the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory to implement the display panel defect detection model training method or each step in the display panel defect detection method as described in the above embodiments.

[0234] In order to better understand the embodiments of the present application, refer to Figure 6 , Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0235] like Figure 6 As shown, the electronic device 60 of this embodiment includes: a processor 601 and a memory 602; wherein:

[0236] Memory 602, used to store computer-executable instructions;

[0237] The processor 601 is used to execute the computer-executable instructions stored in the memory to implement the various steps in the display panel defect detection model training method or the display panel defect detection method described in the above embodiments.

[0238] Optionally, the memory 602 may be independent or integrated with the processor 601 .

[0239] When the memory 602 is independently provided, the device further includes a bus 603 for connecting the memory 602 and the processor 601 .

[0240] An embodiment of the present application also provides a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed by a processor, they are used to implement the display panel defect detection model training method or each step in the display panel defect detection method as described in the above embodiments.

[0241] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it can implement the various steps in the display panel defect detection model training method or the display panel defect detection method described in the above embodiments.

[0242] In the several embodiments provided in the present application, it should be understood that the disclosed devices and systems can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0243] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0244] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0245] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application.

[0246] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0247] The memory may include high-speed memory, and may also include non-volatile storage, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0248] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0249] The storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The storage medium may be any available medium that can be accessed by a general or special purpose computer.

[0250] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A display panel defect detection model training method, characterized in that: The method comprises: Acquire a data set, the data set comprising a training data set and a test data set; the training data set comprises a plurality of first display panel sample images with no surface defects, the test data set comprises a plurality of second display panel sample images with no surface defects, a plurality of third display panel sample images with surface defects, and defect annotation data corresponding to each of the third display panel sample images; The model to be trained is trained based on the training data set to obtain a display panel defect detection model; the model to be trained is used to obtain feature maps of the first display panel sample image at different scales, and after processing the feature map using a cross-scale flow, output a likelihood value corresponding to the feature map, wherein the likelihood value is used to indicate whether there is a defect on the surface of the first display panel sample image at a scale corresponding to the feature map; Using the test data set to test the display panel defect detection model, and adjusting the model to be trained according to the test result, and continuing to train the adjusted model to be trained based on the training data set until the trained display panel defect detection model meets the preset standard; The model to be trained includes a convolutional neural network and a cross-scale flow, and the training of the model to be trained based on the training data set includes: Inputting the first display panel sample image into the convolutional neural network to obtain feature maps of the first display panel sample image at different scales; Inputting each of the feature maps into the cross-scale stream; the cross-scale stream comprises a plurality of coupling blocks, each of the coupling blocks comprises a plurality of cross-convolution layers, and different cross-convolution layers are used to perform reversible transformation on the feature maps at different scales; Calculating the likelihood value of the transformed feature map output by the cross-scale flow in the latent space; Adjusting the parameters of the model to be trained according to the likelihood value; Also includes: Detecting the type of the first display panel sample and / or the resolution or size of the first display panel sample image; determining a plurality of different scales corresponding to the first display panel sample image according to the type of the first display panel sample and / or the resolution or size of the first display panel sample image; or, Acquire scale indication information, the scale indication information including a plurality of bits, and different bits correspond to different scales; determine a plurality of different scales corresponding to the first display panel sample image according to values ​​of each bit in the scale indication information; Part of the third display panel sample images among the plurality of third display panel sample images are obtained based on the following method: Collect and annotate multiple display panel image data with different defects; Processing the plurality of display panel image data with different defects based on a Monte Carlo algorithm to predict important shape features of texture; the important shape features are key factors for shape recognition or anomaly detection; Based on the important shape features of the texture, a plurality of images with different texture features, targets and defects are simulated and generated; wherein the plurality of images with different texture features, targets and defects are the sample images of the part of the third display panel; The simulation generates a plurality of images with different texture features, targets and defects based on the important shape features of the texture, including: By adjusting the texture parameters related to the important shape features, a texture image with different shape features is simulated and generated; or, By adjusting the texture parameters related to the important shape features, a texture image with different shape features is simulated and generated; objects of different shapes, sizes, colors and directions are randomly placed in the texture image as targets; or, different types of defects are simulated by adjusting the texture parameters.

2. The method according to claim 1, characterized in that The transformed feature map retains spatial information; the method further comprises: The spatial information is used to visualize feature values ​​in the latent space to locate abnormal regions in the first display panel sample image.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Collecting images of a plurality of display panels and detecting whether there are defects on the surfaces of the images; When there is no defect in the surface of the image, adding the image to the training data set or the test data set; When there are defects in the surface of the image, the defects on the surface of the image are marked to obtain defect marking data corresponding to the image, and the image and the defect marking data corresponding to the image are added to the test data set; the defect marking data includes at least one of the position, size, shape or type of the defects existing in the image.

4. A display panel defect detection method, characterized in that: The method comprises: Acquire an image of the display panel to be inspected; Inputting the image into a display panel defect detection model, and determining whether there is a defect on the surface of the display panel to be detected according to an output result of the display panel defect detection model; Wherein, the display panel defect detection model is trained by the display panel defect detection model training method according to any one of claims 1 to 3.

5. The method according to claim 4, characterized in that The method further comprises: When it is determined that there is a defect on the surface of the display panel to be inspected, obtaining positioning information output by the display panel defect detection model, the positioning information including an abnormal area where there is a defect on the surface of the display panel to be inspected; The position, size or type information of the abnormal area is marked on the image and recorded in a database.

6. The method according to claim 5, characterized in that The method further comprises: When it is determined that there are defects on the surface of the display panel to be inspected, classifying the defects according to information output by the display panel defect detection model and a preset classification algorithm; Performing a quality assessment on the display panel to be inspected according to the severity of the defect and assigning a corresponding quality grade to it; According to the type of the defect and / or the quality grade of the display panel to be detected, a processing strategy corresponding to the display panel to be detected is selected; the processing strategy includes: automatically repairing the display panel to be detected, or triggering a quality alarm signal.

7. A display panel defect detection model training device, characterized in that: The device comprises: an acquisition module, used to acquire a data set, wherein the data set includes a training data set and a test data set; the training data set includes a plurality of first display panel sample images with no surface defects, and the test data set includes a plurality of second display panel sample images with no surface defects, a plurality of third display panel sample images with surface defects, and defect annotation data corresponding to each of the third display panel sample images; a training module, configured to train a to-be-trained model based on the training data set to obtain a display panel defect detection model; the to-be-trained model is configured to obtain feature maps of the first display panel sample image at different scales, and after processing the feature map using a cross-scale flow, output a likelihood value corresponding to the feature map, wherein the likelihood value is configured to indicate whether there is a defect on the surface of the first display panel sample image at a scale corresponding to the feature map; A testing module, configured to test the display panel defect detection model using the test data set, adjust the model to be trained according to the test result, and continue to train the adjusted model to be trained based on the training data set until the trained display panel defect detection model meets a preset standard; The model to be trained includes a convolutional neural network and a cross-scale flow, and the training module is specifically used for: Inputting the first display panel sample image into the convolutional neural network to obtain feature maps of the first display panel sample image at different scales; Inputting each of the feature maps into the cross-scale stream; the cross-scale stream comprises a plurality of coupling blocks, each of the coupling blocks comprises a plurality of cross-convolution layers, and different cross-convolution layers are used to perform reversible transformation on the feature maps at different scales; Calculating the likelihood value of the transformed feature map output by the cross-scale flow in the latent space; Adjusting the parameters of the model to be trained according to the likelihood value; The training module is also specifically used for: Detecting the type of the first display panel sample and / or the resolution or size of the first display panel sample image; determining a plurality of different scales corresponding to the first display panel sample image according to the type of the first display panel sample and / or the resolution or size of the first display panel sample image; or, Acquire scale indication information, the scale indication information including a plurality of bits, and different bits correspond to different scales; determine a plurality of different scales corresponding to the first display panel sample image according to values ​​of each bit in the scale indication information; The acquisition module is specifically used for: Collect and annotate multiple display panel image data with different defects; Processing the plurality of display panel image data with different defects based on a Monte Carlo algorithm to predict important shape features of texture; the important shape features are key factors for shape recognition or anomaly detection; Based on the important shape features of the texture, a plurality of images with different texture features, targets and defects are simulated and generated; wherein the plurality of images with different texture features, targets and defects are part of the sample images of the third display panel; Wherein, when the acquisition module is used to simulate and generate multiple images with different texture features, targets and defects based on the important shape features of the texture, it is specifically used to: By adjusting the texture parameters related to the important shape features, a texture image with different shape features is simulated and generated; or, By adjusting the texture parameters related to the important shape features, a texture image with different shape features is simulated and generated; objects of different shapes, sizes, colors and directions are randomly placed in the texture image as targets; or, different types of defects are simulated by adjusting the texture parameters.

8. A display panel defect detection device, characterized in that: The device comprises: An acquisition module, used for acquiring an image of a display panel to be inspected; A detection module, used for inputting the image into a display panel defect detection model, and determining whether there is a defect on the surface of the display panel to be detected according to an output result of the display panel defect detection model; Wherein, the display panel defect detection model is trained by the display panel defect detection model training device described in claim 7.

9. An electronic device, characterized in that: include: Processor and memory; The memory is used to store computer-executable instructions; The processor is used to execute the computer-executable instructions stored in the memory to implement the display panel defect detection model training method as described in any one of claims 1 to 3; or to implement the display panel defect detection method as described in any one of claims 4 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the display panel defect detection model training method as described in any one of claims 1 to 3; or to implement the display panel defect detection method as described in any one of claims 4 to 6.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the display panel defect detection model training method as described in any one of claims 1 to 3 is implemented; or the display panel defect detection method as described in any one of claims 4 to 6 is implemented.

Citation Information

Cited By

  • Automatic repair systems, display module manufactured thereby, electronic device manufactured thereby

    EP4838460A1