System and method for line defect detection using preprocessing

Abstract

A system and method for line defect detection using preprocessing. The system is configured to generate a filtered image by preprocessing an input image of a display using at least one filter. The system then identifies line mura candidates by converting the filtered image into a binary image, counting line components along the slope in the binary image, and marking potential candidate positions when the number of counts of line components along the slope exceeds a line threshold. Image patches are then generated, with the candidate position located at the center of each image patch. The image patches are then classified using a machine learning classifier.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 634,125, filed on February 22, 2018, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] Some embodiments of the present disclosure generally relate to display defect detection systems and methods. Background Art

[0004] As display resolution and pixel density increase, the difficulty of performing defect detection also increases. Manual defect detection is too time-consuming for modern manufacturing facilities, while automated inspection techniques are often ineffective. For example, in automated surface inspection, defects in uniform (e.g., non-textured) surfaces can be easily identified when the local anomaly has a sharp contrast with its regular surrounding neighborhood. However, defects in low-contrast images are difficult to detect when the defect does not have a clear edge with its surroundings and the background exhibits uneven lighting.

[0005] A common type of display defect is "Mura". Mura is a large category of defects with local brightness non-uniformity. Mura can be roughly classified into line mura, point mura and area mura according to its size and general shape. Each type of mura may not have obvious edges and may not be obvious in the image. Therefore, it has proven difficult to identify mura using automated test systems in the past. Therefore, a new method for identifying mura defects is needed.

[0006] The above information is only for enhancement of understanding of the background of the embodiments of the present disclosure and therefore may contain information that does not constitute prior art. Summary of the invention

[0007] Some embodiments of the present disclosure provide a system and method for defect detection in a display. In various embodiments, the system includes a memory, and a processor configured to execute instructions stored on the memory. In various embodiments, the instructions cause the processor to: generate a filtered image by preprocessing an input image of the display using at least one filter; and identify line mura candidates. In various embodiments, the system can identify line mura candidates by converting the filtered image into a binary image, counting the line components along the slope in the binary image, and marking the candidate object position when the number of counts of the line components along the slope exceeds the line threshold. In various embodiments, the instructions also cause the processor to: generate an image patch, wherein the candidate object position is located at the center of the image patch; and use a machine learning classifier to classify the image patch.

[0008] In various embodiments, the at least one filter comprises at least one of a Gabor filter and a pyramid filter.

[0009] In various embodiments, converting the filtered image to a binary image includes: determining a mean image intensity and a standard deviation of the filtered image; calculating a binary threshold based on the mean image intensity and the standard deviation; and converting the filtered image to a binary image according to the binary threshold.

[0010] In various embodiments, the line threshold is equal to 70% of the total number of line components along the slope.

[0011] In various embodiments, image patches are generated using an input image.

[0012] In various embodiments, the machine learning classifier comprises a support vector machine.

[0013] In various embodiments, the instructions further cause the processor to extract a feature vector from the image patch, wherein the feature vector includes at least one image moment.

[0014] In various embodiments, the instructions further cause the processor to normalize the input image.

[0015] In various embodiments, a method for identifying defects in a display includes: generating a filtered image by preprocessing an input image of the display using at least one filter; and identifying line mura candidates. In various embodiments, identifying line mura candidates includes: converting the filtered image to a binary image; counting line components along a slope in the binary image; and marking a candidate object location when the number of counts of the line components along the slope exceeds a line threshold. In various embodiments, the method also includes: generating an image patch, wherein the candidate object location is located at the center of the image patch; and using a machine learning classifier to classify the image patch.

[0016] In various embodiments, the at least one filter comprises at least one of a Gabor filter and a pyramid filter.

[0017] In various embodiments, converting the filtered image to a binary image includes: determining a mean image intensity and a standard deviation of the filtered image; calculating a binary threshold based on the mean image intensity and the standard deviation; and converting the filtered image to a binary image according to the binary threshold.

[0018] In various embodiments, the line threshold is equal to 70% of the total number of line components along the slope.

[0019] In various embodiments, image patches are generated using an input image.

[0020] In various embodiments, the machine learning classifier comprises a support vector machine.

[0021] In various embodiments, the method further comprises: extracting a feature vector of the image patch, wherein the feature vector comprises at least one image moment.

[0022] In various embodiments, a method for identifying defects in a display includes: normalizing an input image of the display to a desired data range; generating a filtered image by preprocessing the normalized input image using at least one of a Gabor filter and a pyramid filter; and identifying line mura candidates. In various embodiments, identifying line mura candidates includes: converting the filtered image to a binary image according to a binary threshold; counting line components along at least one slope in the binary image; and marking a candidate object location when the count number of line components along at least one slope exceeds the line threshold. In various embodiments, the method also includes: generating at least one image patch, wherein the candidate object location is located at the center of the image patch; and classifying the at least one image patch using a machine learning classifier.

[0023] In various embodiments, converting the filtered image to a binary image includes: determining a mean image intensity and a standard deviation of the filtered image; calculating a binary threshold based on the mean image intensity and the standard deviation; and converting the filtered image to a binary image according to the binary threshold.

[0024] In various embodiments, the binary threshold is equal to the standard deviation divided by 1.5 plus the mean image intensity.

[0025] In various embodiments, the machine learning classifier comprises a support vector machine.

[0026] In various embodiments, the method further comprises: extracting a feature vector of the image patch, wherein the feature vector comprises at least one image moment. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Some embodiments may be understood in more detail according to the following description in conjunction with the accompanying drawings, in which:

[0028] Figure 1A depicts a system overview according to various embodiments of the present disclosure;

[0029] Figure 1B An overview of a system for training a classifier according to various embodiments of the present disclosure is depicted;

[0030] Figure 2 Depicted are methods of classifying images according to various embodiments of the present disclosure;

[0031] Figure 3A depicts a system overview according to various embodiments of the present disclosure;

[0032] Figure 3B Describes the use of various embodiments according to the present disclosure Figure 3A A method for classifying images by a system;

[0033] Figure 4 depicts a preprocessor according to various embodiments of the present disclosure;

[0034] Figure 5A A candidate object detector according to various embodiments of the present disclosure is depicted, and Figure 5B Candidate object detection methods according to various embodiments of the present disclosure are described. DETAILED DESCRIPTION

[0035] By referring to the detailed description and accompanying drawings of the following embodiments, the features of the inventive concept and its implementation method can be more easily understood. Hereinafter, the embodiments will be described in more detail with reference to the accompanying drawings, wherein the same reference numerals always refer to the same elements. However, the present disclosure can be embodied in various different forms and should not be considered to be limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples to make the present disclosure sufficient and complete, and to fully convey the aspects and features of the present disclosure to those skilled in the art. Therefore, for those of ordinary skill in the art, processes, elements and techniques that are not necessary for a complete understanding of these aspects and features of the present disclosure may not be described. Unless otherwise stated, throughout the drawings and written descriptions, the same reference numerals represent the same elements, and thus their descriptions will not be repeated. In the drawings, for clarity, the relative sizes of elements, layers and regions may be exaggerated.

[0036] In the following description, for the purpose of explanation, many specific details are set forth to provide a thorough understanding of the various embodiments. However, it is apparent that the various embodiments can be practiced without these specific details or with one or more equivalent arrangements. In other instances, well-known structures and devices are shown in block diagram form to avoid unnecessary confusion of the various embodiments.

[0037] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular form "one" is intended to also include the plural form, unless the context clearly indicates otherwise. It will be further understood that when used in this specification, the terms "include", "have" and "comprise" specify the presence of stated features, integers, steps, operations, elements and / components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0038] As used herein, when describing embodiments of the present disclosure, the use of "may" refers to "one or more embodiments of the present disclosure." As used herein, the terms "use" and "used" may be considered synonymous with the terms "utilize" and "utilized," respectively. In addition, the term "exemplary" means an example or illustration.

[0039] When a specific embodiment can be implemented in different ways, the specific processing order can be performed differently from the described order. For example, two consecutively described processes can be performed substantially simultaneously, or in the reverse order of the described order.

[0040] The electronic device or electrical device and / or any other related device or component according to the embodiments of the present disclosure described herein can be implemented using any appropriate hardware, firmware (e.g., application specific integrated circuit), software, or a combination of software, firmware, and hardware. For example, the various components of these devices can be formed on an integrated circuit (IC) chip or a separate IC chip. In addition, the various components of these devices can be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on a substrate. In addition, the various components of these devices can be processes or threads that run on one or more processors in one or more computing devices, execute computer program instructions, and interact with other system components to perform various functions described herein. Computer program instructions are stored in a memory implemented in a computing device using a standard storage device, such as, for example, a random access memory (RAM). Computer program instructions can also be stored in other non-temporary computer-readable media, such as, for example, a CD-ROM, a flash drive, etc. In addition, those skilled in the art should recognize that the functions of various computing devices can be combined or integrated into a single computing device, or the functions of a specific computing device can be distributed across one or more other computing devices without departing from the spirit and scope of the exemplary embodiments of the present disclosure.

[0041] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as those commonly understood by ordinary technicians in the technical field to which the present disclosure belongs. It will be further understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant field and / or in this specification, and should not be interpreted in an idealized or overly formal sense unless explicitly defined in this article. Embodiments of the present disclosure include systems and methods for performing Mura detection on a display. In various embodiments, the system is configured to identify instances of horizontal line Mura. In various embodiments, an input image of a display for displaying a test image is received. Then, the input image is divided into image patches. In various embodiments, the system can pre-process the image using a candidate object detector, which identifies an area of ​​the display with a defect candidate object (e.g., a horizontal line) and generates an image patch based on the location of the defect candidate object. Then, the system uses a machine learning classifier to classify the image patch into an instance with or without horizontal line Mura.

[0042] Figure 1A A system overview according to various embodiments of the present disclosure is depicted. Figure 1B An overview of a system for training a classifier according to various embodiments of the present disclosure is depicted. Figure 2Methods of classifying images according to various embodiments of the present disclosure are described.

[0043] refer to Figure 1A , Figure 1B and Figure 2 In various embodiments, the Mura detection system receives an input image at the preprocessor 100 (step 200). The input image may, for example, include an image of a display displaying a test image. A camera may be used to generate an input image by taking a photo of an OLED displaying a test image. In various embodiments, the test image may include an image of an instance that may cause the display to exhibit horizontal line Mura. For example, the test image may be a uniform image exhibiting a low level of contrast. The input image may also have a sufficiently high resolution to show individual pixels of the display being inspected for defects (e.g., line Mura). In various embodiments, the preprocessor 100 may be configured to receive an input image and perform smoothing on the input image to reduce noise in the image. After reducing the noise in the input image, the preprocessor 100 may be configured to divide the image into a plurality of image patches (step 210). Each of the image patches may then be supplied to the feature extractor 110.

[0044] In various embodiments, the feature extractor 110 is configured to calculate various statistical features of the supplied image patch (step 220). For example, the statistical features may include: one or more image moments (e.g., a weighted average of pixel intensities), one or more texture measurements (e.g., texture analysis using a gray level co-occurrence matrix (GLCM)), or any other statistical features of the input image. For example, in various embodiments, contrast, correlation, energy, uniformity from a GLCM, and moments such as mu03, mu10, hu1, hu5, etc. are extracted by the feature extractor 110.

[0045] In various embodiments, the extracted statistical features of each image patch are supplied as input to the classifier 120 (230). In various embodiments, the classifier 120 is a machine learning classifier that uses the extracted features (e.g., feature vectors) and label classification information to identify instances of defects (e.g., mura) (240). In various embodiments, the classification information is supplied by training the classifier.

[0046] In various embodiments, the classifier 120 uses a supervised learning model and is therefore trained before taking effect. In some embodiments, the supervised learning model used in the classifier 120 is a support vector machine. The supervised learning model (e.g., a support vector machine) can be trained by providing artificial input 130 to the classifier 120 during the training phase. For example, for each image patch, one can visually inspect the patch and mark any instances of white spot Mura. The image patch is also provided to the feature extractor 110. The feature vectors extracted for the image patch and the corresponding patches that are manually inspected and marked are provided to the classifier 120. The classifier 120 generates classification information (i.e., builds a model) using the patches provided for later classification.

[0047] Figure 3A A system overview according to various embodiments of the present disclosure is depicted. Figure 3B Describes the use of various embodiments according to the present disclosure Figure 3A A method for classifying images using a system.

[0048] refer to Figure 3A and Figure 3B In various embodiments, the line mura classification system may employ preprocessing and candidate object detection in order to identify potential instances of line mura. Preprocessing and using a candidate object detector can reduce the total number of image patches analyzed by the classifier, at the expense of increasing the preclassification processing time. Although the preclassification processing time is increased, the overall running time can be significantly reduced due to the reduction in subsequent classification time. For example, when compared with the sliding window image patch generation technique, the total processing time can be reduced by orders of magnitude due to the reduction in the number of image patches that need to be classified. In addition, the use of preprocessing and candidate object detection can also improve classifier training. For example, in various embodiments, the reduction of image patches allows for reduced human intervention during training, thereby reducing human errors provided to the classifier.

[0049] In various embodiments, the system receives an input image at the normalization module 300. In various embodiments, the input image is normalized to a desired data range (step 350). For example, in various embodiments, the normalization may be a linear normalization as described in Equation 1.

[0050]

[0051] Where gMax and gMin are the global maximum and minimum values ​​of the entire image dataset, respectively. input is the input image, and I norm is the normalized image. In this example, normalization causes the image intensity to be inverted.

[0052] In various embodiments, the normalized image is provided to a preprocessor 310, which uses one or more filters to filter the normalized image to make any defect instances (e.g., horizontal line Mura) more obvious while also reducing image noise (step 360). For example, in various embodiments, one or more filters may be configured to converge the energy in the image toward the centerline. In various embodiments, the preprocessed image is provided to a Mura candidate detector 320, which identifies potential Mura candidates (step 370) and generates image patches at the candidate locations (step 380). In various embodiments, image patches are generated, wherein potential Mura candidates are located at the center of each patch. The image patches are provided to a feature extractor 330, which generates a feature vector (e.g., as described above) for each image patch and a classifier 340 uses each feature vector to classify each image patch (step 390).

[0053] Figure 4 A pre-processor according to various embodiments of the present disclosure is depicted.

[0054] refer to Figure 4 In various embodiments, the preprocessor 400 is configured to filter the normalized image to enhance the line mura features, thereby facilitating candidate object detection and classification. For example, in various embodiments, the preprocessor 400 may be configured to remove noise and converge energy toward the center line. In various embodiments, the preprocessor 400 includes a Gabor filter 410 and a pyramid filter 420. The Gabor filter 410 is used to analyze the image frequency content in a specific direction (e.g., angle θ). In various embodiments, different θ values ​​may be used to adjust the Gabor filter 410 to detect different angles of line mura. For example, line mura may not be limited to a completely horizontal or vertical direction. Therefore, the Gabor filter 410 may be used more than once for the image and use different parameters. For example, the horizontal line may be tilted more than 1 degree. In various embodiments, the Gabor filter 410 may include a value of θ of 1 degree and a value including σ equal to 4, λ equal to 19 and γ equal to 0.5. The parameters of the Gabor filter 410 (e.g., θ, σ, λ, and γ) may also be further adjusted to filter out different frequencies or other factors. In various embodiments, the filter size may also be adjusted. For example, in various embodiments, the Gabor filter 410 may have a size of 31×31 pixels. Adjustment of the filter parameters may be done automatically through an iterative process or modified by the user.

[0055] In various embodiments, the pyramid filter 420 may be configured to increase image smoothing at the expense of image resolution. In various embodiments, the pyramid filter 420 is a Gaussian pyramid filter (e.g., a multi-resolution Gaussian pyramid filter). For example, in various embodiments, the Gaussian pyramid filter 420 may be used to generate a four-layer Gaussian pyramid for a preprocessed input image. A 5×5 window filter may then be applied to each layer, and the layers may be merged back into a single filtered image. In various embodiments, the number of layers and size of the window filter may be adjusted according to the input image size. In various other embodiments, other types of pyramid filters may be used. For example, in various other embodiments, the pyramid filter 420 may be a Laplacian pyramid filter, a steerable pyramid filter, or other types of pyramid filters. In various other embodiments, the pyramid filter 420 may be replaced by one or more other filters having a similar effect on the input image. For example, in various embodiments, the pyramid filter 420 may be replaced by a plurality of Gaussian smoothing filters. Each of the Gaussian smoothing filters may have a different distribution from one another, and the output of each of the Gaussian smoothing filters may be averaged.

[0056] Figure 5A A candidate object detector according to various embodiments of the present disclosure is depicted, and Figure 5B Candidate object detection methods according to various embodiments of the present disclosure are described.

[0057] refer to Figure 5A and Figure 5B , the candidate object detector 500 can be used to identify potential instances of line mura (e.g., horizontal line mura). In various embodiments, the candidate object detector 500 is configured to count binary image components along the line. When the number of components along the line is greater than the line threshold, a potential instance of line mura is identified. In various embodiments, the candidate object detector 500 may include an average and standard deviation module 510 that determines the average image intensity and the standard deviation (step 501). In various embodiments, the binary conversion module 520 converts the input image into a binary image (step 502). The binary conversion module 520 can use a binary threshold set with an average value modified by the standard deviation. For example, in various embodiments, the threshold can be defined by Formula 2.

[0058]

[0059] In various embodiments, the candidate object detector 500 may use the binary image to identify potential instances of line mura. For example, the line component counting module 530 may count the length of line segments along the slope (e.g., components marked as "1" along the slope) (step 503). For example, in various embodiments, for each line in the input image, the line component counting module 530 may count the number of segments marked as "1" along a slope of 0 (e.g., a horizontal line) in the binary image. In another example, the line component counting module 530 may count along slopes of ±0.5, ±1.0, ±1.5, etc. In various embodiments, when the number of line segments counted along the slope exceeds a line threshold, a candidate object may be detected. In various embodiments, the threshold comparison module 540 compares each number of counted line components with a corresponding line threshold for the relevant slope (step 504). For example, the line threshold may be set to 70% of the total number of line components along the slope. The QHD display has a resolution of 2560×1440. Thus, in this example, for a horizontal line (e.g., with a slope of 0), the line threshold may be set to 1000 components. In various embodiments, the candidate object marker 550 marks the position and orientation of each identified candidate object (step 505). The original image may then be used to generate an image patch for each identified candidate object and provided to the feature extractors 110 and 330 and the classifiers 120 and 340 as described above. In various embodiments, the size of the image patch may be determined based on the display size. For example, in various embodiments, the display may be a QHD display having a resolution of 2560×1440, and the image patch may have a size of 43×1408 pixels. In other embodiments, the display may have a resolution of 1480×720, and the image patch may have a size of 21×704 pixels.

[0060] Therefore, the above embodiments of the present disclosure provide systems and methods for identifying instances of line mura on a display panel. In various embodiments, preprocessing and candidate object detection can reduce the number of candidate object image patches to be classified, thereby reducing the overall processing time while improving detection accuracy. Image patches can be generated with mura candidates located at the center of the patches, and by creating patches with lines located at the center of the patches, the system provides a clear data set for training and classification.

[0061] The foregoing is an illustration of example embodiments and should not be construed as limiting thereof. Although several example embodiments have been described, those skilled in the art will readily appreciate that many modifications may be made to the example embodiments without materially departing from the novel teachings and advantages of the example embodiments. Therefore, all such modifications are intended to be included within the scope of the example embodiments as defined in the claims. In the claims, means-plus-function clauses are intended to cover structures described herein as performing the functions, not just structural equivalents, but also equivalent structures. Therefore, it should be understood that the foregoing is an illustration of example embodiments and should not be construed as limited to the specific embodiments disclosed, and modifications to the disclosed example embodiments as well as other example embodiments are intended to be included within the scope of the appended claims. The inventive concept is defined by the appended claims, including the equivalents of the claims.

Claims

1. A system for identifying defects in a display, the system comprising: Memory; a processor configured to execute instructions stored on the memory, the instructions, when executed by the processor, causing the processor to: generating a filtered image by preprocessing an input image to the display using at least one filter; Line mura candidates are identified by: converting the filtered image into a binary image; counting line components along the slope in the binary image; and marking a candidate object location when the number of counts along the line component of the slope exceeds a line threshold; generating an image patch, wherein the candidate object position is located at the center of the image patch; as well as using a machine learning classifier to classify the image patches, Wherein, converting the filtered image into a binary image comprises: determining a mean image intensity and a standard deviation of the filtered image; calculating a binary threshold based on the mean image intensity and the standard deviation; and The filtered image is converted into the binary image according to the binary threshold.

2. The system according to claim 1, wherein: The at least one filter includes at least one of a Gabor filter and a pyramid filter.

3. The system according to claim 1, wherein: The line threshold is equal to 70% of the total number of the line components along the slope.

4. The system according to claim 1, wherein: The image patches are generated using the input image.

5. The system according to claim 1, wherein: The machine learning classifier includes a support vector machine.

6. The system according to claim 1, wherein: The instructions further cause the processor to extract a feature vector from the image patch, wherein the feature vector includes at least one image moment.

7. The system according to claim 1, wherein: The instructions further cause the processor to normalize the input image.

8. A method for identifying defects in a display, comprising: generating a filtered image by preprocessing an input image to the display using at least one filter; Identify line mura candidate objects, where the identification of line mura candidate objects includes: converting the filtered image into a binary image; counting line components along the slope in the binary image; and marking a candidate object location when the number of counts along the line component of the slope exceeds a line threshold; generating an image patch, wherein the candidate object position is located at the center of the image patch; and using a machine learning classifier to classify the image patches, Wherein, converting the filtered image into a binary image comprises: determining a mean image intensity and a standard deviation of the filtered image; calculating a binary threshold based on the mean image intensity and the standard deviation; and The filtered image is converted into the binary image according to the binary threshold.

9. The method according to claim 8, wherein: The at least one filter includes at least one of a Gabor filter and a pyramid filter.

10. The method according to claim 8, wherein: The line threshold is equal to 70% of the total number of the line components along the slope.

11. The method according to claim 8, wherein: The image patches are generated using the input image.

12. The method according to claim 8, wherein: The machine learning classifier includes a support vector machine.

13. The method according to claim 8, further comprising: A feature vector of the image patch is extracted, wherein the feature vector includes at least one image moment.

14. A method for identifying defects in a display, comprising: Normalizing the input image of the display to a desired data range; generating a filtered image by preprocessing the normalized input image using at least one of a Gabor filter and a pyramid filter; Identify line mura candidate objects, wherein identifying line mura candidate objects includes: converting the filtered image into a binary image according to a binary threshold; counting line components along at least one slope in the binary image; and marking a candidate object location when the number of counts of the line component along the at least one slope exceeds a line threshold; generating at least one image patch, wherein the candidate object position is located at the center of the image patch; and classifying the at least one image patch using a machine learning classifier, Wherein, converting the filtered image into a binary image comprises: determining a mean image intensity and a standard deviation of the filtered image; calculating the binary threshold based on the mean image intensity and the standard deviation; and The filtered image is converted into the binary image according to the binary threshold.

15. The method according to claim 14, wherein: The binary threshold is equal to the standard deviation divided by 1.5 plus the mean image intensity.

16. The method according to claim 14, wherein: The machine learning classifier includes a support vector machine.

17. The method according to claim 14, further comprising: A feature vector of the image patch is extracted, wherein the feature vector includes at least one image moment.

Images