A method for detecting centerline defects on LCD screens and extracting defect information

Abstract

The present invention discloses a method, system, and computer-readable storage medium for detecting and extracting linear defects in LCD screens. The method comprises: S1. acquiring an image to be inspected and performing ROI image cropping on the image to extract the actual area to be inspected in the image; S2. Gaussian filtering the obtained ROI image using filter kernels of different sizes to obtain a foreground image and a background image of the ROI image; S3. performing image difference on the foreground image and the background image to obtain a foreground-background separation image; S4. performing Canny edge detection on the foreground-background separation image to obtain a Canny detection image with the defects outlined; S5. performing a progressive probabilistic Hough line transform on the Canny detection image to obtain linear defect information; and S6. re-clustering the extracted defect information to obtain accurate defect information. This invention improves the accuracy and efficiency of linear defect detection in screens.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and more specifically, to a method, system, and computer-readable storage medium for detecting midline defects and extracting defect information on LCD screens. Background Art

[0002] Linear defect detection for LCD screens is a crucial component of defect detection. Industry requirements require not only the ability to detect and annotate defects in the original image, but also the ability to extract line defect information, primarily including location, length, and width. This extracted defect information serves as a valuable reference for improving and adjusting manufacturing processes. Consequently, industry requirements for defect detection have evolved from simply detecting the presence of defects to also include the ability to extract defect information.

[0003] Existing line defect detection methods can only detect the presence of defects, but cannot extract defect information. There are two general approaches to detecting line defects: one involves separating the foreground and background of the original image, binarizing it, and then performing defect contour detection on the binarized image. This method can detect defects, but for intersecting lines, contour detection only yields a single outline. Therefore, to address this issue, erosion is often performed after binarization: horizontal erosion extracts vertical lines, and vertical erosion extracts horizontal lines. However, erosion significantly interferes with defect information extraction. Erosion works better for horizontal and vertical lines, but has inconsistent results for diagonal lines of varying angles. Therefore, this method is not suitable for defect information extraction. Another approach involves performing Canny edge detection on the image after foreground and background separation, followed by a Hough line transform to obtain a set of line segments. However, due to the initial Gaussian filtering and differencing of the image, as well as the color gradient of the defect edges, the defect edges detected by Canny edge detection are not smooth. Consequently, the Hough line transform may extract multiple collinear line segments, hindering defect information extraction.

[0004] The prior art discloses a fast corridor vanishing point detection algorithm based on the K-means method. After obtaining the real-time image data returned by the robot, it is first pre-processed, including downsampling, grayscale processing, histogram equalization, and Canny edge detection. The probabilistic Hough transform algorithm is then used to extract lines from the image obtained in the previous step. Finally, the K-means algorithm is used to cluster the detected lines into four categories based on their slopes. The mean of the midpoint of each cluster is calculated. The slopes and midpoint means obtained from the clustering are then used to construct four lines to replace the lines detected in the previous step. The four lines are then randomly divided into two groups, and their intersection points are calculated. The midpoint of the intersection is used as the vanishing point. Although this prior art solution uses the K-means method, it focuses on point detection and does not involve line detection. Summary of the Invention

[0005] The present invention provides a method, system and computer-readable storage medium for detecting centerline defects of an LCD screen and extracting defect information, thereby improving the accuracy and efficiency of centerline defect detection of the screen.

[0006] The primary purpose of the present invention is to solve the above technical problems, and the technical solutions of the present invention are as follows:

[0007] A first aspect of the present invention provides a method for detecting centerline defects on an LCD screen and extracting defect information, comprising the following steps:

[0008] S1. Obtain the image to be detected and perform ROI image cropping on the image to be detected to extract the actual area to be detected in the image;

[0009] S2. Gaussian filtering is performed on the obtained ROI image using filter kernels of different sizes to obtain the foreground and background images of the ROI image;

[0010] S3. Perform image difference on the foreground image and the background image to obtain a front-back separation image;

[0011] S4. Perform Canny edge detection on the foreground and background separation map to obtain a Canny detection map of the defect stroked;

[0012] S5. Perform progressive probabilistic Hough line transform on the Canny detection image to obtain line defect information;

[0013] S6. Re-cluster the extracted defect information to obtain accurate defect information.

[0014] Furthermore, the ROI image cropping includes two methods: one is manual cropping, and the other is calculating the outline of the outermost edge of the screen and then cropping the ROI image according to the outline.

[0015] Further, the different-sized filter kernels in step S2 include: a 5×5 small kernel and a 105×105 large kernel. The 5×5 small kernel is used to eliminate the influence of moiré patterns as much as possible, and the 105×105 large kernel is used to obtain a background image with balanced colors.

[0016] Further, in step S4, Canny edge detection is performed by setting double thresholds, denoted as high threshold H and low threshold L. Here, the size of the high threshold is a set multiple of the low threshold. The specific detection is as follows:

[0017] If the gradient value t of any pixel in the image is t > H, it is retained;

[0018] If the gradient value t of any pixel in the image is t < L, it is discarded;

[0019] If the gradient value L ≤ t ≤ H of any pixel in the image, search for the pixel gradient value in the neighborhood of this pixel. If there is a pixel gradient value higher than the high threshold, it is retained; if not, it is discarded;

[0020] Depict the defect edges based on the retained pixels.

[0021] Further, the specific process of performing progressive probabilistic Hough line transform on the Canny detection image is as follows:

[0022] S501. Randomly select a feature point in the Canny detection image, that is, an edge point. If this edge point has been marked as a point on a certain straight line, continue to randomly select an edge point from the remaining edge points until all edge points have been selected;

[0023] S502. Perform Hough transform on the selected edge points and calculate the cumulative sum;

[0024] S503. Select the point with the largest value in the Hough space. If this point is greater than the given threshold, proceed to step S504; otherwise, return to S501;

[0025] S504. Based on the maximum value obtained from the Hough transform, displace along the direction of the straight line starting from this maximum value point to find the two endpoints of the straight line;

[0026] S505. Calculate the length of the straight line. If it is greater than the given threshold, it is considered a good straight line and output, then return to step S501;

[0027] S506. Output the set of line segments represented by the two line segment endpoints.

[0028] Further, the specific steps for reclustering the extracted defect information to obtain accurate defect information are as follows:

[0029] S601. Calculate the angle θ (θ∈{θ|-90°<θ<90°}) of each line segment in the line segment set, and then cluster the line segments by angle to obtain clusters of line segments classified by angle; among them, line segments with different angles are obviously not on the same line.

[0030] S602. Project the line segments in the cluster obtained in step S601 into the coordinate system. Line segments with an angle of 0 are projected onto the Y axis, and line segments with an angle of 90 are projected onto the X axis. For line segments with angles between (0, 90) and (-90, 0), the coordinate system is first rotated, the coordinates of the rotated points are calculated, and then the points are projected into the coordinate system.

[0031] S603. For line segments with an angle of (90, -90), first rotate the coordinate system and then project it onto the rotated coordinate system;

[0032] S604. Clustering the projected points;

[0033] S605. Process the clustered line segments to obtain defect information of the line defects.

[0034] Furthermore, the coordinate rotation steps are as follows:

[0035] In the original coordinate system XoY, if the angle θ of the line segment is (0,90), then rotate XoY counterclockwise around the origin by θ degrees to become the coordinate system SoT; if the angle θ of the line segment is (-90,0), rotate it clockwise around the origin by θ degrees to become the coordinate system SoT;

[0036] The coordinate rotation formula is as follows:

[0037] Suppose there is a point p, whose coordinates in the original coordinate system are (x, y), and whose new coordinates after rotation are (s, t), then:

[0038] s=x·cos(θ)+y·sin(θ)

[0039] t=y·cos(θ)-x·sin(θ)

[0040] Calculate the coordinates of the endpoints of the line segment in the rotated coordinate system SoT, and then project them onto the SoT coordinate axis to obtain the projection point (s, t).

[0041] A second aspect of the present invention provides a system for detecting and extracting centerline defects in LCD screens. The system includes a memory and a processor. The memory includes a program for detecting and extracting centerline defects in LCD screens. When the program is executed by the processor, the following steps are implemented:

[0042] S1. Obtain the image to be detected and perform ROI image cropping on the image to be detected to extract the actual area to be detected in the image;

[0043] S2. Gaussian filtering is performed on the obtained ROI image using filter kernels of different sizes to obtain the foreground and background images of the ROI image;

[0044] S3. Perform image difference on the foreground image and the background image to obtain a front-back separation image;

[0045] S4. Perform Canny edge detection on the foreground and background separation map to obtain a Canny detection map of the defect stroked;

[0046] S5. Perform progressive probabilistic Hough line transform on the Canny detection image to obtain line defect information;

[0047] S6. Re-cluster the extracted defect information to obtain accurate defect information.

[0048] Furthermore, the ROI image cropping includes two methods: one is manual cropping, and the other is calculating the outline of the outermost edge of the screen and then cropping the ROI image according to the outline.

[0049] The third aspect of the present invention provides a computer-readable storage medium, which includes a program for a method for detecting centerline defects and extracting defect information on an LCD screen. When the program for a method for detecting centerline defects and extracting defect information on an LCD screen is executed by a processor, the steps of the method for detecting centerline defects and extracting defect information on an LCD screen are implemented.

[0050] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0051] The present invention extracts line segment defect information after Hough line transformation by using a clustering method, and obtains accurate defect information by changing the measurement method of the relationship between line segments in the clustering algorithm, that is, converting the line segments into projection points on the coordinate axis by projection, and using the distance of the projection points as the measurement method for measuring the relationship between line segments, thereby improving the accuracy and efficiency of line defect detection in the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of a method for detecting centerline defects and extracting defect information on an LCD screen according to the present invention.

[0053] Figure 2 The camera of the embodiment of the present invention takes images.

[0054] Figure 3 This is the ROI cropping diagram of an embodiment of the present invention.

[0055] Figure 4 This is a foreground diagram of an embodiment of the present invention.

[0056] Figure 5 This is a background diagram of an embodiment of the present invention.

[0057] Figure 6 This is a diagram showing the separation of the front and background of an embodiment of the present invention.

[0058] Figure 7 This is a Canny edge detection image according to an embodiment of the present invention.

[0059] Figure 8 Schematic diagram of a set of defective line segments obtained by the progressive probabilistic Hough line transform according to an embodiment of the present invention.

[0060] Figure 9 FIG. 4 is a diagram of the clustering process according to an embodiment of the present invention.

[0061] Figure 10 Graph showing the detection results of an embodiment of the present invention.

[0062] Figure 11 Schematic diagram of points to be clustered using the existing K-means clustering algorithm in an embodiment of the present invention.

[0063] Figure 12 Schematic diagram of line segment projection according to an embodiment of the present invention.

[0064] Figure 13 Schematic diagram of coordinate transformation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0066] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0067] Example 1

[0068] like Figure 1 As shown, the first aspect of the present invention provides a method for detecting centerline defects of an LCD screen and extracting defect information, comprising the following steps:

[0069] S1. Obtain the image to be detected and perform ROI image cropping on the image to be detected to extract the actual area to be detected in the image;

[0070] It should be noted that in actual industrial production scenarios, the field of view of the camera often exceeds the actual size of the screen. In order to capture the entire area of the screen, the captured image will include not only the screen but also the background outside the screen. The image captured by the camera is as Figure 2 shown. Due to the presence of the background outside the screen, it will affect the detection algorithm. Therefore, the screen area in the image is often cropped, and only the ROI image of the screen is given to the detection algorithm for detection. In this invention, the ROI image cropping includes two types: one is manual cropping, and the other is to calculate the outermost contour of the screen and then crop the ROI image according to the contour.

[0071] S2. Use Gaussian filters with different sizes to perform Gaussian filtering on the obtained ROI image to obtain the foreground image and background image of the ROI image;

[0072] It should be noted that due to the influence of the grating and the lighting environment during camera imaging, the finally captured image will have moiré patterns. Therefore, a Gaussian filter with a small kernel is used to尽可能消除摩尔纹的影响。而采用大核的高斯滤波则是为了获取一张颜色均衡的背景图,本发明中不同尺寸的滤波核包括:5×5的小核,105×105的大核。

[0073] S3. Perform image difference on the foreground image and the background image to obtain a foreground-background separation image;

[0074] It should be noted that since there is an obvious color difference between the defects in the ROI image and the surrounding area, foreground-background difference is used. When the image is processed, it can be represented as a matrix of pixel values. The principle of foreground-background separation is to perform subtraction on a pixel matrix of the same size, that is, subtract the pixel values at the same position. Since the background image is obtained by performing Gaussian filtering with a large kernel on the ROI image, the pixel value matrix of the background image can be regarded as a pixel matrix with the same pixel value size. Therefore, after subtraction, the pixel value at the defect position will not be 0, and the pixel value at the non-defect position will be 0.

[0075] S4. Perform Canny edge detection on the foreground-background separation image to obtain a Canny detection image with the defects outlined;

[0076] It should be noted that the purpose of Canny edge detection is to outline the defects to facilitate the extraction of defects by the progressive probabilistic Hough line transform in the next step. Canny edge detection is performed by setting double thresholds, which are denoted as the high threshold H and the low threshold L. Among them, the size of the high threshold is 2-3 times that of the low threshold. The specific detection is as follows: If the gradient value t of any pixel in the image > H, it is retained;

[0077] If the gradient value t of any pixel in the image < L, it is discarded;

[0078] If the gradient value of any pixel in the image is L<=t<=H, then the pixel gradient value is searched from the pixel's neighborhood. If there is a pixel gradient value higher than the high threshold, it is retained; if not, it is discarded.

[0079] Delineate the defect edges based on the retained pixels.

[0080] S5. Perform progressive probabilistic Hough line transform on the Canny detection image to obtain line defect information;

[0081] It should be noted that the specific process of performing progressive probabilistic Hough line transform on the Canny detection image is as follows:

[0082] S501. Randomly extract a feature point, i.e., an edge point, from the Canny detection image. If the edge point has been calibrated as a point on a straight line, then continue to randomly extract an edge point from the remaining edge points until all edge points are extracted.

[0083] S502. Perform Hough transform on the extracted edge points and perform cumulative sum calculation;

[0084] S503. Select the point with the largest value in the Hough space. If the point is greater than a given threshold, proceed to step S504; otherwise, return to S501.

[0085] S504. Based on the maximum value obtained by the Hough transform, shift along the direction of the straight line from the maximum value point to find the two endpoints of the straight line;

[0086] S505. Calculate the length of the line. If it is greater than a given threshold, it is considered a good line and output, and return to step S501.

[0087] S506. Output a set of line segments represented by two line segment endpoints.

[0088] S6. Re-cluster the extracted defect information to obtain accurate defect information.

[0089] It should be noted that K in the K-means clustering algorithm is a very important parameter, which means how many clusters your algorithm can eventually be divided into.

[0090] K-means is an iterative process. The algorithm is roughly divided into four steps:

[0091] ① Randomly select K objects in the data space as initial centers, each object represents a cluster center.

[0092] ② For the data objects in the sample, according to their Euclidean distance from these cluster centers, they are divided into the class corresponding to the cluster center closest to them (most similar) according to the criterion of the closest distance.

[0093] ③ Update the cluster center: calculate the mean corresponding to all objects in each category and use it as the next cluster center of the category.

[0094] ④Whether the values ​​of this cluster center and the next cluster center change. If they do not change, it means that the algorithm has converged. If they change, re-cluster with the cluster center calculated in the third step. Figure 11 The figure shows the points to be clustered using the existing K-means clustering algorithm. The coordinates of the points are shown in Table 1.

[0095] Table 1

[0096]

[0097]

[0098] From the table above, we can see that these 6 points can be divided into two clusters, so:

[0099] 1. Select K = 2 initial centers, i.e. centroids, then select A and B

[0100] 2. Calculate the distance between other points and the initial center

[0101] The distance from C to A can also be seen from the diagram (Pythagorean theorem), which is √10=3.16;

[0102] The distance between C and B is √((3-1)²+(1-2)²=√5=2.24, so C is closer to B, and C is in the same cluster with B.

[0103] Similarly, D, E, and F are calculated in the same way. The positions of the points after the first iteration are as shown in Table 2:

[0104] Table 2

[0105] A B C 3.16 2.24 D 11.3 9.22 E 13.5 11.3 F 12.2 10.3

[0106] Results after the first grouping

[0107] Cluster 1:A

[0108] Cluster 2: B, C, D, E, F

[0109] 3. Calculate the new center

[0110] The center of cluster 1 is still A=(0,0)

[0111] The new center coordinates of cluster 2 are: P = ((1+3+8+9+10) / 5, (2+1+8+10+7) / 5) = (6.2, 5.6)

[0112] The positions of the points after the second iteration are shown in Table 3 below:

[0113] Table 3

[0114] A P B 2.23 6.32 C 3.16 4.6 D 11.3 3 E 13.5 5.22 F 12.2 4.05

[0115] Second grouping results

[0116] Cluster 1: A, B, C

[0117] Cluster 2: D, E, F

[0118] 4. Calculate the center of mass again

[0119] P1=(1.33,1)

[0120] P2=(9,8.33)

[0121] The positions of the points after the third iteration are shown in Table 4 below:

[0122] Table 4

[0123] P1 P2 A 1.64 12.26 B 1.05 10.20 C 1.67 9.47 D 9.64 1.05 E 11.80 1.63 F 10.42 1.66

[0124] Results of the third grouping

[0125] Cluster 1: P1, P2, P3

[0126] Cluster 2: P4, P5, P6

[0127] The third grouping result matches the second, indicating convergence and clustering is complete. The above iterative process demonstrates the importance of choosing the value of K. To ensure that the clustering algorithm can generate an appropriate number of clusters even when the value of K is uncertain, a reference quantity—variance—is needed. Variance, in probability theory and statistics, measures the degree of dispersion of a random variable or a set of data. Therefore, variance can be used to measure the sparsity of a cluster: smaller variances mean denser clusters, while larger variances mean looser clusters. The data we want to cluster has a relatively small variance. Therefore, during clustering, we can also calculate the variance of each cluster. After clustering converges, we calculate the average variance of the clusters. If the average variance is less than the set variance, we consider K to be an appropriate value. If the calculated average variance is greater than the set variance, we increase K+1 and re-cluster until the average variance is less than the set variance.

[0128] B. Distance Metrics

[0129] When clustering line segments, another issue worth considering is how to describe the relationships between them. Clearly, Euclidean distance is insufficient to describe these relationships, so a new approach is needed. Using the previous example of clustering point clusters, this paper converts line segments into points for clustering.

[0130] The present invention re-clusters the extracted defect information to obtain accurate defect information. The specific steps are as follows:

[0131] S601. Calculate the angle θ (θ∈{θ|-90°<θ<90°}) of each line segment in the line segment set, and then cluster the line segments by angle to obtain clusters of line segments classified by angle; among them, line segments with different angles are obviously not on the same line.

[0132] S602. Project the line segments in the cluster obtained in step S601 into the coordinate system. Line segments with an angle of 0 are projected onto the Y axis, and line segments with an angle of 90 are projected onto the X axis. For line segments with angles at (0, 90) or (-90, 0), the coordinate system is first rotated to obtain the coordinates of the points after the rotation, and then projected onto the coordinate system. Taking the projection of the line segment with an angle of 0 as an example, Figure 12 shown.

[0133] S603. For the line segment with an angle of (90, -90), the coordinate system is first rotated and then projected onto the rotated coordinate system; the line segment after the coordinate system rotation is represented by the coordinate rotation formula.

[0134] The coordinate rotation steps are as follows:

[0135] In the original coordinate system XoY, if the angle θ of the line segment is (0,90), then rotate XoY counterclockwise around the origin by θ degrees to become the coordinate system SoT; if the angle θ of the line segment is (-90,0), rotate it clockwise around the origin by θ degrees to become the coordinate system SoT;

[0136] The coordinate rotation formula is as follows:

[0137] like Figure 13 As shown, suppose there is a point p, whose coordinates in the original coordinate system are (x, y) and whose new coordinates after rotation are (s, t), then:

[0138] oa=y·si n (θ) (Equation 2.1)

[0139] as=x·cos(θ) (Equation 2.2)

[0140] Combining the above two formulas, we can get:

[0141] s=os=oa+as=x·cos(θ)+ysin(θ) (Equation 2.3)

[0142] t=ot=ay-ab=y·cos(θ)-x·sin(θ) (Equation 2.4)

[0143] The coordinates of the endpoints of the line segment in the rotated coordinate system SoT are calculated, and then projected onto the SoT coordinate axis to obtain the projection point (s, t).

[0144] S604. Clustering the projected points;

[0145] S605. Process the clustered line segments to obtain defect information of the line defects.

[0146] Example 2

[0147] A second aspect of the present invention provides a system for detecting and extracting centerline defects in LCD screens. The system includes a memory and a processor. The memory includes a program for detecting and extracting centerline defects in LCD screens. When the program is executed by the processor, the following steps are implemented:

[0148] S1. Obtain the image to be detected and perform ROI image cropping on the image to be detected to extract the actual area to be detected in the image;

[0149] S2. Gaussian filtering is performed on the obtained ROI image using filter kernels of different sizes to obtain the foreground and background images of the ROI image;

[0150] S3. Perform image difference on the foreground image and the background image to obtain a front-back separation image;

[0151] S4. Perform Canny edge detection on the foreground and background separation map to obtain a Canny detection map of the defect stroked;

[0152] S5. Perform progressive probabilistic Hough line transform on the Canny detection image to obtain line defect information;

[0153] S6. Re-cluster the extracted defect information to obtain accurate defect information.

[0154] Furthermore, the ROI image cropping includes two methods: one is manual cropping, and the other is calculating the outline of the outermost edge of the screen and then cropping the ROI image according to the outline.

[0155] The third aspect of the present invention provides a computer-readable storage medium, which includes a program for a method for detecting centerline defects and extracting defect information on an LCD screen. When the program for a method for detecting centerline defects and extracting defect information on an LCD screen is executed by a processor, the steps of the method for detecting centerline defects and extracting defect information on an LCD screen are implemented.

[0156] Example 3

[0157] This embodiment is described in detail in conjunction with the specific detection process.

[0158] Place the product to be inspected under the camera, adjust the distance between the product and the camera so that the product is exactly in the camera's field of view, then turn on the power to light up the screen, and then use the camera to capture images and transfer the captured images to the computer. The captured images are as follows: Figure 2 Start detection.

[0159] Figure 2 The black area in the image is the background outside the screen, which will affect the actual detection and needs to be removed. The screen area can be cropped out through ROI cropping to facilitate the subsequent detection process. The effect after cropping is as follows Figure 3 shown.

[0160] Since the moiré patterns generated during the image acquisition process will interfere with subsequent detection, the image is processed with a Gaussian filter with a filter kernel of 5×5, and the filtered result image is used as the foreground image in the subsequent image difference operation. Figure 4 As shown:

[0161] For the subsequent image difference operation, we also need to obtain the background image. We also perform a Gaussian filter on the foreground image, but this time the filter kernel is 105×105. Figure 5 As shown:

[0162] Subtracting the foreground image from the background image can yield a foreground-background separation image. Performing Canny edge detection on the foreground-background separation image can outline line defects. Figure 6 As shown, the Canny edge detection diagram is as follows Figure 7 shown.

[0163] Performing a progressive probabilistic Hough line transform on the Canny edge detection image can obtain a set of line segments. Figure 6 For example, ideally, the result of Canny edge detection should be two smooth straight lines. However, due to some image processing and color gradient of defect edges during the detection process, Figure 6The actual Canny edge detection does not produce two smooth straight lines, but two jagged straight lines. Then the progressive probabilistic Hough line transform detects these two non-smooth straight lines and extracts multiple line segments. The line segment set is as follows: Figure 8 As shown:

[0164] The line segment set obtained by the progressive probabilistic Hough line transform is marked on the ROI image. The line defect detection result is shown in the figure below. Figure 9 As shown:

[0165] In order to extract the number of line defects and the defect information of each line defect from such a disordered set, the set is clustered, and the clustering convergence variance is set to 20.

[0166] Before clustering begins, the projected points of line segments at the same angle are read. These projected points correspond one-to-one with the line segments, and these points serve as the clustering dataset. Clustering is performed three times, and the best result (i.e., the lowest mean square error) is selected. The initial K for clustering is 1. If the clustering converges but does not meet the set variance, k is increased by 1 and clustering is restarted.

[0167] like Figure 10 The defect detection result is shown in Figure 5. After clustering, the defect information of each line set is extracted, and the extracted defect information is shown in Table 5.

[0168] Table 5

[0169]

[0170] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting and extracting midline defects on an LCD screen, characterized in that: It includes the following steps: S1. Obtain the image to be detected, and perform ROI image cropping on the image to be detected to extract the actually detected area in the image; S2. Use Gaussian filters with different sizes of filter kernels to perform Gaussian filtering on the obtained ROI image to obtain the foreground image and background image of the ROI image. The different sizes of filter kernels include: a small kernel of 5×5 and a large kernel of 105×105. The 5×5 small kernel is used to eliminate the influence of moiré as much as possible, and the 105×105 large kernel is used to obtain a background image with balanced colors; S3. Perform image difference on the foreground image and the background image to obtain a foreground-background separation image; S4. Perform Canny edge detection on the foreground-background separation image to obtain a Canny detection image with the defects outlined; S5. Perform progressive probabilistic Hough line transformation on the Canny detection image to obtain information about line defects; S6. Re-cluster the extracted defect information to obtain accurate defect information; The specific steps for re-clustering the extracted defect information to obtain accurate defect information are as follows: S601. Find the angle θ of each line segment in the line segment set ( { | < < }), then cluster the line segments by angle to obtain clusters of line segments classified by angle; among them, line segments with different angles will obviously not be on the same line; S602. Project the line segments in the cluster obtained in step S601 onto the coordinate system. The line segment with an angle of 0 is projected onto the Y-axis, the line segment with an angle of 90 is projected onto the X-axis. For a line segment with an angle in (0, 90) or (-90, 0), first rotate the coordinate system, find the coordinates of the points after coordinate system rotation, and then project them onto the coordinate system; [[ID=⑨]]S603. For line segments with an angle in (90, -90), first perform coordinate system rotation and then project them onto the rotated coordinate system; [[ID=⑩]]S604. Cluster the projected points; [[ID=⑪]]S605. Process the clustered line segments to obtain the defect information of line defects.

2. The method for detecting and extracting midline defects of an LCD screen according to claim 1, wherein: [[ID=⑫]]The ROI image cropping includes two types: one is manual cropping, and the other is to calculate the contour of the outermost edge of the screen and then crop the ROI image according to the contour.

3. The method for detecting and extracting midline defects of an LCD screen according to claim 1, wherein: [[ID=⑬]]In step S4, Canny edge detection is performed by setting double thresholds, which are denoted as high threshold H and low threshold L. Among them, the size of the high threshold is a set multiple of the low threshold. The specific detection is as follows: If the gradient value t of any pixel in the image > H, it is retained; [[ID=⑭]]If the gradient value t of any pixel in the image < L, it is discarded; [[ID=⑮]]If the gradient value L <= t <= H of any pixel in the image, then search for the pixel gradient value from the neighborhood of this pixel. If there is a pixel gradient value higher than the high threshold, it is retained; if not, it is discarded; [[ID=⑯]]Depict the defect edge according to the retained pixels.

4. The method for detecting and extracting midline defects of an LCD screen according to claim 1, wherein: [[ID=⑰]]The specific process of performing progressive probabilistic Hough line transformation on the Canny detection image is as follows: [[ID=⑱]]S501. Randomly select a feature point in the Canny detection image, that is, an edge point. If this edge point has been marked as a point on a certain straight line, then continue to randomly select an edge point from the remaining edge points until all edge points have been selected; [[ID=⑲]]S502. Perform Hough transformation on the selected edge points and perform cumulative sum calculation; [[ID=⑳]]S503. Select the point with the largest value in the Hough space. If this point is greater than the given threshold, perform step S504; otherwise, return to S501; S504. Based on the maximum value obtained by the Hough transform, shift along the direction of the straight line from the maximum value point to find the two endpoints of the straight line; S505. Calculate the length of the line. If it is greater than a given threshold, it is considered a good line and output, and return to step S501. S506. Output a set of line segments represented by two line segment endpoints.

5. The method for detecting and extracting midline defects of an LCD screen according to claim 1, wherein: The coordinate rotation steps are as follows: In the original coordinate system XoY, if the angle θ of the line segment is (0,90), then rotate XoY counterclockwise around the origin by θ degrees to become the coordinate system SoT; if the angle θ of the line segment is (-90,0), rotate it clockwise around the origin by θ degrees to become the coordinate system SoT; The coordinate rotation formula is as follows: Suppose there is a point p, whose coordinates in the original coordinate system are (x, y), and whose new coordinates after rotation are (s, t), then: Find the coordinates of the endpoints of the line segment in the rotated coordinate system SoT, and then project them onto the SoT coordinate axis to obtain the projection point (s, t).

6. A system for detecting and extracting midline defects of LCD screens, characterized in that: The system includes: a memory and a processor. The memory includes a method program for detecting and extracting defect information of a centerline defect of an LCD screen. When the method program is executed by the processor, the following steps are implemented: S1. Obtain the image to be detected and perform ROI image cropping on the image to be detected to extract the actual area to be detected in the image; S2. Performing Gaussian filtering on the obtained ROI image using filter kernels of different sizes to obtain a foreground image and a background image of the ROI image. The filter kernels of different sizes include a small 5×5 kernel and a large 105×105 kernel. The small 5×5 kernel is used to minimize the effect of moiré patterns, and the large 105×105 kernel is used to obtain a color-balanced background image. S3. Perform image difference on the foreground image and the background image to obtain a front-back separation image; S4. Perform Canny edge detection on the foreground and background separation map to obtain a Canny detection map of the defect stroked; S5. Perform progressive probabilistic Hough line transform on the Canny detection image to obtain line defect information; S6. Re-cluster the extracted defect information to obtain accurate defect information; The specific steps for re-clustering the extracted defect information and obtaining accurate defect information are as follows: S601. Find the angle θ of each line segment in the line segment set ( { | < < }), and then cluster the line segments by angle to obtain clusters of line segments classified by angle; among them, line segments with different angles are obviously not on the same line S602. Project the line segments in the cluster obtained in step S601 into the coordinate system. Line segments with an angle of 0 are projected onto the Y axis, and line segments with an angle of 90 are projected onto the X axis. For line segments with an angle between (0, 90) and (-90, 0), the coordinate system is first rotated, the coordinates of the rotated points are calculated, and then the points are projected into the coordinate system. S603. For line segments with an angle of (90, -90), first rotate the coordinate system and then project it onto the rotated coordinate system. S604. Cluster the projected points; S605. Process the clustered line segments to obtain defect information of the line defects.

7. The LCD screen midline defect detection and defect information extraction system according to claim 6, characterized in that: The ROI image cropping includes two methods: one is manual cropping, and the other is calculating the outline of the outermost edge of the screen and then cropping the ROI image according to the outline.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a method program for detecting centerline defects and extracting defect information of an LCD screen. When the method program for detecting centerline defects and extracting defect information of an LCD screen is executed by a processor, the steps of a method for detecting centerline defects and extracting defect information of an LCD screen as described in any one of claims 1 to 5 are implemented.

Images