A high-precision method for detecting the border overlap in the packing of liquid crystal panels

Abstract

The present invention discloses a high-precision detection method for the edge overlap of a liquid crystal panel during packing, which includes the following steps: placing four line laser light sources at the four corner positions above the liquid crystal panel during packing; the line laser light sources form a regular "#" shape on the surface of the product; a matrix camera set above the packing obtains a real-time detection image; through color space conversion, the obtained real-time detection image is binarized to obtain a binary image; the binary image is subjected to a dilation operation; the obtained binary image is calculated to obtain a boundary set V i , i = 0, 1, 2, 3; respectively perform linear fitting on the obtained boundary set V i to determine whether there is edge overlap; send the detected edge overlap result to the alarm system, and if it shows edge overlap, the alarm system gives an early warning. Through the present invention, the tiny edge overlap of the panel can be detected, with an accuracy within 1 mm, and it is not restricted by the shape of the box, having a wide application range, high detection accuracy, and simple algorithm implementation.

Description

Technical Field

[0001] The present invention belongs to the technical field of automation and computer vision, and particularly relates to a high-precision edge-overlapping detection method for liquid crystal panel packing boxes. Background Art

[0002] With the improvement of the level of automation and industrial intelligence, the degree of automation in liquid crystal panel production is also getting higher and higher. In the actual use process, when the grabbed liquid crystal panel is placed into a packing box made of foam or plastic by an automatic transfer device, there will be a situation of panel placement deviation due to reasons such as equipment accuracy or interference. In this case, the liquid crystal panel will overlap on the edge of the box. If the automatic detection device cannot alarm and intervene for the edge-overlapping situation, it will lead to the occurrence of panel breakage, resulting in serious losses.

[0003] Therefore, it is very beneficial to the liquid crystal automation production of enterprises to design a non-contact edge-overlapping detection method that is simple to use, has high precision, and can detect quickly to automatically detect and discriminate the edge-overlapping situation during liquid crystal panel packing and give an automatic alarm. The Chinese invention patent with the publication number "CN115791791A" discloses a visual detection method for liquid crystal panel packing edge-overlapping. The invention includes the following steps: industrial area array cameras are respectively arranged above the four corners at the bottom of the packing box to capture the grayscale images of the box corners; for each corner of the packing box, the grayscale image of the box corner is captured for visual detection: the box corner is positioned: the positioning coordinates (x, y) of the box corner are obtained; according to the box corner positioning coordinates (x, y), it is judged whether the liquid crystal panel overlaps on the packing box within the detection range. Although the invention does not need to rely on line lasers, point lasers or other three-dimensional data, nor does it require a large number of training images like the deep learning detection algorithm based on images, and the edge-overlapping detection function can be realized only with one training image, but it requires steps such as downsampling, edge detection, centroid calculation, K-means clustering, and region growing algorithm. The algorithm implementation is relatively complex, and at the same time, it can only detect the case where the box has a right-angle edge, and the detection accuracy for the case where the box position changes during implementation is not high. Summary of the Invention

[0004] The present invention provides a high-precision edge-overlapping detection method for liquid crystal panel packing, aiming to solve the problems of low detection accuracy, relatively complex detection algorithm, and narrow practical range in the current panel packing edge-overlapping detection.

[0005] To solve the above technical problems, the present invention provides the following technical solutions:

[0006] A high-precision edge-overlapping detection method for liquid crystal panel packing includes the following steps:

[0007] S1: Four line laser light sources are placed at the four corner positions above the liquid crystal panel packing box.

[0008] S2: Each line laser source irradiates along the inner edge of the box onto the opposite product, forming a regular "#" shape on the product surface.

[0009] S3: The area array camera set above the packing box obtains the "#" shaped image formed by the line laser detected in real time along the panel edge.

[0010] S4: By converting the color space, perform binarization processing on the obtained real-time detection image to obtain a binary image.

[0011] S5: Perform a dilation operation on the binary image to increase the number of pixels on the target boundary, fill the internal holes of the image, and connect the disconnected target areas.

[0012] S6: Calculate the obtained binary image to obtain the upper, lower, left, and right boundary sets V i , i = 0, 1, 2, 3.

[0013] S7: Respectively perform piecewise linear fitting on the obtained boundary sets V i and calculate the mathematical expectation and variance of the straight line after piecewise linear fitting of each boundary set V i . If the variance is greater than the set threshold, it is determined that the laser is bent, and thus it is determined that the liquid crystal panel has edge overlap.

[0014] S8: Send the detected edge overlap result to the alarm system. If edge overlap is displayed, the alarm system gives an early warning.

[0015] Preferably, the step S4 specifically includes the following steps:

[0016] S4-1: Convert the obtained real-time detection image into the HSV color space.

[0017] S4-2: Since the real-time detection image obtained by the line laser source has a high brightness, select the brightness channel V to adjust the image brightness.

[0018] S4-3: Perform binarization processing on the obtained V-channel image to obtain a binary image.

[0019] Preferably, the structural element for the dilation operation in the step S5 is selected as a rectangle with a size of 3*3.

[0020] Preferably, the step S6 specifically includes the following steps:

[0021] S6-1: Starting from the central region (center_x, center_y) of the image, from the center to the edge, successively take the increasing and decreasing boundaries of each row or each column and form the increasing and decreasing boundary set PE i , i = 0, 1, 2, 3.

[0022] S6-2: Determine whether the calculated number and distance of the increasing and decreasing boundaries are in the laser imaging area.

[0023] Preferably, the step S6-1 specifically includes the following steps:

[0024] S6-1-1: For the area above and below the center point of the image, starting from the center point, calculate the boundaries where the pixels change from black to white and from white to black along the y-axis direction and form an increasing and decreasing boundary set PE i , i = 0, 1. Divide the obtained pixel points into increasing and decreasing boundaries. Determine the boundary where the pixels change from black to white as an increasing boundary, and the boundary where the pixels change from white to black as a decreasing boundary.

[0025] S6-1-2: For the area to the left and right of the center point of the image, starting from the center point, calculate the boundaries where the pixels change from black to white and from white to black along the x-axis direction and form an increasing and decreasing boundary set PE i , i = 2, 3. Divide the obtained pixel points into increasing and decreasing boundaries. Determine the boundary where the pixels change from black to white as an increasing boundary, and the boundary where the pixels change from white to black as a decreasing boundary.

[0026] Preferably, the step S6-2 specifically includes the following steps:

[0027] S6-2-1: Judge the number of boundaries in the current row or column of the increasing and decreasing boundary set PE i . If the number is less than or equal to 2, no corresponding boundary is found, and this row or column is skipped.

[0028] S6-2-2: If the number of boundaries is greater than 2, analyze the distance between paired increasing and decreasing boundaries, where the paired increasing and decreasing boundaries are defined as the two-tuple p-v:

[0029] p-v = {(p j , v j+1 ) | p j ∈ PE i , v j+1 ∈ PE i , j = 0, 2, 4,...}

[0030] where p j is the j-th element in the increasing and decreasing boundary set PE i , and v j+1 is the (j + 1)-th element in the increasing and decreasing boundary set PE i .

[0031] Calculate the distance d between paired increasing and decreasing boundaries:

[0032] d = |v j+1 - p j |

[0033] If the distance d is greater than the threshold d max , or less than the threshold d min , it is determined that the area included in the pair of increase and decrease boundaries is not the laser imaging area, otherwise it is added to the boundary set V i .

[0034] Preferably, the step S7 specifically includes the following steps:

[0035] S7-1: Respectively perform segmentation on the upper, lower, left, and right boundary sets V i , segment them into N parts, and calculate the number of points DD i in each part:

[0036]

[0037] where DD i is the number of points in the i-th part, V sum is the boundary set V i , and N is the segmentation quantity.

[0038] S7-2: Use the random sample consensus algorithm RANSAC to perform line fitting on the points in each DD i to obtain the line slope k i of each segment.

[0039] S7-3: For all the line slopes k i calculated in each boundary set V i , calculate the data expectation and variance. If the variance is greater than the set threshold, it is determined that the liquid crystal panel has an edge overlap.

[0040] Compared with the prior art, the present invention has the following technical effects:

[0041] 1. A high-precision method for detecting the edge overlap of a liquid crystal panel during packing according to the present invention uses line laser to detect the edge overlap of the panel. Through the present invention, the minute edge overlap of the panel can be detected, with an accuracy within 1 mm, and it is not restricted by the shape of the box, having a wide application range, high detection accuracy, and simple algorithm implementation.

[0042] 2. A high-precision method for detecting the edge overlap of a liquid crystal panel during packing according to the present invention, compared with directly performing binarization on the RGB image, first performs HSV color space conversion on the original image collected by the line laser and selects the brightness channel V to process the image, and then performs binarization. The HSV color space transformation has stronger robustness. By performing binarization on the V channel, the brightness information in the image can be more accurately extracted, making the binarized image clearer and more distinct, and reducing the influence of illumination changes on the binarization result.

[0043] 3. In the high-precision edge detection method for packing liquid crystal panels of the present invention, after binarization operation, dilation operation is performed on the image. When there are holes at the image edge, they are filled to make it more complete and continuous, the broken parts in the target are connected, and the boundary of the target is smoothed, reducing the serrated or irregular shape of the boundary, making the edge of the target clearer and smoother, and forming an integral whole, which helps to accurately extract its shape and structure. By the dilation operation, the quality of the binary image is improved, the features of the target object are enhanced, and the accuracy of subsequent image processing and analysis is improved.

[0044] 4. In the high-precision edge detection method for packing liquid crystal panels of the present invention, a graphics processing algorithm is used to judge whether it is a laser imaging area by calculating the increase and decrease of the boundary set PE i quantity and distance. The process is simple and intuitive. Compared with the deep learning algorithm, it does not require a large amount of data to be labeled for training, only simple pixel operations need to be performed on the image, and it can clearly display the shape and features of the image boundary, and intuitively understand and analyze the boundary information of the image.

[0045] 5. In the high-precision edge detection method for packing liquid crystal panels of the present invention, the random sample consensus algorithm RANSAC is used for line fitting, and for each boundary set V i all the line segment slopes k i calculated in it, the data expectation and variance are obtained. If the variance is greater than the set threshold, it is judged that the laser is bent, and thus the panel edge is determined. The random sample consensus algorithm used in the present invention can remove the outliers in the data through random sampling and iteration, thereby improving the stability and robustness of the fitted line, and only needs to implement iterative fitting based on the random sampling method. Compared with other slope estimation algorithms, such as the gradient descent method, the implementation of the random sample consensus algorithm is simpler. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flowchart of the high-precision edge detection method for packing liquid crystal panels of the present invention;

[0047] Figure 2 is a schematic diagram of the real-time detection of the liquid crystal panel without edge for the high-precision edge detection method for packing liquid crystal panels of the present invention;

[0048] Figure 3 is a schematic diagram of the real-time detection of the liquid crystal panel with edge for the high-precision edge detection method for packing liquid crystal panels of the present invention;

[0049] Figure 4 is a schematic diagram of the binary image of the luminance channel for the high-precision edge detection method for packing liquid crystal panels of the present invention;

[0050] Figure 5 It is a schematic diagram of a binary image of the non - dilation operation of a high - precision method for detecting the edge overlap in liquid crystal panel packing according to the present invention;

[0051] Figure 6 It is a schematic diagram of a binary image of the processed binary image of the high - precision method for detecting the edge overlap in liquid crystal panel packing according to the present invention after dilation operation;

[0052] Figure 7 It is a schematic diagram of increasing and decreasing the boundary distance of a high - precision method for detecting the edge overlap in liquid crystal panel packing according to the present invention. Detailed implementation manners

[0053] To make the objectives, technical solutions and advantages of the present invention clearer, the following will combine specific embodiments of the present application and refer to the accompanying drawings to clearly and completely describe the technical solutions of the present invention.

[0054] Refer to Figure 1 As shown, it is a flowchart of a high - precision method for detecting the edge overlap in liquid crystal panel packing proposed by the present invention, including the following steps:

[0055] S1: Place four line laser light sources at the four corner positions above the liquid crystal panel packing.

[0056] S2: Each line laser light source irradiates along the inner edge of the box onto the opposite - side product, forming a regular "#" shape on the product surface.

[0057] S3: Refer to Figure 2 、 Figure 3 As shown, the area array camera set above the packing obtains the "#" - shaped image formed by the line laser along the panel edge in real - time line laser detection. It should be noted that the size of the obtained laser image is more than 10 million pixels, depending on the actual scene.

[0058] S4: Refer to Figure 4 As shown, through converting the color space, the obtained real - time detection image is binarized to obtain a binary image. Specifically:

[0059] S4 - 1: Convert the obtained real - time detection image to the HSV color space.

[0060] S4 - 2: The V brightness channel, S saturation channel, and H hue channel are the ways to process the image in the HSV color space. The image is enhanced through the HSV color space, that is, by selecting a suitable channel for processing. Since the brightness of the real - time detection image obtained through the line laser light source is relatively high, the brightness channel V is selected to adjust the image brightness.

[0061] S4-3: Binarize the acquired V-channel image. Set the pixels with pixel values greater than the threshold to 255 and the pixels less than or equal to the threshold to 0 to obtain a binary image.

[0062] S5: Refer to Figure 5 , Figure 6 As shown, perform a dilation operation on the binary image to make the image boundaries continuous. Specifically: Select a rectangular structuring element for the dilation operation, with a size of 3*3. Traverse all the pixel points of the image with the dilation kernel, and calculate the maximum value of the overlapping part between the dilation kernel and the surrounding area of the corresponding pixel point; Use the calculated maximum value as the pixel value after dilation to overwrite the pixel value at the corresponding position in the original image.

[0063] The dilation operation increases the number of pixels at the target boundary, fills the holes that appear at the image edge to make it more complete and continuous, connects the broken parts in the target and smooths the boundary of the target, reduces the jagged or irregular shape of the boundary, makes the edge of the target clearer and smoother, and forms a whole, which helps to accurately extract its shape and structure, improve the quality of the binary image, enhance the features of the target object, and improve the accuracy of subsequent image processing and analysis.

[0064] S6: Calculate the acquired binary image to obtain the upper, lower, left, and right boundary sets V i , i = 0, 1, 2, 3. Specifically:

[0065] S6-1: Refer to Figure 7 As shown, starting from the central region (center_x, center_y) of the image, from the center to the edge, sequentially take the increasing and decreasing boundaries of each row or column and form the increasing and decreasing boundary set PE i , i = 0, 1, 2, 3. Specifically:

[0066] S6-1-1: For the regions above and below the center point of the image, starting from the center point, calculate the boundaries where the pixels change from black to white and from white to black along the y-axis direction and form the increasing and decreasing boundary set PE i , i = 0, 1. Divide the acquired pixel points into increasing and decreasing boundaries. Judge the boundary where the pixels change from black to white as the increasing boundary, and the boundary where the pixels change from white to black as the decreasing boundary.

[0067] S6-1-2: For the regions to the left and right of the center point of the image, starting from the center point, calculate the boundaries where the pixels change from black to white and from white to black along the x-axis direction and form the increasing and decreasing boundary set PE i , i = 2, 3. Divide the acquired pixel points into increasing and decreasing boundaries. Judge the boundary where the pixel values change from black to white as the increasing boundary, and the boundary where the pixels change from white to black as the decreasing boundary.

[0068] S6-2: Determine whether the calculated number and distance of the increasing and decreasing boundaries are within the laser imaging area, specifically:

[0069] S6-2-1: Determine the number of boundaries in the current row or column's increasing and decreasing boundary set PE i If the number is less than or equal to 2, no corresponding boundary is found, and this row or column is skipped.

[0070] S6-2-2: If the number of boundaries is greater than 2, analyze the distance between paired increasing and decreasing boundaries, where paired increasing and decreasing boundaries are defined as the two-tuple p-v:

[0071] p-v = {(p j , v j+1 ) | p j ∈PE i , v j+1 ∈PE i , j = 0, 2, 4, ……}

[0072] where p j is the j-th element in the increasing and decreasing boundary set PE i , and v j+1 is the (j + 1)-th element in the increasing and decreasing boundary set PE i .

[0073] Calculate the distance d between paired increasing and decreasing boundaries:

[0074] d = |v j+1 - p j |

[0075] If the distance d is greater than the threshold d max , or less than the threshold d min , determine that the area included in this pair of increasing and decreasing boundaries is not the laser imaging area; otherwise, add it to the boundary set V i . It should be noted that the value ranges of the selected thresholds d max and d min are determined according to the actual imaging on the image obtained by the selected product from the laser scanner. In this embodiment, the value of d max is 7, and the value of d min is 1.

[0076] S7: Perform piecewise linear fitting on the obtained boundary set V i respectively, and calculate the mathematical expectation and variance of the straight line after piecewise linear fitting for each boundary set V i . If the variance is greater than the set threshold, it is determined that the laser is bent, and thus it is determined that the liquid crystal panel has a border overlap, specifically:

[0077] S7-1: Perform piecewise linear fitting on the upper, lower, left, and right boundary sets V iPerform segmentation into N parts. It should be noted that the value of N is determined according to the actual situation. In this embodiment, since the collected images are generally bent at both ends after the edge overlap occurs when the liquid crystal panels are packed, the value of N is 3, 4, 5, or 6. Calculate the number of collection points DD for each part. i :

[0078]

[0079] where DD i is the number of points in the i-th part, V sum is the total number of interior points in the boundary set V i , and N is the number of segmentation.

[0080] S7-2: For each point in each DD i , perform line fitting using the Random Sample Consensus (RANSAC) algorithm to obtain the slope k of each line segment, specifically as follows: i :

[0081] Select random samples: Randomly select two points from the original data set DD i to form a sample set of a line as the sample set for the current iteration, and solve the line parameters formed by it.

[0082] Discriminate data points: For each point in the original data set, calculate its distance to the fitting model, and determine whether it is an inlier or an outlier according to the distance threshold.

[0083] Calculate the number of inliers: Count the number of inliers and the error through discrimination. If the current number of inliers is greater than the previous inlier number threshold, update the inlier number threshold and simultaneously update the line fitting sample set.

[0084] Repeat the above steps until a certain number of iterations is reached or the required accuracy and stability requirements are met. The sample set with the largest number of inliers is the maximum consensus set. Fit a line using the least squares method or other fitting methods for this maximum consensus set, and thus calculate the slope k of this line i . It should be noted that the number of iterations set therein needs to be determined according to the actual situation. In this embodiment, the number of iterations selected is 500.

[0085] The Random Sample Consensus (RANSAC) algorithm can remove outliers in the data through random sampling and iteration, thereby improving the stability and robustness of the fitted line, and only needs to implement iterative fitting based on random sampling. Compared with other slope estimation algorithms, such as the gradient descent method, the implementation of the Random Sample Consensus algorithm is simpler.

[0086] S7-3: For each boundary set V i calculate all the line segment slopes k obtainedi , calculate the data expectation and variance, refer to Figure 2 As shown, when the liquid crystal panel is normally packed without overlapping edges, the slopes of the straight lines after linear fitting of the obtained image boundaries are equal, and its variance is 0. Refer to Figure 3 As shown, when there is an overlapping edge situation in the liquid crystal panel packing, the obtained image boundary is curved, the slopes of the straight lines after linear fitting are not equal, and its variance is greater than 0. It should be noted that the set variance threshold needs to be determined according to the actual situation. In this embodiment, the set variance threshold is 0.1. If the variance is greater than the set threshold, it is determined that the panel has an overlapping edge.

[0087] S8: Send the detected overlapping edge result to the alarm system, and the alarm system gives an early warning to prompt the production personnel to conduct a check.

[0088] The above are only the preferred embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the inventive concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. A high-precision detection method for the edge overlap of liquid crystal panel packing, characterized in that, It includes the following steps: S1: Place four line laser light sources at the four corner positions above the liquid crystal panel packing box; S2: Each line laser light source irradiates along the inner edge of the box onto the opposite product, forming a regular "#" shape on the product surface; S3: The area array camera set above the packing box obtains the "#" shape image formed by the line laser along the panel edge detected in real time by the line laser; S4: Through color space conversion, perform binary processing on the obtained real-time detection image to obtain a binary image; S5: Perform dilation operation on the binary image to increase the number of pixels of the target boundary, fill the internal holes of the image, and connect the disconnected target areas; S6: Calculate the obtained binary image to obtain the upper, lower, left, and right boundary sets , ; S7: Segment and perform linear fitting on the obtained boundary sets respectively, and calculate the mathematical expectation and variance of the straight lines after segmental linear fitting for each boundary set If the variance is greater than the set threshold, it is determined that the laser is bent, and thus it is determined that there is an edge overlap in the liquid crystal panel. S8: Send the detected overlap result to the alarm system. If overlap is displayed, the alarm system gives an early warning; The step S6 specifically includes the following steps: S6-1: Starting from the central region of the image and moving from the center to the edge, sequentially obtain the increasing and decreasing boundaries of each row or column and form a set of increasing and decreasing boundaries , ; S6-2: Judge whether the calculated increase and decrease in the number of boundaries and the distance are in the laser imaging area; The step S6-1 specifically includes the following steps: S6-1-1: For the regions above and below the center point of the image, starting from the center point, calculate the boundaries where the pixels change from black to white and from white to black along the axis direction and form an increase-decrease boundary set , , divide the obtained pixel points into increase-decrease boundaries, judge the boundary where the pixels change from black to white as an increase boundary, and judge the boundary where the pixels change from white to black as a decrease boundary; S6-1-2: For the regions to the left and right of the center point of the image, starting from the center point, calculate the boundaries where the pixels change from black to white and from white to black along the axis direction and form an increase and decrease boundary set , , divide the obtained pixel points into increase and decrease boundaries, judge the boundary where the pixels change from black to white as an increase boundary, and judge the boundary where the pixels change from white to black as a decrease boundary; The step S6-2 specifically includes the following steps: S6-2-1: Determine the increase and decrease boundary sets of the current row or column For the number of middle boundaries, if the number is less than or equal to 2, the corresponding boundary is not found, and this row or column is skipped; S6-2-2: If the number of boundaries is greater than 2, analyze the distances of pairwise increasing and decreasing boundaries, where pairwise increasing and decreasing boundaries are defined as a two-tuple : wherein is the th element in the set of increasing and decreasing boundaries, is the th element in the set of increasing and decreasing boundaries; Calculate the distance of paired increase and decrease boundaries : If the distance is greater than the threshold , or less than the threshold , it is determined that the area included in the pair of increase and decrease boundaries is not the laser imaging area; otherwise, it is added to the boundary set .

2. The high-precision edge detection method for liquid crystal panel packing according to claim 1, characterized in that The step S4 specifically includes the following steps: S4-1: Convert the obtained real-time detection image into the HSV color space; S4-2: Select the brightness channel V of the real-time detection image obtained by the line laser light source to adjust the image brightness; S4-3: Perform binary processing on the obtained V-channel image to obtain a binary image.

3. A high-precision edge detection method for packing liquid crystal panels according to claim 1, characterized in that, In the step S5, the structural element for the dilation operation is selected as a rectangle with a size of 3*3.

4. A high-precision edge detection method for packing liquid crystal panels according to claim 1, characterized in that, The step S7 specifically includes the following steps: S7-1: Split the upper, lower, left, and right boundary sets respectively into parts, and calculate the number of points in each part of the set : wherein is the number of points in the th portion, is the total number of interior points in the boundary set , and is the number of partitions; S7-2: For each point inside, perform line fitting using the Random Sample Consensus algorithm RANSAC to obtain the line slope of each segment ; S7-3: For each boundary set calculate the slopes of all the line segments obtained, and calculate the data expectation and variance. If the variance is greater than the set threshold, it is determined that there is an edge overlap in the liquid crystal panel.

Images