Display panel detection method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention discloses a display panel detection method and device, electronic equipment and a readable storage medium, which are used for improving the robustness of display panel detection. The method provided by the embodiment of the invention comprises the following steps: when a display panel is placed on a Tray, shooting a target image through an industrial camera; performing image cutting on the target image according to the center coordinate of each grid in a pre-stored Tray and the width and height of each grid to obtain an ROI corresponding to each grid; based on the ROI, extracting vertex coordinates of the display panel in the target image; wherein the number of the extracted vertex coordinates corresponding to each display panel is 2 or 4; for any display panel, when the number of the vertex coordinates of the display panel is two, the warping type is determined according to the vertex coordinates; wherein the warping type comprises long side warping and wide side warping; calculating all vertex coordinates according to the warping type and the vertex coordinates; and determining whether the display panel has a landing edge or not according to the coordinates of all the vertexes and the ROI.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of display panel detection, and in particular, to a display panel detection method, device, electronic device, and readable storage medium. Background Art

[0002] In the production processes such as panel manufacturing, packaging, and testing, it is often necessary to automatically detect the display panels placed on a Tray (carrier) to determine whether their placement states meet the process requirements. The Tray usually contains multiple regular grid positions for accommodating display panels. During the placement process, the display panels may exhibit phenomena such as offset, tilt, warping, or overlapping edges. If these abnormal states are not detected and processed in a timely manner, it may lead to failures in subsequent handling, bonding, or packaging processes, and even damage the panel body, resulting in a decrease in the yield.

[0003] In the existing detection solutions, an industrial camera is often used to capture images of the Tray, and image processing methods are used to determine whether there are problems such as offset, warping, or out-of-bounds of the panels in each grid position. However, in practical applications, due to interference factors such as lighting conditions, panel shape errors, or image noise, the existing methods are prone to misjudgments in key links such as vertex recognition and position determination, resulting in poor stability of the detection results and unable to meet the application requirements of high-precision and high-versatility production lines.

[0004] Therefore, how to improve the robustness of the display panel placement state detection method so that it can still stably extract effective features and make accurate judgments in a complex environment has become an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] The embodiments of the present application provide a display panel detection method, device, electronic device, and readable storage medium, which can improve the robustness of detecting display panels.

[0006] The first aspect of the embodiments of the present application provides a display panel detection method, including: When the display panel is placed on the Tray, an industrial camera is used to capture a target image; The target image is cut according to the center coordinates, width, and height of each grid in the Tray stored in advance to obtain the ROI corresponding to each grid; Based on the ROI, the vertex coordinates of the display panel in the target image are extracted; among them, the number of vertex coordinates corresponding to each display panel extracted is 2 or 4; For any display panel, when the vertex coordinates of the display panel are 2, the warping type is determined according to the vertex coordinates; where the warping type includes long-edge warping and wide-edge warping; Calculate all vertex coordinates based on the warping type and the vertex coordinates; Determine whether there is an overlap edge on the display panel based on the all vertex coordinates and the ROI.

[0007] Optionally, the determining the warping type based on the vertex coordinates includes: Calculate the Euclidean distance between the vertex coordinates; Determine the warping type based on the Euclidean distance and the standard size of the display panel.

[0008] Optionally, the calculating all vertex coordinates based on the warping type and the vertex coordinates includes: Calculate the perpendicular unit normal vector corresponding to the edge direction vector formed by the vertex coordinates according to the Euclidean distance; Calculate all vertex coordinates based on the warping type, the perpendicular unit normal vector, and the vertex coordinates.

[0009] Optionally, the determining whether there is an overlap edge on the display panel based on the all vertex coordinates and the ROI includes: Establish a diagonal vector based on any diagonal of the ROI; Determine the corresponding diagonal vertices and the starting point among the all vertex coordinates according to the diagonal vector; Construct a first vector and a second vector with the starting point and the vertices other than the diagonal vertices as the endpoints respectively; Determine whether there is an overlap edge on the display panel according to the first vector, the second vector, and the diagonal vector.

[0010] Optionally, the determining whether there is an overlap edge on the display panel according to the first vector, the second vector, and the diagonal vector includes: Calculate a first cross product value of the first vector and the diagonal vector, and calculate a second cross product value of the second vector and the diagonal vector; Calculate the difference between the absolute value of the first cross product value and the absolute value of the second cross product value; Determine whether the difference is greater than or equal to a preset threshold; If so, determine that there is an overlap edge on the display panel; if not, determine that there is no overlap edge on the display panel.

[0011] Optionally, before capturing the target image by the industrial camera, the method further includes: When the display panel is not placed on the Tray, capture an image of the empty tray by the industrial camera; Identify the center coordinates of each grid in the empty tray image through a single-grid template.

[0012] Optionally, after extracting the vertex coordinates of the display panel in the target image based on the ROI, the method further includes: For any display panel, when the number of vertex coordinates of the display panel is 4, obtain the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate based on the ROI; Determine whether there is a border overlap for the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

[0013] A second aspect of the embodiments of the present application provides a display panel detection device, including: A first photographing unit, configured to photograph a target image through an industrial camera when a display panel is placed on a Tray tray; A cutting unit, configured to perform image cutting on the target image according to the center coordinates of each grid and the width and height of each grid stored in advance in the Tray tray to obtain an ROI corresponding to each grid; An extraction unit, configured to extract the vertex coordinates of the display panel in the target image based on the ROI; wherein, the number of vertex coordinates corresponding to each extracted display panel is 2 or 4; A first determination unit, configured to, for any display panel, when the number of vertex coordinates of the display panel is 2, determine the warping type according to the vertex coordinates; wherein, the warping type includes long-side warping and wide-side warping; A calculation unit, configured to calculate all vertex coordinates according to the warping type and the vertex coordinates; A second determination unit, configured to determine whether there is a border overlap for the display panel according to all the vertex coordinates and the ROI.

[0014] Optionally, the first determination unit is specifically configured to: Calculate the Euclidean distance between the vertex coordinates; Determine the warping type according to the Euclidean distance and the standard size of the display panel.

[0015] Optionally, the calculation unit is specifically configured to: Calculate a vertical unit normal vector corresponding to the edge direction vector formed by the vertex coordinates according to the Euclidean distance; Calculate all vertex coordinates according to the warping type, the vertical unit normal vector, and the vertex coordinates.

[0016] Optionally, the second determination unit includes: A building module, configured to build a diagonal vector based on any diagonal of the ROI; The first determination module is configured to determine the corresponding diagonal vertices and starting point among all vertex coordinates according to the diagonal vector. The construction module is configured to construct a first vector and a second vector with the starting point and the vertices other than the diagonal vertices as the endpoints respectively. The second determination module is configured to determine whether there is an overlap edge on the display panel according to the first vector, the second vector and the diagonal vector.

[0017] Optionally, the second determination module is specifically configured to: Calculate a first cross product value of the first vector and the diagonal vector, and calculate a second cross product value of the second vector and the diagonal vector. Calculate the difference between the absolute value of the first cross product value and the absolute value of the second cross product value. Determine whether the difference is greater than or equal to a preset threshold. If so, determine that there is an overlap edge on the display panel; if not, determine that there is no overlap edge on the display panel.

[0018] Optionally, the device further includes a calibration unit, and the calibration unit is configured to: When the display panel is not placed on the Tray tray, capture an image of the empty tray through an industrial camera. Identify the center coordinates of each grid in the empty tray image through a single-grid template.

[0019] Optionally, the device further includes a third determination unit, and the third determination unit is configured to: For any display panel, when the vertex coordinates of the display panel are 4, obtain the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate and the minimum y-axis coordinate based on the ROI. Determine whether there is an overlap edge on the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate and the minimum y-axis coordinate.

[0020] A third aspect of the embodiments of the present application provides an electronic device, including: A processor, a memory, an input / output unit and a bus; The processor is connected to the memory, the input / output unit and the bus; The memory stores a program, and the processor calls the program to execute the method in the first aspect and any possible implementation manner of the first aspect.

[0021] In the fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. A program is stored on the computer-readable storage medium, and when the program is executed on a computer, the computer is caused to execute the method in the first aspect and any possible implementation manner of the first aspect.

[0022] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages: In the embodiments of the present application, by controlling an industrial camera to collect images of the Tray tray and combining with the Tray tray template for operations such as image segmentation, display panel vertex extraction, warping type recognition, vertex completion, and tabbing judgment, automatic recognition of the attitude and position state of the display panel in the Tray tray is achieved. In the case where the panel has warping, occlusion, or partially missing corner points, the complete vertex coordinates can still be accurately obtained through calculation, and then it can be determined whether tabbing occurs, thereby effectively improving the accuracy and robustness of abnormal panel recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic flowchart of an embodiment of the display panel detection method in the embodiments of the present application; Figure 2 It is a schematic flowchart of an embodiment of determining the warping type in the embodiments of the present application; Figure 3 It is a schematic flowchart of an embodiment of calculating all vertex coordinates in the embodiments of the present application; Figure 4 It is a schematic flowchart of an embodiment of determining whether there is tabbing for the display panel in the embodiments of the present application; Figure 5 It is a schematic flowchart of another embodiment of determining whether there is tabbing for the display panel in the embodiments of the present application; Figure 6 It is a schematic flowchart of an embodiment of identifying the center coordinates of each grid in the empty tray image in the embodiments of the present application; Figure 7 It is a schematic flowchart of another embodiment of determining whether there is tabbing for the display panel in the embodiments of the present application; Figure 8 It is a schematic structural diagram of an embodiment of the display panel detection device in the embodiments of the present application; Figure 9 It is a schematic structural diagram of an embodiment of the electronic device in the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The embodiments of the present application provide a display panel detection method, device, electronic device, and readable storage medium for improving the robustness of detecting a display panel.

[0025] The method of the present application can be applied to servers, terminals, or other devices with logical processing capabilities. In this regard, the present application does not make any limitations. For the sake of convenience in description, the following will take the server as the execution subject for description.

[0026] The embodiments in the present application will be described below in conjunction with the accompanying drawings.

[0027] Please refer to Figure 1 , an embodiment of the display panel detection method in the embodiments of the present application includes: 101. When the display panel is placed on the Tray tray, capture a target image through an industrial camera; After the detection process is started, the server controls the industrial camera connected to it to take a picture of the display panel on the Tray tray (also known as the material tray, carrier tray), and obtains a high-resolution image containing multiple panels as the target image. The industrial camera has characteristics such as high frame rate, high pixel, and low distortion, and is suitable for high-precision image acquisition. The Tray trays are usually arranged in a matrix, and multiple grids are used to hold the display panels, and each panel is placed separately in a grid. The purpose of this step is to collect an image containing all the display panels to be detected, laying a foundation for subsequent image processing.

[0028] 102. Perform image cutting on the target image according to the center coordinates, width, and height of each grid in the Tray tray stored in advance, to obtain the ROI corresponding to each grid; The server calls the Tray tray template information pre-calibrated and stored in the local database. This template records the center coordinates, width (i.e., the horizontal dimension), and height (i.e., the vertical dimension) of each grid in the Tray tray. The server performs an image segmentation operation on the target image obtained in step 101 according to these parameters, divides the entire image into several regions, and each region is called an ROI (Region of Interest), and each ROI corresponds to a grid in the Tray tray, that is, the expected position of a display panel. The image cutting uses a rectangular cropping method based on a geometric template to achieve separate processing of each small area.

[0029] 103. Based on the ROI, extract the vertex coordinates of the display panel in the target image; wherein, the number of vertex coordinates corresponding to each extracted display panel is 2 or 4; The server performs image feature extraction operations on each ROI image to identify the vertex coordinate information of the display panel. The identification methods usually rely on edge detection (such as the Canny algorithm), contour extraction (such as the findContours function in OpenCV), and corner detection (such as Harris corners, Shi-Tomasi corners, etc.), combined with geometric screening rules (such as side length ratios, angle ranges) to locate the vertices of the display panel. Since some panels may be warped or occluded, only 2 vertex coordinates can be extracted instead of the complete 4 vertices. The server records the number of vertices extracted for each display panel and the corresponding coordinates, providing a data basis for subsequent calculation and processing.

[0030] 104. For any display panel, when the number of vertex coordinates of the display panel is 2, determine the warping type according to the vertex coordinates; wherein, the warping types include long-edge warping and wide-edge warping. The server further determines the warping type of each display panel for which only 2 vertex coordinates are extracted. Warping refers to the deformation of the panel caused by forces, heat, etc., resulting in one of its sides no longer being in close contact with the bottom surface of the Tray. The server makes a judgment by calculating the relative position relationship of the 2 vertices in the ROI image (such as whether the horizontal distance is greater than the vertical distance): if the vertices are mainly distributed at both ends of the horizontal direction of the image, it is determined as "long-edge warping"; if mainly distributed at both ends of the vertical direction, it is determined as "wide-edge warping". This classification helps to accurately simulate the warping trend in subsequent vertex completion operations.

[0031] 105. Calculate all vertex coordinates according to the warping type and vertex coordinates. The server combines the warping type determined in step 104 with the existing 2 vertex coordinates, and uses geometric calculation or rule-based modeling methods to calculate and complete the remaining 2 vertices. For example, when there is long-edge warping, the server takes the two short sides as known sides, and through the known point coordinates, side length ratios, and preset panel length and width parameters, calculates the 4 vertex coordinates of the complete rectangle. The completion methods can include symmetric mapping, boundary extension, fitting rectangle frames, etc. The finally obtained 4 vertex coordinates are used for subsequent positioning and analysis to ensure the integrity of the panel state judgment.

[0032] 106. Determine whether there is edge overlap for the display panel according to all vertex coordinates and the ROI.

[0033] Based on the four complete vertex coordinates calculated in step 105, the server combines the ROI boundary information to which the display panel belongs to determine whether there is a phenomenon of overlapping the edge. Overlapping the edge means that any part of the panel (usually the vertex or the edge) exceeds the boundary of the grid to which it belongs, which may cause the failure of the manipulator to carry, jamming during loading and unloading, or failure of the subsequent fitting and alignment. The determination method can be to calculate whether the panel vertices are outside the ROI, or whether there is an intersection or exceeding between the panel rectangle and the ROI boundary. Once it is determined that there is an overlapping edge, the server can mark the panel as an abnormal state for subsequent processes to skip or process.

[0034] The server controls the industrial camera to collect the Tray tray image, and combines the Tray tray template to perform operations such as image segmentation, display panel vertex extraction, warping type recognition, vertex completion, and overlapping edge judgment, realizing the automatic recognition of the posture and position state of the display panel in the Tray tray. In the case of warping, occlusion, or partial missing corner points of the panel, the complete vertex coordinates can still be accurately obtained through calculation, and then it can be judged whether there is an overlapping edge, thereby effectively improving the accuracy and robustness of abnormal panel recognition.

[0035] Please refer to Figure 2 , in some embodiments of the present application, for any display panel in step 104 above, when the vertex coordinates of the display panel are two, determining the warping type according to the vertex coordinates may include the following steps: 201. Calculate the Euclidean distance between the vertex coordinates; For any display panel that only extracts two vertex coordinates, the server can obtain the image coordinates (usually pixel-level coordinate points) of the corresponding two vertices, denoted as point . Subsequently, the server is based on the Euclidean distance formula: Formula 1 Calculate the straight-line distance between the two points to obtain the actual pixel distance value between the vertices. The Euclidean distance is used here as a measure of the geometric interval between two vertices to preliminarily judge whether the two points may be at both ends of the long side or the short side of the panel. The server temporarily stores the calculation result for the next step to perform size ratio comparison.

[0036] 202. Determine the warping type according to the Euclidean distance and the standard size of the display panel.

[0037] The server converts the Euclidean distance value calculated in step 201, and combines the pre-set standard size of the display panel (for example: the long side is L, the short side is W, and the unit can be millimeters), and converts the Euclidean distance between pixels and physical units (if calibration data is used, it can be directly mapped from the pixel distance to the millimeter unit). The server compares the converted distance with the standard long side L and short side W of the panel to judge which size it is closer to: If the Euclidean distance is close to L, the server determines that the two points are at both ends of the long side of the panel, that is, the long side of the panel is warped; If the Euclidean distance is close to W, the server determines that the two points are at both ends of the short side of the panel, that is, the short side of the panel is warped.

[0038] To improve robustness, the server can set a threshold range (e.g., ±5%) as the proximity condition to avoid misjudgment caused by image deformation, tilt, or noise.

[0039] By further performing Euclidean distance calculation and comparison with the standard size based on the two vertex coordinates extracted, the server can accurately identify the boundary type of panel warping (long side warping or short side warping) even with only partial corner information. This judgment result provides a directional basis for subsequent vertex completion, ensuring that even if the panel attitude is abnormal or the image information is missing, the server can still reconstruct its complete bounding box, improving the reliability and accuracy of automatic positioning and anomaly recognition.

[0040] Please refer to Figure 3 , in some embodiments of the present application, step 105 in the above embodiments for calculating all vertex coordinates according to the warping type and vertex coordinates may include the following steps: 301. Calculate the corresponding perpendicular unit normal vector of the edge direction vector formed by the vertex coordinates according to the Euclidean distance; Assume that the two vertex coordinates are , and the vector is the identified edge direction vector, then , and the server can calculate the perpendicular unit normal vector according to formula 2: Formula 2 where, is the perpendicular unit normal vector, is the Euclidean distance between the two vertices.

[0041] 302. Calculate all vertex coordinates according to the warping type, perpendicular unit normal vector, and vertex coordinates.

[0042] If the warping type is short side warping, the server can calculate the coordinates of the other two vertices according to the following formula: Formula 3 Formula 4 where, L is the standard long side size of the display panel, is the x component of the perpendicular unit normal vector (i.e., in formula 2), is the y component of the perpendicular unit normal vector (i.e., in formula 2).

[0043] If the warping type is long edge warping, the server can calculate the coordinates of the other two vertices according to the following formula: Formula 5 Formula 6 Where W is the standard width of the display panel. is the x component of the vertical unit normal vector (i.e. ), is the y component of the vertical unit normal vector (i.e. ).

[0044] The server constructs a direction vector based on the edge formed by two known vertices, and further derives the perpendicular unit normal vector, thereby accurately calculating the positions of the remaining two vertices based on the warping type and standard size, and successfully constructing a complete panel bounding box. This process ensures that even if only two corner points can be identified in the initial image, the complete rectangle can be restored through geometric relationships and vector calculations, effectively improving the vertex completion capability and positioning accuracy in the case of image defects, occlusion or warping, and providing a solid foundation for subsequent edge detection, posture correction and other functions.

[0045] See also Figure 4 In some embodiments of the present application, step 106 in the above embodiment determines whether there is an overlapped edge on the display panel according to all vertex coordinates and ROI, and may include the following steps: 401. Establish a diagonal vector based on any diagonal line of the ROI; Based on the rectangular area of ​​the corresponding grid in each ROI, the server selects two diagonal points of the ROI and establishes a diagonal vector based on these two points. This vector represents the reference direction of the ROI in the Tray, and is used for subsequent geometric comparison with the vector formed by the display panel vertices. The server usually selects the upper left corner and the lower right corner, or the upper right corner and the lower left corner as the starting and ending points of the diagonal line, and calculates the vector based on their coordinate difference. For example, from the upper left corner point To the lower right corner , generating the vector: Formula 7 The server uses this vector as a reference to determine whether the vertex of the display panel exceeds the edge of the frame.

[0046] 402. Determine the corresponding diagonal vertices and starting points in all vertex coordinates according to the diagonal vector; After obtaining the diagonal vector, the server can compare all vertex coordinates and determine which pair of vertices are closest to the diagonal endpoints in space, and use them as the "diagonal starting point" and "diagonal end point" respectively. This pair of vertices is a diagonal line on the panel that is closest to the ROI diagonal direction, and is used as a reference line direction for subsequent edge judgment. For example, the diagonal vector selected in step 401 is a vector formed from the upper left corner point (starting point) to the lower right corner point (end point), then the diagonal vertices of the display panel determined by the server are also the upper left corner point and the lower right corner point, and the starting point is also the upper left corner point.

[0047] 403. Take the vertices other than the diagonal vertices as the end points and construct the first vector and the second vector with the starting point; The server uses the remaining two vertices as end points, and constructs a first vector and a second vector with the determined diagonal starting point. The two vectors represent the edge extension direction of the display panel in the ROI. The three vectors (diagonal vector, first vector, and second vector) together define the spatial relationship of the panel in the area, providing a basis for the subsequent calculation of geometric quantities such as angles or cross products. For example, according to the selection in the example of step 402, the server can construct a vector from the upper left corner point to the lower left corner point, and a vector from the upper left corner point to the upper right corner point. These two vectors are used as the first vector and the second vector. In this embodiment, there is no restriction on which of the two is used as the first vector, and the same applies to the second vector.

[0048] 404. Determine whether there is an overlapping edge on the display panel according to the first vector, the second vector, and the diagonal vector.

[0049] The server analyzes whether there is overlap on the display panel based on the spatial geometric relationship between the first vector, the second vector and the diagonal vector. The server determines whether the panel has abnormal contact with the ROI boundary or exceeds the grid range by calculating the relative orientation (such as the angle or the sign and amplitude of the cross product) between the above vectors, thereby completing the determination of the overlap.

[0050] The server constructs the ROI diagonal vector and uses it as a reference to accurately calculate the vertex direction and spatial relationship of the display panel, and can determine whether the display panel deviates from its grid and has overlapping edges. This method does not rely on the absolute position of the panel, but makes spatial layout judgments based on relative vector geometric relationships. It has higher adaptability and fault tolerance, effectively avoids misjudgments caused by image noise, warping or angle deviation, improves detection accuracy, and ensures the reliability of automatic recognition.

[0051] See also Figure 5 In some embodiments of the present application, step 404 in the above embodiment determines whether there is an overlap on the display panel according to the first vector, the second vector and the diagonal vector, and may include the following steps: 501. Calculate the first cross product value of the first vector and the diagonal vector, and calculate the second cross product value of the second vector and the diagonal vector; Based on the diagonal vector, the server calculates the spatial geometric relationship between the first vector and the diagonal vector respectively to obtain the first cross product value, and then calculates the spatial geometric relationship between the second vector and the diagonal vector to obtain the second cross product value. These two cross product values respectively reflect the relative offset degrees of the first vector and the second vector in the direction perpendicular to the diagonal, and can characterize the geometric deviation between the edge of the display panel and the ROI diagonal. Specifically, the server can calculate the first cross product value and the second cross product value according to the following formula respectively: Formula 8 where is the cross product value, which is the first cross product value when i is 2 and the second cross product value when i is 4.

[0052] 502. Calculate the difference between the absolute value of the first cross product value and the absolute value of the second cross product value; The server compares the absolute value of the first cross product value and the absolute value of the second cross product value and calculates the difference between them. This difference reflects the symmetry offset degree of the left and right sides (or the upper and lower sides) of the display panel under the diagonal reference. The larger the difference, the more obvious the offset difference between the two sides, and the more unbalanced the overall placement state of the panel.

[0053] 503. Determine whether the difference is greater than or equal to a preset threshold; The server compares the above difference with the preset threshold to determine whether there is a structural offset abnormality in the display panel. If the difference is greater than or equal to the threshold, it means that the offset difference is large and there is a risk of edge overlap; if the difference is less than the threshold, it means that the offset difference is small and it can be considered within the allowable range.

[0054] 504. Determine that there is an edge overlap in the display panel; When the difference is greater than or equal to the preset threshold, the server determines that there is an edge overlap in the display panel, indicating that at least a part of the edge of the panel has crossed the effective boundary of the grid area, which may affect subsequent processes or quality inspections.

[0055] 505. Determine that there is no edge overlap in the display panel.

[0056] When the difference is less than the preset threshold, the server determines that there is no edge overlap in the display panel, indicating that the placement state of the panel is good and all four vertices are within the reasonable area range of the ROI.

[0057] Based on the symmetry principle of the vector cross product result, the server successfully establishes a mathematical judgment mechanism for detecting whether there is an overlap on the display panel by calculating the area projection difference between two non-diagonal sides and the diagonal of the ROI. This method does not rely on pixel-level contour judgment or image segmentation accuracy, has good robustness and anti-interference ability, effectively improves the accuracy and efficiency of overlap detection, and provides strong mathematical support for batch detection and intelligent recognition under industrial cameras.

[0058] Please refer to Figure 6 , in some embodiments of the present application, the display panel detection method may further include the following steps: 601. When the display panel is not placed on the Tray tray, take an image of the empty tray through an industrial camera; At the initialization stage of the detection process, the server controls the industrial camera to take a picture of the Tray tray without the display panel currently placed to obtain an empty tray image. This image only contains the Tray tray body and the regularly arranged blank spaces on it, without being blocked or interfered by the display panel, and the image edges are clear, which is conducive to subsequent spatial structure analysis. This step aims to establish an original reference model for the current Tray tray structure and provide a reference coordinate system for subsequent panel detection.

[0059] 602. Identify the center coordinates of each grid in the empty tray image through a single-grid template.

[0060] Based on the pre-trained or set single-grid template image, the server performs template matching operations on the empty tray image to locate the center coordinates of each Tray tray grid. Here, the "single-grid template" refers to a predefined standard Tray grid area pattern, which contains features such as typical border structures, shadow contours, or material textures, and has uniqueness and matchability.

[0061] The server slides the template window in the image, calculates the similarity through a matching algorithm (such as Normalized Cross-Correlation NCC or Structural Similarity SSIM), identifies all matching positions, and extracts the corresponding geometric center point coordinates. Assume that the Tray tray has grids, then the center coordinates of each grid are , where i = 1, 2,..., M, j = 1, 2,..., N.

[0062] Before the display panel detection process, the server introduces the operations of empty tray image acquisition and single-grid template recognition, which can automatically construct the Tray tray coordinate system and grid layout information in the absence of manual calibration or a preset model. This method performs recognition based on the image data captured in the real environment, can effectively adapt to Tray tray structures of different models, sizes, or arrangements, improve the versatility and flexibility of the system in multi-specification production lines, and at the same time provide an accurate spatial reference for subsequent steps such as panel positioning and ROI extraction, improving the overall detection accuracy.

[0063] Please refer to Figure 7 , in some embodiments of the present application, after step 103 in the above embodiments extracts the vertex coordinates of the display panel in the target image based on the ROI, the display panel detection method may further include the following steps: 701. For any display panel, when the vertex coordinates of the display panel are 4, obtain the maximum x-axis coordinate, minimum x-axis coordinate, maximum y-axis coordinate, and minimum y-axis coordinate based on the ROI; After the server extracts the four vertex coordinates of the display panel from the ROI, first calculate the maximum and minimum values of the x-axis and y-axis in the vertex coordinates to obtain the bounding box range of the display panel in the ROI. The specific steps are as follows: Maximum x-axis coordinate ( ): Select the maximum value of the x coordinate from the four vertex coordinates.

[0064] Minimum x-axis coordinate ( ): Select the minimum value of the x coordinate from the four vertex coordinates.

[0065] Maximum y-axis coordinate ( ): Select the maximum value of the y coordinate from the four vertex coordinates.

[0066] Minimum y-axis coordinate ( ): Select the minimum value of the y coordinate from the four vertex coordinates.

[0067] These coordinate values together determine an external bounding rectangle (bounding box), which frames the position range of the display panel in the ROI. This step is to further quantitatively analyze the spatial position of the display panel and provide geometric data support for subsequent overlap judgment.

[0068] 702. Determine whether there is an overlap of the display panel according to the vertex coordinates, maximum x-axis coordinate, minimum x-axis coordinate, maximum y-axis coordinate, and minimum y-axis coordinate.

[0069] The server compares the bounding box coordinate values (maximum x, minimum x, maximum y, minimum y) obtained in step 701 with the four vertex coordinates extracted to determine whether there is any overlap of the display panel. The server can make the determination according to the following formula: Formula 9 If any x value or y value among the four vertex coordinates does not conform to the above Formula 9, it can be determined that there is an overlap of the display panel.

[0070] Otherwise, if all vertices are within the bounding box, it is determined that there is no overlap of the display panel.

[0071] By introducing the comparison between the maximum and minimum coordinate values and the bounding box of the ROI, the server can accurately determine whether the display panel has exceeded the boundary or overlapped. This method can more quickly and reliably identify the abnormality of the display panel position by means of the geometric relationship of the bounding box, especially when the display panel is not fully placed or slightly tilted. Through this method, the automatic detection accuracy of the system is improved, while the misjudgment caused by offset, warping or positioning error is reduced, further optimizing the robustness and adaptability of the detection process.

[0072] Please refer to Figure 8 , an embodiment of the display panel detection device in the embodiment of the present application includes: The first shooting unit 801 is configured to shoot a target image through an industrial camera when the display panel is placed on the Tray tray; The cutting unit 802 is configured to perform image cutting on the target image according to the center coordinates of each grid in the Tray tray and the width and height of each grid stored in advance to obtain the ROI corresponding to each grid; The extraction unit 803 is configured to extract the vertex coordinates of the display panel in the target image based on the ROI; wherein, the number of vertex coordinates corresponding to each extracted display panel is 2 or 4; The first determination unit 804 is configured to, for any display panel, when the vertex coordinates of the display panel are 2, determine the warping type according to the vertex coordinates; wherein, the warping type includes long-edge warping and wide-edge warping; The calculation unit 805 is configured to calculate all vertex coordinates according to the warping type and the vertex coordinates; The second determination unit 806 is configured to determine whether there is any overlap of the display panel according to all vertex coordinates and the ROI.

[0073] In this embodiment, the display panel detection device controls an industrial camera to collect images of the Tray tray, and combines with the Tray tray template to perform operations such as image segmentation, extraction of display panel vertices, identification of warping types, vertex completion, and overlap determination, thereby realizing the automatic identification of the posture and position status of the display panel in the Tray tray. In the case where the panel is warped, occluded, or has partially missing corner points, the complete vertex coordinates can still be accurately obtained through calculation, and then it can be determined whether overlap occurs, thereby effectively improving the accuracy and robustness of abnormal panel identification.

[0074] Optionally, the first determination unit 804 is specifically configured to: Calculate the Euclidean distance between vertex coordinates; Determine the warping type according to the Euclidean distance and the standard size of the display panel.

[0075] Optionally, the calculation unit 805 is specifically configured to: Calculate the vertical unit normal vector corresponding to the edge direction vector formed by vertex coordinates according to the Euclidean distance; Calculate all vertex coordinates according to the warping type, the vertical unit normal vector, and the vertex coordinates.

[0076] Optionally, the second determination unit 806 includes: A building module for building a diagonal vector based on any diagonal of the ROI; A first determination module for determining the corresponding diagonal vertices and starting point among all vertex coordinates according to the diagonal vector; A construction module for constructing a first vector and a second vector with the starting point and the vertices other than the diagonal vertices as endpoints respectively; A second determination module for determining whether there is overlap of the display panel according to the first vector, the second vector, and the diagonal vector.

[0077] Optionally, the second determination module is specifically configured to: Calculate a first cross product value of the first vector and the diagonal vector, and calculate a second cross product value of the second vector and the diagonal vector; Calculate the difference between the absolute value of the first cross product value and the absolute value of the second cross product value; Determine whether the difference is greater than or equal to a preset threshold; If so, determine that there is overlap of the display panel; if not, determine that there is no overlap of the display panel.

[0078] Optionally, the device further includes a calibration unit, and the calibration unit is used for: When the display panel is not placed on the Tray tray, an image of the empty tray is captured by the industrial camera; Identify the center coordinates of each grid in the empty tray image through a single-grid template.

[0079] Optionally, the device further includes a third determination unit, and the third determination unit is configured to: For any display panel, when the vertex coordinates of the display panel are 4, obtain the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate based on the ROI; Determine whether there is a border for the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

[0080] In this embodiment, the functions of each unit and module correspond to the steps in the foregoing Figures 1 to 7 illustrated embodiment, and will not be elaborated herein.

[0081] Please refer to Figure 9 , an embodiment of the electronic device in the embodiment of the present application includes: A processor 901, a memory 902, an input / output unit 903, and a bus 904; The processor 901 is connected to the memory 902, the input / output unit 903, and the bus 904; A program is stored on the memory 902, and the processor 901 calls the program to execute Figures 1 to 7 the steps in the illustrated embodiment.

[0082] In this embodiment, the function of the processor 901 corresponds to the steps in the foregoing Figures 1 to 7 illustrated embodiment, and will not be elaborated herein.

[0083] The embodiment of the present application further provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the computer is caused to execute the method in any of the foregoing Figures 1 to 7 possible implementation manners.

[0084] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0085] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in an electrical, mechanical, or other form.

[0086] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0087] In addition, each functional unit in various embodiments of the present application may be integrated into a processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0088] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, read-only memory), random access memories (RAM, random access memory), magnetic disks, or optical discs that can store program codes.

Claims

1. A display panel detection method, characterized in that, Including: When the display panel is placed on the Tray, an industrial camera takes a target image; According to the pre-stored center coordinates, width, and height of each grid in the Tray, the target image is image-cut to obtain the ROI corresponding to each grid; Based on the ROI, the vertex coordinates of the display panel in the target image are extracted; among them, the number of vertex coordinates corresponding to each extracted display panel is 2 or 4; For any display panel, when the vertex coordinates of the display panel are 2, the warping type is determined according to the vertex coordinates; among them, the warping type includes long-edge warping and wide-edge warping; All vertex coordinates are calculated according to the warping type and the vertex coordinates; Whether there is an overlap edge of the display panel is determined according to all the vertex coordinates and the ROI; 2. The method according to claim 1, characterized in that The determining the warping type according to the vertex coordinates includes: Calculating the Euclidean distance between the vertex coordinates; Determining the warping type according to the Euclidean distance and the standard size of the display panel; 3. The method according to claim 2, wherein The calculating all vertex coordinates according to the warping type and the vertex coordinates includes: Calculating the vertical unit normal vector corresponding to the edge direction vector formed by the vertex coordinates according to the Euclidean distance; Calculating all vertex coordinates according to the warping type, the vertical unit normal vector, and the vertex coordinates; 4. The method according to claim 1, wherein The determining whether there is an overlap edge of the display panel according to all the vertex coordinates and the ROI includes: Based on any diagonal of the ROI, a diagonal vector is established; According to the diagonal vector, the corresponding diagonal vertices and the starting point among all the vertex coordinates are determined; Taking the vertices other than the diagonal vertices as the endpoints respectively, the first vector and the second vector are constructed with the starting point; Whether there is an overlap edge of the display panel is determined according to the first vector, the second vector, and the diagonal vector; 5. The method according to claim 4, wherein The determining whether there is an overlap edge of the display panel according to the first vector, the second vector, and the diagonal vector includes: Calculating the first cross product value of the first vector and the diagonal vector, and calculating the second cross product value of the second vector and the diagonal vector; Calculating the difference between the absolute value of the first cross product value and the absolute value of the second cross product value; Judging whether the difference is greater than or equal to a preset threshold; If so, it is determined that the display panel has an overlap edge; if not, it is determined that the display panel does not have an overlap edge; 6. The method according to any one of claims 1 to 5, characterized in that, Before taking the target image by the industrial camera, the method further includes: When the display panel is not placed on the Tray, an industrial camera takes an empty tray image; Identifying the center coordinates of each grid in the empty tray image through a single-grid template; 7. The method according to any one of claims 1 to 5, characterized in that After extracting the vertex coordinates of the display panel in the target image based on the ROI, the method further includes: For any display panel, when the vertex coordinates of the display panel are 4, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate are obtained based on the ROI; Determine whether there is an edge overlap on the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

8. A display panel detection device, characterized in that, Including: A first photographing unit, configured to photograph a target image through an industrial camera when the display panel is placed on a Tray tray. A cutting unit, configured to perform image cutting on the target image according to the center coordinates of each grid and the width and height of each grid pre-stored in the Tray tray, to obtain an ROI corresponding to each grid. An extraction unit, configured to extract vertex coordinates of the display panel in the target image based on the ROI; wherein, the number of vertex coordinates corresponding to each extracted display panel is 2 or 4. A first determination unit, configured to, for any display panel, when the number of vertex coordinates of the display panel is 2, determine a warping type according to the vertex coordinates; wherein, the warping type includes long-side warping and wide-side warping. A calculation unit, configured to calculate all vertex coordinates according to the warping type and the vertex coordinates. A second determination unit, configured to determine whether there is an edge overlap on the display panel according to the all vertex coordinates and the ROI.

9. An electronic device, characterized in that, Including: A processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored on the computer-readable storage medium, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images