Display panel detection method, device, electronic device and readable storage medium

Abstract

The embodiment of the present application discloses a display panel detection method, device, electronic device and readable storage medium for improving the robustness of detecting display panels. The embodiment of the present application method includes: when the display panel is placed on a tray, shooting a target image with an industrial camera; performing image segmentation on the target image according to the center coordinates of each grid and the width and height of each grid in the pre-stored tray to obtain the ROI corresponding to each grid; based on the ROI, extracting the vertex coordinates of the display panel in the target image; wherein the number of vertex coordinates corresponding to each display panel extracted is 2 or 4; for any display panel, when the vertex coordinates of the display panel are 2, determining the warping type according to the vertex coordinates; wherein the warping type includes long side warping and wide side warping; calculating all vertex coordinates according to the warping type and vertex coordinates; determining whether the display panel has overlapping edges according to all vertex coordinates and ROI.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of display panel detection technology, and in particular to a display panel detection method, device, electronic device, and readable storage medium. Background Art

[0002] During production processes such as panel manufacturing, packaging, and testing, automated inspection of display panels placed on trays (carriers) is often required to determine whether their placement conforms to process requirements. Trays typically contain multiple regular grids for accommodating display panels. During placement, display panels may shift, tilt, warp, or overlap. Failure to promptly detect and address these anomalies can lead to failures in subsequent handling, lamination, or packaging processes, or even damage the panel itself, resulting in a decrease in yield.

[0003] Existing inspection solutions often use industrial cameras to capture tray images and use image processing methods to determine whether the panel in each grid is misaligned, warped, or out of bounds. However, in practice, due to interference factors such as lighting conditions, panel shape errors, and image noise, existing methods are prone to misjudgment in key steps such as vertex recognition and position determination. This leads to poor inspection result stability and fails to meet the application requirements of high-precision and high-versatility production lines.

[0004] Therefore, how to improve the robustness of the display panel placement status detection method so that it can still stably extract effective features and make accurate judgments in complex environments has become an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] Embodiments of the present application provide a display panel detection method, device, electronic device, and readable storage medium, which can improve the robustness of display panel detection.

[0006] A first aspect of an embodiment of the present application provides a display panel detection method, including:

[0007] When the display panel is placed on the tray, the target image is captured by the industrial camera;

[0008] Performing image segmentation on the target image according to the pre-stored center coordinates of each grid in the tray and the width and height of each grid to obtain the ROI corresponding to each grid;

[0009] Extracting vertex coordinates of the display panel in the target image based on the ROI; wherein the number of vertex coordinates corresponding to each display panel extracted is 2 or 4;

[0010] For any display panel, when the display panel has two vertex coordinates, determining a warping type according to the vertex coordinates; wherein the warping type includes long-side warping and wide-side warping;

[0011] Calculate all vertex coordinates according to the warping type and the vertex coordinates;

[0012] It is determined whether there is an overlapping edge on the display panel according to the coordinates of all vertices and the ROI.

[0013] Optionally, determining the warping type according to the vertex coordinates includes:

[0014] Calculating the Euclidean distance between the vertex coordinates;

[0015] A warping type is determined according to the Euclidean distance and a standard size of the display panel.

[0016] Optionally, the calculating all vertex coordinates according to the warping type and the vertex coordinates includes:

[0017] Calculate the vertical unit normal vector corresponding to the edge direction vector formed by the vertex coordinates according to the Euclidean distance;

[0018] All vertex coordinates are calculated according to the warping type, the perpendicular unit normal vector, and the vertex coordinates.

[0019] Optionally, determining whether the display panel has an overlapping edge according to the coordinates of all vertices and the ROI includes:

[0020] Establishing a diagonal vector based on any diagonal line of the ROI;

[0021] Determine the corresponding diagonal vertices and starting points in all vertex coordinates according to the diagonal vector;

[0022] Taking vertices other than the diagonal vertices as end points, respectively, and constructing a first vector and a second vector with the starting point;

[0023] It is determined whether there is an overlap on the display panel according to the first vector, the second vector and the diagonal vector.

[0024] Optionally, determining whether the display panel has an overlap according to the first vector, the second vector, and the diagonal vector includes:

[0025] Calculating a first cross product value between the first vector and the diagonal vector, and calculating a second cross product value between the second vector and the diagonal vector;

[0026] Calculating a difference between an absolute value of the first cross-product value and an absolute value of the second cross-product value;

[0027] Determining whether the difference is greater than or equal to a preset threshold;

[0028] If so, it is determined that the display panel has an overlap; if not, it is determined that the display panel does not have an overlap.

[0029] Optionally, before capturing the target image with the industrial camera, the method further includes:

[0030] When the display panel is not placed on the tray, an industrial camera is used to capture an image of the empty tray.

[0031] The center coordinates of each grid in the empty disk image are identified using a single grid template.

[0032] Optionally, after extracting the vertex coordinates of the display panel in the target image based on the ROI, the method further includes:

[0033] For any display panel, when the display panel has four vertex coordinates, obtaining a maximum x-axis coordinate, a minimum x-axis coordinate, a maximum y-axis coordinate, and a minimum y-axis coordinate based on the ROI;

[0034] It is determined whether there is an overlapping edge on the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

[0035] A second aspect of an embodiment of the present application provides a display panel detection device, including:

[0036] The first shooting unit is used to shoot a target image through an industrial camera when the display panel is placed on the tray;

[0037] a cutting unit, configured to cut the target image according to the pre-stored center coordinates of each grid and the width and height of each grid in the tray, to obtain an ROI corresponding to each grid;

[0038] An extraction unit, configured to extract vertex coordinates of the display panel in the target image based on the ROI; wherein the number of vertex coordinates corresponding to each display panel extracted is 2 or 4;

[0039] a first determining unit configured to determine, for any display panel, a warping type according to the vertex coordinates when the display panel has two vertex coordinates; wherein the warping type includes long-side warping and wide-side warping;

[0040] a calculation unit, configured to calculate all vertex coordinates according to the warping type and the vertex coordinates;

[0041] The second determining unit is configured to determine whether there is an overlapping edge on the display panel according to the coordinates of all vertices and the ROI.

[0042] Optionally, the first determining unit is specifically configured to:

[0043] Calculating the Euclidean distance between the vertex coordinates;

[0044] A warping type is determined according to the Euclidean distance and a standard size of the display panel.

[0045] Optionally, the computing unit is specifically configured to:

[0046] Calculate the vertical unit normal vector corresponding to the edge direction vector formed by the vertex coordinates according to the Euclidean distance;

[0047] All vertex coordinates are calculated according to the warping type, the perpendicular unit normal vector, and the vertex coordinates.

[0048] Optionally, the second determining unit includes:

[0049] An establishing module, configured to establish a diagonal vector based on any diagonal line of the ROI;

[0050] A first determining module, configured to determine corresponding diagonal vertices and starting points in all vertex coordinates according to the diagonal vector;

[0051] A construction module, configured to construct a first vector and a second vector with the starting point and the vertices other than the diagonal vertices as end points respectively;

[0052] The second determining module is configured to determine whether there is an overlap on the display panel according to the first vector, the second vector and the diagonal vector.

[0053] Optionally, the second determining module is specifically configured to:

[0054] Calculating a first cross product value between the first vector and the diagonal vector, and calculating a second cross product value between the second vector and the diagonal vector;

[0055] Calculating a difference between an absolute value of the first cross-product value and an absolute value of the second cross-product value;

[0056] Determining whether the difference is greater than or equal to a preset threshold;

[0057] If so, it is determined that the display panel has an overlap; if not, it is determined that the display panel does not have an overlap.

[0058] Optionally, the device further includes a calibration unit, wherein the calibration unit is configured to:

[0059] When the display panel is not placed on the tray, an industrial camera is used to capture an image of the empty tray.

[0060] The center coordinates of each grid in the empty disk image are identified using a single grid template.

[0061] Optionally, the apparatus further includes a third determining unit, configured to:

[0062] For any display panel, when the display panel has four vertex coordinates, obtaining a maximum x-axis coordinate, a minimum x-axis coordinate, a maximum y-axis coordinate, and a minimum y-axis coordinate based on the ROI;

[0063] It is determined whether there is an overlapping edge on the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

[0064] A third aspect of the embodiments of the present application provides an electronic device, including:

[0065] processor, memory, input and output units, and buses;

[0066] The processor is connected to the memory, the input and output unit, and the bus;

[0067] A program is stored in the memory, and the processor calls the program to execute the method in the first aspect and any possible implementation of the first aspect.

[0068] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the computer executes the method in the first aspect and any possible implementation of the first aspect.

[0069] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0070] In the embodiments of this application, an industrial camera is controlled to capture tray images and, in conjunction with a tray template, image segmentation, panel vertex extraction, warp type identification, vertex completion, and overlap determination are performed to automatically identify the posture and position of the display panel in the tray. Even if the panel is warped, obscured, or has partially missing corner points, the complete vertex coordinates can still be accurately calculated to determine whether overlap has occurred, effectively improving the accuracy and robustness of abnormal panel identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a flow chart of an embodiment of a display panel detection method in an embodiment of the present application;

[0072] Figure 2 This is a flow chart of an embodiment of determining the warpage type in the embodiment of the present application;

[0073] Figure 3 This is a flow chart of an embodiment of calculating the coordinates of all vertices in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of a process for determining whether a display panel has overlap in an embodiment of the present application;

[0075] Figure 5 This is a flow chart of another embodiment of determining whether a display panel has overlap in an embodiment of the present application;

[0076] Figure 6 This is a flow chart of an embodiment of identifying the center coordinates of each grid in an empty disk image in an embodiment of the present application;

[0077] Figure 7 This is a flow chart of another embodiment of determining whether a display panel has overlap in an embodiment of the present application;

[0078] Figure 8 This is a structural diagram of an embodiment of a display panel detection device in an embodiment of the present application;

[0079] Figure 9 This is a structural diagram of an embodiment of an electronic device in the embodiments of the present application. DETAILED DESCRIPTION

[0080] Embodiments of the present application provide a display panel detection method, device, electronic device, and readable storage medium for improving the robustness of display panel detection.

[0081] The method of the present application can be applied to a server, a terminal or other device with logic processing capabilities, and the present application does not limit this. For the convenience of description, the following description is based on an example in which the execution subject is a server.

[0082] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0083] See also Figure 1 In one embodiment of the display panel detection method of the present application, the following steps are included:

[0084] 101. When the display panel is placed on the tray, the target image is captured by the industrial camera;

[0085] After the inspection process starts, the server controls the connected industrial camera to capture the display panels on the tray (also known as the material tray or carrier tray), acquiring a high-resolution image of the multiple panels as the target image. Industrial cameras offer high frame rates, high resolution, and low distortion, making them suitable for high-precision image acquisition. Trays are typically arranged in a matrix, with multiple grids for holding display panels, each panel individually placed in a grid. This step aims to capture a single image of all panels to be inspected, laying the foundation for subsequent image processing.

[0086] 102. Perform image segmentation on the target image according to the pre-stored center coordinates of each grid in the tray and the width and height of each grid to obtain the ROI corresponding to each grid;

[0087] The server retrieves pre-calibrated and stored tray template information from a local database. This template records the center coordinates, width (i.e., horizontal dimension), and height (i.e., vertical dimension) of each grid in the tray. Based on these parameters, the server performs image segmentation on the target image acquired in step 101, dividing the entire image into several regions, each called a Region of Interest (ROI). Each ROI corresponds to a grid in the tray, i.e., the expected location of a display panel. Image segmentation uses a rectangular cropping method based on a geometric template to enable individual processing of each small region.

[0088] 103. Extract vertex coordinates of the display panel in the target image based on the ROI; wherein the number of vertex coordinates corresponding to each display panel extracted is 2 or 4;

[0089] The server performs image feature extraction on each ROI image to identify the vertex coordinate information of the display panel. Identification methods are usually based on edge detection (such as the Canny algorithm), contour extraction (such as the findContours function in OpenCV), and corner detection (such as Harris corners and Shi-Tomasi corners), combined with geometric screening rules (such as side ratio and angle range) to locate the vertices of the display panel. Because some panels may be warped or obscured, only two vertex coordinates can be extracted instead of the full four vertices. The server records the number of vertices extracted and the corresponding coordinates for each display panel, providing a data basis for subsequent inference processing.

[0090] 104. For any display panel, when the display panel has two vertex coordinates, determine a warping type according to the vertex coordinates; wherein the warping type includes long side warping and wide side warping;

[0091] For each display panel for which only two vertex coordinates have been extracted, the server further determines the panel's warp type. Warp refers to deformation of the panel due to stress or heat, causing one side to lose contact with the bottom of the tray. The server makes this determination by calculating the relative position of the two vertices in the ROI image (for example, whether the horizontal distance is greater than the vertical distance). If the vertices are primarily distributed horizontally at the ends of the image, it is considered "long-side warp"; if they are primarily distributed vertically, it is considered "wide-side warp." This classification helps subsequent vertex completion operations accurately simulate warp trends.

[0092] 105. Calculate all vertex coordinates according to the warping type and vertex coordinates;

[0093] The server combines the warping type determined in step 104 with the existing two vertex coordinates, using geometric inference or rule-based modeling to calculate and complete the remaining two vertices. For example, when the long side is warped, the server uses the two short sides as known edges and, using the known point coordinates, side length ratio, and preset panel length and width parameters, infers the coordinates of the four vertices of the complete rectangle. Completion methods may include symmetric mapping, boundary extension, and rectangular box fitting. The final four vertex coordinates are used for subsequent positioning and analysis to ensure the integrity of the panel status judgment.

[0094] 106. Determine whether there is any overlapping edge on the display panel based on all vertex coordinates and the ROI.

[0095] Based on the coordinates of the four complete vertices calculated in step 105 and the boundary information of the ROI to which the display panel belongs, the server determines whether the panel has overlap. Overlapping occurs when any part of the panel (usually a vertex or edge) extends beyond the boundaries of the grid to which it belongs. This can cause robotic handling failure, material jamming during loading and unloading, or subsequent assembly and alignment failure. This determination can be made by calculating whether the panel's vertices are outside the ROI, or whether the panel's rectangular frame intersects or exceeds the ROI boundary. If overlap is determined, the server can mark the panel as abnormal, allowing subsequent processes to skip or process it.

[0096] The server automatically identifies the display panel's posture and position by controlling an industrial camera to capture tray images. Using tray templates, the server performs image segmentation, panel vertex extraction, warp type identification, vertex completion, and overlap detection. Even if the panel is warped, obscured, or has missing corner points, it can still accurately calculate complete vertex coordinates to determine whether overlap exists, effectively improving the accuracy and robustness of abnormal panel identification.

[0097] See also Figure 2In some embodiments of the present application, step 104 in the above embodiment is for any display panel. When the display panel has two vertex coordinates, determining the warping type according to the vertex coordinates may include the following steps:

[0098] 201. Calculate the Euclidean distance between vertex coordinates;

[0099] For any display panel that only extracts two vertex coordinates, the server can obtain the image coordinates of the corresponding two vertices (usually pixel-level coordinate points), recorded as point . The server then uses the Euclidean distance formula:

[0100] Formula 1

[0101] Calculate the straight-line distance between two points to obtain the actual pixel distance between vertices. Euclidean distance is used here to measure the geometric separation between two vertices, providing a preliminary assessment of whether the two points are likely to lie on either the long or wide edge of the panel. The server temporarily stores the calculated results for use in the next step, performing a size ratio comparison.

[0102] 202. Determine the warping type according to the Euclidean distance and the standard size of the display panel.

[0103] The server converts the Euclidean distance calculated in step 201 into physical units (e.g., long side L, short side W, in millimeters) based on the pre-set standard display panel dimensions (e.g., long side L, short side W). (If calibration data is used, pixel distance can be directly mapped to millimeters.) The server compares the converted distance with the standard long side L and short side W of the panel to determine which dimension it is closer to:

[0104] If the Euclidean distance is close to L, the server determines that the two points are located at the two ends of the long side of the panel, that is, the long side of the panel is warped;

[0105] If the Euclidean distance is close to W, the server determines that the two points are located at the two ends of the short side of the panel, that is, the panel is warped at the wide side.

[0106] To improve robustness, the server can set a threshold interval (e.g., ±5%) as a proximity condition to avoid misjudgment due to image deformation, tilt, or noise.

[0107] The server further calculates the Euclidean distance between the extracted two vertex coordinates and compares them to standard dimensions. This allows it to accurately identify the boundary type (long-side warping or wide-side warping) of the panel warp, even with only partial corner information. This determination provides directional guidance for subsequent vertex completion, ensuring that even if the panel's posture is abnormal or image information is missing, the server can still reconstruct its complete bounding box, improving the reliability and accuracy of automatic positioning and anomaly identification.

[0108] See also Figure 3 In some embodiments of the present application, step 105 in the above embodiment calculates all vertex coordinates according to the warping type and vertex coordinates, and may include the following steps:

[0109] 301. Calculate the perpendicular unit normal vector corresponding to the edge direction vector formed by the vertex coordinates according to the Euclidean distance;

[0110] Assume that the coordinates of the two vertices are ,vector is the identified edge direction vector, then , the server can calculate the vertical unit normal vector according to Formula 2:

[0111] Formula 2

[0112] in, is the vertical unit normal vector, is the Euclidean distance between two vertices.

[0113] 302. Calculate all vertex coordinates according to the warping type, the vertical unit normal vector and the vertex coordinates.

[0114] If the warping type is broadside warping, the server can calculate the coordinates of the other two vertices according to the following formula:

[0115] Formula 3

[0116] Formula 4

[0117] Wherein, L is the standard long side size of the display panel, is the x component of the vertical unit normal vector (i.e. ), is the y component of the vertical unit normal vector (i.e. ).

[0118] If the warping type is long-edge warping, the server can calculate the coordinates of the other two vertices according to the following formula:

[0119] Formula 5

[0120] Formula 6

[0121] Where W is the standard wide side size of the display panel, is the x component of the vertical unit normal vector (i.e. ), is the y component of the vertical unit normal vector (i.e. ).

[0122] The server constructs a direction vector based on the edge formed by the two known vertices and further derives the perpendicular unit normal vector. This, combined with the warping type and standard dimensions, accurately calculates the positions of the remaining two vertices, successfully constructing the complete panel bounding box. This process ensures that even if only two corner points are identified in the initial image, the complete rectangle can be restored through geometric relationships and vector calculations. This effectively improves vertex completion capabilities and positioning accuracy in the presence of image defects, occlusions, or warping, providing a solid foundation for subsequent functions such as edge detection and posture correction.

[0123] See also Figure 4 In some embodiments of the present application, step 106 in the above embodiment may include the following steps to determine whether there is an overlap on the display panel based on all vertex coordinates and the ROI:

[0124] 401. Establish a diagonal vector based on any diagonal line of the ROI;

[0125] Based on the rectangular area of ​​the corresponding grid in each ROI, the server selects the two diagonal points of the ROI and establishes a diagonal vector based on these two points. This vector represents the reference direction of the ROI in the Tray disk and is used for subsequent geometric comparison with the vector formed by the display panel vertices. The server usually selects the upper left corner and the lower right corner, or the upper right corner and the lower left corner as the starting and ending points of the diagonal line, and calculates the vector based on their coordinate difference. For example, from the upper left corner point Go to the lower right corner , generating the vector:

[0126] Formula 7

[0127] The server uses this vector as a reference to determine whether the vertex of the display panel exceeds the edge of the frame.

[0128] 402. Determine the corresponding diagonal vertices and starting points in all vertex coordinates according to the diagonal vectors;

[0129] After obtaining the diagonal vector, the server can compare all vertex coordinates and determine which pair of vertices is closest in spatial position to the diagonal endpoints, respectively serving as the "diagonal start point" and "diagonal end point." This pair of vertices is the diagonal line on the panel closest to the ROI diagonal direction, used as the reference line direction for subsequent edge detection. For example, if the diagonal vector selected in step 401 is the vector from the upper left corner (starting point) to the lower right corner (end point), then the diagonal vertices of the display panel determined by the server are also the upper left corner and the lower right corner, and the starting point is also the upper left corner.

[0130] 403. Use vertices other than the diagonal vertices as the end points and construct the first vector and the second vector with the starting point;

[0131] The server uses the remaining two vertices as endpoints, and constructs a first vector and a second vector with the determined diagonal starting point. These two vectors represent the direction of the edge extension of the display panel within the ROI. The three vectors (diagonal vector, first vector, and second vector) together define the spatial relationship of the panel in the region, providing a basis for subsequent calculations of geometric quantities such as angles or cross products. For example, based on the selection in step 402, the server can construct a vector from the upper left corner to the lower left corner, and a vector from the upper left corner to the upper right corner. These two vectors serve as the first vector and the second vector. In this embodiment, there is no restriction on which of the two serves as the first vector, and the same applies to the second vector.

[0132] 404. Determine whether there is an overlap on the display panel according to the first vector, the second vector, and the diagonal vector.

[0133] The server analyzes the display panel for overlap based on the spatial geometric relationship between the first and second vectors and the diagonal vector. The server determines overlap by calculating the relative orientations between these vectors (e.g., the angle or the sign and magnitude of the cross product) to determine whether the panel is in abnormal contact with the ROI boundary or exceeds the grid range.

[0134] By constructing a ROI diagonal vector and using it as a reference to accurately calculate the display panel's vertex orientation and spatial relationships, the server can determine whether the display panel has deviated from its grid and exhibited overlapping. This method, which does not rely on the panel's absolute position but instead makes spatial layout judgments based on relative vector geometry, offers greater adaptability and fault tolerance, effectively avoiding misjudgments caused by image noise, warping, or angular deviation, improving detection accuracy and ensuring the reliability of automated recognition.

[0135] See also Figure 5In some embodiments of the present application, step 404 in the above embodiment determines whether there is an overlap on the display panel based on the first vector, the second vector, and the diagonal vector, and may include the following steps:

[0136] 501. Calculate a first cross product value between the first vector and the diagonal vector, and calculate a second cross product value between the second vector and the diagonal vector;

[0137] The server uses the diagonal vector as a reference to calculate the spatial geometric relationship between the first vector and the diagonal vector, obtaining a first cross product value. It then calculates the spatial geometric relationship between the second vector and the diagonal vector to obtain a second cross product value. These two cross product values ​​respectively reflect the relative offset between the first vector and the second vector in the direction perpendicular to the diagonal, and can characterize the geometric deviation between the edge of the display panel and the diagonal line of the ROI. Specifically, the server can calculate the first cross product value and the second cross product value according to the following formula:

[0138] Formula 8

[0139] in, is the cross product value, when i is 2, it is the first cross product value, and when i is 4, it is the second cross product value.

[0140] 502. Calculate the difference between the absolute value of the first cross product value and the absolute value of the second cross product value;

[0141] The server compares the absolute value of the first cross-product value with the absolute value of the second cross-product value and calculates the difference between them. This difference reflects the degree of symmetry deviation between the left and right sides (or top and bottom) of the display panel based on the diagonal reference. A larger difference indicates a more significant difference in edge deviation and a more uneven overall panel placement.

[0142] 503. Determine whether the difference is greater than or equal to a preset threshold;

[0143] The server compares this difference with a preset threshold to determine whether the display panel has structural offset anomalies. If the difference is greater than or equal to the threshold, the offset difference is significant, indicating a risk of overlap. If the difference is less than the threshold, the offset difference is minor and considered within the acceptable range.

[0144] 504. Determine whether the display panel has overlap.

[0145] When the difference is greater than or equal to a preset threshold, the server determines that the display panel has overlap, indicating that at least a portion of the edge of the panel has crossed the effective boundary of the grid area, which may affect subsequent processes or quality inspections.

[0146] 505. Determine that there is no overlap on the display panel.

[0147] When the difference is less than a preset threshold, the server determines that there is no overlapping edge of the display panel, indicating that the panel is well placed and the four vertices are all within the reasonable ROI area.

[0148] Based on the symmetry of vector cross product results, the server successfully established a mathematical mechanism for detecting overlapping display panels by calculating the difference in the projected area between two non-diagonal edges and the ROI diagonal. This method, which does not rely on pixel-level contour analysis or image segmentation accuracy, exhibits excellent robustness and interference resistance, effectively improving the accuracy and efficiency of overlapping detection and providing strong mathematical support for batch inspection and intelligent recognition using industrial cameras.

[0149] See also Figure 6 In some embodiments of the present application, the display panel detection method may further include the following steps:

[0150] 601. When the display panel is not placed on the tray, an image of the empty tray is captured by an industrial camera;

[0151] During the initialization phase of the inspection process, the server controls an industrial camera to capture an image of the tray, currently empty of a display panel. This image captures only the tray itself and its regularly spaced spaces, unobstructed by the display panel. The image's edges are clear, facilitating subsequent spatial structure analysis. This step establishes a reference model of the current tray structure, providing a baseline coordinate system for subsequent panel inspection.

[0152] 602. Identify the center coordinates of each grid in the empty disk image using a single grid template.

[0153] The server performs template matching on the empty tray image based on a pre-trained or pre-defined single-grid template image to locate the center coordinates of each tray grid. Here, "single-grid template" refers to a pre-defined standard tray grid area pattern that includes typical features such as border structure, shadow outline, or material texture, ensuring uniqueness and matching.

[0154] The server slides the template window in the image, calculates the similarity through matching algorithms (such as normalized cross correlation (NCC) or structural similarity (SSIM), identifies all matching positions, and extracts the corresponding geometric center coordinates. grids, then the center coordinates of each grid are , where i=1,2,...,M, j=1,2,...,N.

[0155] The server incorporates blank tray image capture and single-grid template recognition before the display panel inspection process. This automatically constructs the tray coordinate system and grid layout information without manual calibration or pre-set models. This method, based on real-world image data, effectively adapts to tray structures of varying types, sizes, and layouts, enhancing the system's versatility and flexibility across diverse production lines. It also provides a precise spatial reference for subsequent steps like panel positioning and ROI extraction, improving overall inspection accuracy.

[0156] See also Figure 7 In some embodiments of the present application, after extracting the vertex coordinates of the display panel in the target image based on the ROI in step 103 of the above embodiment, the display panel detection method may further include the following steps:

[0157] 701. For any display panel, when the display panel has four vertex coordinates, obtain a maximum x-axis coordinate, a minimum x-axis coordinate, a maximum y-axis coordinate, and a minimum y-axis coordinate based on the ROI;

[0158] After the server extracts the coordinates of the four vertices of the display panel from the ROI, it first calculates the maximum and minimum values ​​of the x-axis and y-axis in the vertex coordinates to obtain the bounding box range of the display panel in the ROI. The specific steps are as follows:

[0159] Maximum x-axis coordinate ( ): Select the maximum x coordinate from the four vertex coordinates.

[0160] Minimum x-axis coordinate ( ): Select the minimum x coordinate from the four vertex coordinates.

[0161] Maximum y-axis coordinate ( ): Select the maximum y coordinate from the four vertex coordinates.

[0162] Minimum y-axis coordinate ( ): Select the minimum y coordinate from the four vertex coordinates.

[0163] These coordinate values ​​together define a bounding box, which defines the display panel's position within the ROI. This step further quantitatively analyzes the display panel's spatial position, providing geometric data support for subsequent edge detection.

[0164] 702. Determine whether there is an overlapping edge on the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

[0165] The server compares the bounding box coordinates (maximum x, minimum x, maximum y, minimum y) obtained in step 701 with the extracted four vertex coordinates to determine whether there is an overlap on the display panel. The server can determine this using the following formula:

[0166] Formula 9

[0167] If any x value or y value among the four vertex coordinates does not conform to the above formula 9, it can be determined that there is an overlap on the display panel.

[0168] Otherwise, if all vertices are within the bounding box, it is determined that the display panel does not have overlapping edges.

[0169] By comparing the maximum and minimum coordinate values ​​with the bounding box of the ROI, the server can accurately determine whether the display panel has crossed or overlapped its boundaries. Leveraging the geometric relationships of the bounding box, this method can more quickly and reliably identify anomalies in the display panel's position, especially when the display panel is not fully positioned or slightly tilted. This approach improves the system's automated inspection accuracy while reducing false positives caused by offset, warping, or positioning errors, further optimizing the robustness and adaptability of the inspection process.

[0170] See also Figure 8 In one embodiment of the present application, a display panel detection device includes:

[0171] The first shooting unit 801 is used to shoot a target image using an industrial camera when the display panel is placed on the tray;

[0172] The cutting unit 802 is used to cut the target image according to the center coordinates of each grid and the width and height of each grid stored in the tray, and obtain the ROI corresponding to each grid;

[0173] An extraction unit 803 is configured to extract vertex coordinates of the display panels in the target image based on the ROI; wherein the number of vertex coordinates corresponding to each display panel extracted is 2 or 4;

[0174] A first determining unit 804 is configured to determine, for any display panel, a warping type according to the vertex coordinates when the display panel has two vertex coordinates; wherein the warping type includes long-side warping and wide-side warping;

[0175] A calculation unit 805 is used to calculate all vertex coordinates according to the warping type and the vertex coordinates;

[0176] The second determining unit 806 is configured to determine whether there is an overlapping edge on the display panel according to all vertex coordinates and the ROI.

[0177] In this embodiment, the display panel inspection device automatically identifies the display panel's posture and position within the tray by controlling an industrial camera to capture tray images and, in conjunction with a tray template, performing image segmentation, display panel vertex extraction, warp type identification, vertex completion, and overlap determination. Even in the presence of warping, occlusion, or partially missing corner points, the device can still accurately calculate complete vertex coordinates to determine whether overlap exists, effectively improving the accuracy and robustness of abnormal panel identification.

[0178] Optionally, the first determining unit 804 is specifically configured to:

[0179] Calculate the Euclidean distance between vertex coordinates;

[0180] The warping type is determined based on the Euclidean distance and the standard size of the display panel.

[0181] Optionally, the calculation unit 805 is specifically configured to:

[0182] Calculate the vertical unit normal vector corresponding to the edge direction vector formed by the vertex coordinates based on the Euclidean distance;

[0183] Calculates all vertex coordinates based on the warp type, perpendicular unit normal, and vertex coordinates.

[0184] Optionally, the second determining unit 806 includes:

[0185] A building module for building a diagonal vector based on any diagonal line of the ROI;

[0186] A first determining module is used to determine the corresponding diagonal vertices and starting points in all vertex coordinates according to the diagonal vector;

[0187] A construction module, configured to construct a first vector and a second vector with vertices other than the diagonal vertices as end points and the starting point respectively;

[0188] The second determining module is configured to determine whether there is an overlap on the display panel according to the first vector, the second vector and the diagonal vector.

[0189] Optionally, the second determining module is specifically configured to:

[0190] Calculate a first cross product value between the first vector and the diagonal vector, and calculate a second cross product value between the second vector and the diagonal vector;

[0191] Calculate the difference between the absolute value of the first cross product value and the absolute value of the second cross product value;

[0192] Determine whether the difference is greater than or equal to a preset threshold;

[0193] If so, it is determined that the display panel has an overlap; if not, it is determined that the display panel does not have an overlap.

[0194] Optionally, the device further includes a calibration unit, which is configured to:

[0195] When the display panel is not placed on the tray, an industrial camera is used to capture an image of the empty tray.

[0196] The center coordinates of each grid in the empty disk image are identified using a single grid template.

[0197] Optionally, the device further includes a third determining unit, configured to:

[0198] For any display panel, when the display panel has four vertex coordinates, obtain the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate based on the ROI;

[0199] It is determined whether there is an overlap on the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

[0200] In this implementation, the functions of each unit and module are the same as those mentioned above. Figures 1 to 7 The steps in the illustrated embodiment correspond to each other and will not be repeated here.

[0201] See also Figure 9 In the embodiments of the present application, an electronic device includes:

[0202] Processor 901, memory 902, input and output unit 903 and bus 904;

[0203] The processor 901 is connected to the memory 902, the input and output unit 903, and the bus 904;

[0204] The memory 902 stores a program, and the processor 901 calls the program to execute Figures 1 to 7 Steps in the illustrated embodiment.

[0205] In this embodiment, the function of the processor 901 is the same as that of the aforementioned Figures 1 to 7 The steps in the illustrated embodiment correspond to each other and will not be repeated here.

[0206] The embodiment of the present application further provides a computer-readable storage medium having a program stored thereon, which, when executed on a computer, causes the computer to execute the aforementioned Figures 1 to 7 A method in any possible embodiment.

[0207] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0208] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0210] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0211] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A display panel detection method, characterized in that: include: When the display panel is placed on the tray, the target image is captured by the industrial camera; Performing image segmentation on the target image according to the pre-stored center coordinates of each grid in the tray and the width and height of each grid to obtain the ROI corresponding to each grid; Extracting vertex coordinates of the display panel in the target image based on the ROI; wherein the number of vertex coordinates corresponding to each display panel extracted is 2 or 4; For any display panel, when the display panel has two vertex coordinates, determining a warping type according to the vertex coordinates; wherein the warping type includes long-side warping and wide-side warping; Calculate all vertex coordinates according to the warping type and the vertex coordinates; It is determined whether there is an overlapping edge on the display panel according to the coordinates of all vertices and the ROI.

2. The method according to claim 1, characterized in that Determining the warping type according to the vertex coordinates includes: Calculating the Euclidean distance between the vertex coordinates; A warping type is determined according to the Euclidean distance and a standard size of the display panel.

3. The method according to claim 2, characterized in that Calculating all vertex coordinates according to the warping type and the vertex coordinates includes: Calculate the vertical unit normal vector corresponding to the edge direction vector formed by the vertex coordinates according to the Euclidean distance; All vertex coordinates are calculated according to the warping type, the perpendicular unit normal vector, and the vertex coordinates.

4. The method according to claim 1, wherein Determining whether there is an overlapping edge on the display panel according to all vertex coordinates and the ROI includes: Establishing a diagonal vector based on any diagonal line of the ROI; Determine the corresponding diagonal vertices and starting points in all vertex coordinates according to the diagonal vector; Taking vertices other than the diagonal vertices as end points, respectively, and constructing a first vector and a second vector with the starting point; It is determined whether there is an overlap on the display panel according to the first vector, the second vector and the diagonal vector.

5. The method according to claim 4, characterized in that The determining whether the display panel has an overlap according to the first vector, the second vector, and the diagonal vector includes: Calculating a first cross product value between the first vector and the diagonal vector, and calculating a second cross product value between the second vector and the diagonal vector; Calculating a difference between an absolute value of the first cross-product value and an absolute value of the second cross-product value; Determining whether the difference is greater than or equal to a preset threshold; If so, it is determined that the display panel has an overlap; if not, it is determined that the display panel does not have an overlap.

6. The method according to any one of claims 1 to 5, characterized in that Before capturing the target image by the industrial camera, the method further includes: When the display panel is not placed on the tray, an industrial camera is used to capture an image of the empty tray. The center coordinates of each grid in the empty disk image are identified using a single grid template.

7. The method according to any one of claims 1 to 5, characterized in that After extracting the vertex coordinates of the display panel in the target image based on the ROI, the method further includes: For any display panel, when the display panel has four vertex coordinates, obtaining a maximum x-axis coordinate, a minimum x-axis coordinate, a maximum y-axis coordinate, and a minimum y-axis coordinate based on the ROI; It is determined whether there is an overlapping edge on the display panel according to the vertex coordinates, the maximum x-axis coordinate, the minimum x-axis coordinate, the maximum y-axis coordinate, and the minimum y-axis coordinate.

8. A display panel detection device, characterized in that: include: The first shooting unit is used to shoot a target image through an industrial camera when the display panel is placed on the tray; a cutting unit, configured to cut the target image according to the pre-stored center coordinates of each grid and the width and height of each grid in the tray, to obtain an ROI corresponding to each grid; An extraction unit, configured to extract vertex coordinates of the display panel in the target image based on the ROI; wherein the number of vertex coordinates corresponding to each display panel extracted is 2 or 4; a first determining unit configured to determine, for any display panel, a warping type according to the vertex coordinates when the display panel has two vertex coordinates; wherein the warping type includes long-side warping and wide-side warping; a calculation unit, configured to calculate all vertex coordinates according to the warping type and the vertex coordinates; The second determining unit is configured to determine whether there is an overlapping edge on the display panel according to the coordinates of all vertices and the ROI.

9. An electronic device, characterized in that: include: processor, memory, input and output units, and buses; The processor is connected to the memory, the input and output unit, and the bus; A program is stored in the memory, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images