A large panel size measurement method and system with multi-camera joint automatic calibration

Abstract

The present application discloses a method and system for measuring the size of a large panel with multi-camera joint automatic calibration, which is used to achieve efficient and accurate large panel size detection. The method of the present application includes: obtaining a calibration image of the corresponding area of ​​the calibration plate by shooting with at least two cameras; extracting the centroid coordinates of the target characters in the calibration image and the circular area in the calibration image; calculating the physical coordinates of the calibration plate based on the actual size information of the target characters and the calibration plate, and establishing an affine transformation matrix between the camera and the calibration plate according to the centroid coordinates and the physical coordinates; obtaining a regional image of the display panel to be measured by shooting with the camera, and extracting the coordinates of the corner points of the display panel to be measured based on the regional image; mapping the corner point coordinates to the coordinate system of the calibration plate through the affine transformation matrix, obtaining the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and calculating the size information of the display panel to be measured based on the target coordinates.

Description

Technical Field

[0001] The present application relates to the technical field of large panel size measurement using multi-camera joint automatic calibration, and in particular to a large panel size measurement method and system using multi-camera joint automatic calibration. Background Art

[0002] With the rapid development of modern industrial manufacturing technology, large-size panels are increasingly used in displays, touch screens, photovoltaic modules, semiconductor substrates, and other fields, and market demand continues to grow. As core components of these high-tech products, the dimensional accuracy of large panels directly affects product performance, quality, and production efficiency. For example, in flat-panel display production, even slight deviations in panel size can result in display artifacts or assembly difficulties.

[0003] In the field of large-panel size inspection, due to the high requirements for large panel size inspection and the low error tolerance, the accuracy of size inspection using a single camera often fails to meet the required standards. Therefore, the existing technology generally requires the use of a multi-camera joint calibration method to improve measurement accuracy. However, most existing multi-camera calibration methods require manual input of the coordinate numbers on the calibration plate, which is cumbersome and inefficient. Furthermore, the results of manual input by different personnel can vary. This human error directly affects the consistency and reliability of the calibration results, resulting in reduced overall calibration accuracy and measurement system stability. Summary of the Invention

[0004] The present application provides a large panel size measurement method and system with multi-camera joint automatic calibration, which is used to achieve efficient and accurate large panel size detection.

[0005] The first aspect of the present application provides a large panel size measurement method using multi-camera joint automatic calibration, comprising:

[0006] A calibration image of a corresponding area of ​​a calibration plate is obtained by photographing with at least two cameras, wherein a plurality of circular areas are regularly distributed on the calibration plate, and each of the circular areas contains corresponding row and column index character information;

[0007] Extracting the target character in the calibration image and the center of gravity coordinates of the circular area in the calibration image;

[0008] Calculating the physical coordinates of the calibration plate based on the actual size information of the target character and the calibration plate, and establishing an affine transformation matrix between the camera and the calibration plate according to the center of gravity coordinates and the physical coordinates;

[0009] Obtaining a regional image of the display panel to be tested by photographing with the camera, and extracting the coordinates of corner points of the display panel to be tested according to the regional image;

[0010] The corner point coordinates are mapped to the coordinate system of the calibration plate through the affine transformation matrix to obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and the size information of the display panel to be measured is calculated based on the target coordinates.

[0011] Optionally, extracting the target character in the calibration image and the center of gravity coordinates of the circular area in the calibration image includes:

[0012] Binarizing the calibration image, and performing hole filling and connected domain segmentation on the binarized calibration image;

[0013] Calculating the circularity and area characteristics of each connected domain, and screening out valid circular areas in the calibration image based on a preset threshold;

[0014] The target character in the effective circular area is extracted, and the coordinates of the center of gravity of the effective circular area are calculated.

[0015] Optionally, extracting the target character in the valid circular area includes:

[0016] Correcting the calibration image by image rotation or perspective transformation so that the effective circular areas are located in the same horizontal direction;

[0017] Extracting target characters in the effective circular area, and performing horizontal and vertical segmentation on the target characters to obtain corresponding single characters;

[0018] The single character is differentially compared with a standard character to determine the character content of the single character.

[0019] Optionally, after performing a differential comparison between the single character and a standard character to determine the character content of the single character, the method further includes:

[0020] All the identified character contents are detected for common rows and columns, and abnormal character contents detected are corrected.

[0021] Optionally, calculating the circularity and area feature values ​​of each connected domain and screening out valid circular areas in the calibration image based on a preset threshold value includes:

[0022] Calculate the circularity and area characteristics of each connected domain, and calculate the average area of ​​several connected domains with the top 10% area size;

[0023] setting a dynamic area threshold based on the average area and a preset deviation parameter;

[0024] A valid circular area in the calibration image is screened out based on the dynamic area threshold.

[0025] Optionally, the calculating the physical coordinates of the calibration plate based on the actual size information of the target character and the calibration plate, and establishing an affine transformation matrix between the camera and the calibration plate according to the barycenter coordinates and the physical coordinates, includes:

[0026] Converting the target character into the physical coordinates of the circular area according to the distance between adjacent circle centers on the calibration plate;

[0027] Pairing the center of gravity coordinates of each circular area with the corresponding physical coordinates to form a point pair set;

[0028] An affine transformation matrix between the camera and the calibration plate is calculated based on the point pair set, and the affine transformation matrix is ​​iteratively optimized using a RANSAC algorithm.

[0029] Optionally, obtaining a regional image of the display panel to be tested by photographing with the camera, and extracting the coordinates of corner points of the display panel to be tested according to the regional image, includes:

[0030] The camera is used to obtain a regional image of the display panel to be tested, and a caliper tool is used to extract sub-pixel coordinate points of the edge of the screen to generate an edge point set;

[0031] Fitting the edge point set by the least square method to obtain a fitting straight line corresponding to the edge of the screen;

[0032] The coordinates of the corner points of the display panel to be tested are determined according to the intersection points of the fitted straight lines.

[0033] A second aspect of the present application provides a large panel size measurement system with multi-camera joint automatic calibration, comprising:

[0034] A first shooting unit is configured to obtain a calibration image of an area corresponding to a calibration plate by shooting with at least two cameras, wherein the calibration plate has a plurality of circular areas regularly distributed thereon, and each of the circular areas contains corresponding row and column index character information;

[0035] an extraction unit, configured to extract the target character in the calibration image and the centroid coordinates of the circular area in the calibration image;

[0036] a calibration unit, configured to calculate the physical coordinates of the calibration plate based on the actual size information of the target character and the calibration plate, and establish an affine transformation matrix between the camera and the calibration plate according to the barycenter coordinates and the physical coordinates;

[0037] a second shooting unit, configured to obtain a regional image of the display panel to be tested by shooting with the camera, and extract the coordinates of corner points of the display panel to be tested according to the regional image;

[0038] The detection unit is used to map the corner point coordinates to the coordinate system of the calibration plate through the affine transformation matrix, obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and calculate the size information of the display panel to be measured based on the target coordinates.

[0039] Optionally, the extraction unit is specifically configured to:

[0040] Binarizing the calibration image, and performing hole filling and connected domain segmentation on the binarized calibration image;

[0041] Calculating the circularity and area characteristics of each connected domain, and screening out valid circular areas in the calibration image based on a preset threshold;

[0042] The target character in the effective circular area is extracted, and the coordinates of the center of gravity of the effective circular area are calculated.

[0043] Optionally, the extraction unit is further configured to:

[0044] Correcting the calibration image by image rotation or perspective transformation so that the effective circular areas are located in the same horizontal direction;

[0045] Extracting target characters in the effective circular area, and performing horizontal and vertical segmentation on the target characters to obtain corresponding single characters;

[0046] The single character is differentially compared with a standard character to determine the character content of the single character.

[0047] Optionally, the extraction unit is further configured to:

[0048] All the identified character contents are detected for common rows and columns, and abnormal character contents detected are corrected.

[0049] Optionally, the extraction unit is further configured to:

[0050] Calculate the circularity and area characteristics of each connected domain, and calculate the average area of ​​several connected domains with the top 10% area size;

[0051] setting a dynamic area threshold based on the average area and a preset deviation parameter;

[0052] A valid circular area in the calibration image is screened out based on the dynamic area threshold.

[0053] Optionally, the calibration unit is specifically used to:

[0054] Converting the target character into the physical coordinates of the circular area according to the distance between adjacent circle centers on the calibration plate;

[0055] Pairing the center of gravity coordinates of each circular area with the corresponding physical coordinates to form a point pair set;

[0056] An affine transformation matrix between the camera and the calibration plate is calculated based on the point pair set, and the affine transformation matrix is ​​iteratively optimized using a RANSAC algorithm.

[0057] Optionally, the second shooting unit is specifically configured to:

[0058] The camera is used to obtain a regional image of the display panel to be tested, and a caliper tool is used to extract sub-pixel coordinate points of the edge of the screen to generate an edge point set;

[0059] Fitting the edge point set by the least square method to obtain a fitting straight line corresponding to the edge of the screen;

[0060] The coordinates of the corner points of the display panel to be tested are determined according to the intersection points of the fitted straight lines.

[0061] A third aspect of the present application provides a large panel size measurement device with multi-camera joint automatic calibration, the device comprising:

[0062] processor, memory, input and output units, and buses;

[0063] The processor is connected to the memory, the input and output unit, and the bus;

[0064] The memory stores a program, and the processor calls the program to execute the first aspect and any optional large panel size measurement method of multi-camera joint automatic calibration in the first aspect.

[0065] In a fourth aspect, the present application provides a computer-readable storage medium having a program stored thereon. When the program is executed on a computer, the method for measuring the size of a large panel with multi-camera joint automatic calibration according to the first aspect and any optional method in the first aspect is executed.

[0066] It can be seen from the above technical solutions that this application has the following advantages:

[0067] The system automatically captures coordinate information on a calibration plate with row and column index characters by capturing it with a camera. Character recognition technology is then used to automatically obtain coordinate information on the plate. The physical coordinates are calculated based on the plate's actual dimensions. Combined with the coordinates of the center of gravity of the circular area in the calibration image, an affine transformation matrix is ​​established between each camera and the calibration plate, achieving a fully automated calibration process. Subsequently, using multiple cameras working together, the corner coordinates of the display panel under test are mapped to the calibration plate's coordinate system using the affine transformation matrix. The panel dimensions are then calculated by combining the mapping results from multiple cameras. This system can fully cover large panels and meet high-precision measurement requirements.

[0068] This method eliminates the need for manual input of calibration plate coordinates, making it easy to use and significantly improving calibration efficiency. It also avoids errors that can be introduced by manual input, ensuring the consistency and reliability of the calibration results. Furthermore, this method overcomes the limitations of traditional single-camera field of view and the tediousness of multi-camera calibration, enabling efficient and accurate inspection of large panels. It demonstrates strong versatility and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in this application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0070] Figure 1 A schematic flow chart of an embodiment of a large panel size measurement method using multi-camera joint automatic calibration provided in this application;

[0071] Figure 2 This is a schematic diagram of multi-camera calibration using four cameras for calibration in practical applications of this application as an example;

[0072] Figure 3 This is a calibration image obtained by taking four cameras for calibration in an actual application of this application as an example;

[0073] Figure 4 A schematic diagram showing the size measurement of the display panel in this application;

[0074] Figure 5 A schematic flow chart of another embodiment of the large panel size measurement method using multi-camera joint automatic calibration provided by this application;

[0075] Figure 6 Schematic diagram of the binarization process, small hole filling process and connected domain segmentation process in this application;

[0076] Figure 7This is a schematic diagram of the structure of an embodiment of a large panel size measurement system with multi-camera joint automatic calibration provided by this application;

[0077] Figure 8 This is a schematic structural diagram of an embodiment of a large panel size measurement device with multi-camera joint automatic calibration provided in this application. DETAILED DESCRIPTION

[0078] The present application provides a large panel size measurement method and system with multi-camera joint automatic calibration, which is used to achieve efficient and accurate large panel size detection.

[0079] It should be noted that the multi-camera joint automatic calibration large panel size measurement method provided in this application can be applied to terminals as well as servers. For example, the terminal can be a smartphone, computer, tablet computer, portable computer terminal, or a fixed terminal such as a desktop computer. For ease of explanation, this application uses a terminal as the execution subject for example.

[0080] See also Figure 1 , Figure 1 An embodiment of a large panel size measurement method using multi-camera joint automatic calibration provided in this application includes:

[0081] 101. Obtain a calibration image of a corresponding area of ​​a calibration plate by photographing with at least two cameras, wherein a plurality of circular areas are regularly distributed on the calibration plate, and each circular area contains corresponding row and column index character information;

[0082] The area of ​​a large-size panel usually exceeds the field of view of a single camera, and a single camera cannot capture all key points at once, such as the four corners of the panel. Therefore, in this embodiment, at least two cameras are used to shoot the calibration plate from different angles to obtain several calibration images. These calibration images need to cover all or key areas of the calibration plate to ensure that there are enough reference points for the subsequent calibration process. It should be noted that the calibration plate in this embodiment has several circular areas regularly distributed on the surface, usually arranged in a grid shape. Each circular area is black, and a white character is embedded in the center of the circular area to indicate the row and column index character information of the circular area on the calibration plate. The specific row and column index is in the form of "m,n", which indicates the position of the circular area in the calibration plate grid, such as "3,4" represents the 3rd row and 4th column.

[0083] When capturing calibration images, first secure the calibration plate on a stable platform in the inspection area, ensuring that it is within the field of view of all cameras. Secondly, control the lighting conditions, using a uniform diffuse light source (such as a ring LED light) to prevent shadows or highlights from interfering with the image quality of the calibration plate. Each camera captures the corresponding area of ​​the calibration plate synchronously or sequentially to obtain the calibration image. Figure 2 and Figure 3 , Figure 2 This is a schematic diagram of multi-camera calibration using four cameras for calibration in actual application. Figure 3 The actual size of the calibration plate is 36*50, the center distance is 1.5cm, and the center diameter is D=1cm.

[0084] 102. Extracting the target character in the calibration image and the center of gravity coordinates of the circular area in the calibration image;

[0085] The calibration image is the fundamental data source for establishing the transformation relationship between the camera and calibration plate coordinate systems. The circular areas distributed on the calibration plate and the row and column index character information embedded in them provide a precise geometric and positional reference. After capturing the calibration image, key features must be extracted from it. Specifically, the target characters in the calibration image and the centroid coordinates of the circular areas in the calibration image must be extracted to correlate the image coordinates with the physical coordinates of the calibration plate.

[0086] Specifically, the row and column index characters (such as "3,4") embedded in each circular area on the calibration plate indicate the position of the circular area in the calibration plate grid. Therefore, by performing character recognition on the calibration image to extract the target characters, the row and column identifiers corresponding to each circular area can be obtained, providing a basis for subsequent physical coordinate calculations. The circular areas on the calibration plate represent known physical locations. By calculating the centroid coordinates of the circular areas in the calibration image, the positions of these feature points in the camera image can be represented. The centroid coordinates are the center position of the circular area in the image coordinate system and represent the projection point of the feature points on the calibration plate.

[0087] 103. Calculate the physical coordinates of the calibration plate based on the actual size information of the target character and the calibration plate, and establish an affine transformation matrix between the camera and the calibration plate based on the center of gravity coordinates and the physical coordinates;

[0088] Since the target characters (such as row and column index "m,n") directly indicate the position of the circular area in the calibration grid, combined with the actual size information of the calibration plate, such as the center distance CenterDistance, the physical coordinates of these circular areas in the calibration coordinate system can be calculated. That is, the physical coordinates (Xi, Yi) are the actual positions of the circular areas on the calibration plate, with the upper left corner of the calibration plate (or other specified origin) as the reference, which can be calculated by the target characters and the center distance. For example, the circular area is 、 、 ,..., , the corresponding physical coordinates of the calibration plate are:

[0089] .

[0090] In this formula, is the vertical coordinate number corresponding to the circular area, that is, the column index character; is the horizontal coordinate number corresponding to the circular area, that is, the row index character.

[0091] The physical coordinates (Xi, Yi) represent the real-world location of the circular area on the calibration plate. The barycentric coordinates (xi, yi) identified in step 102 represent the pixel location in the image coordinate system, representing the projected center of the circular area in the camera image. By pairing the barycentric coordinates with the physical coordinates, an affine transformation matrix can be established between the camera and the calibration plate, achieving a precise mapping between the camera image coordinate system and the calibration plate's physical coordinate system, ensuring that the multi-camera system can work collaboratively within the calibration plate's coordinate system.

[0092] 104. Obtain a regional image of the display panel to be tested by photographing with a camera, and extract the coordinates of the corner points of the display panel to be tested according to the regional image;

[0093] In the previous steps, the calibration plate was used to establish the affine transformation matrix between each camera and the calibration plate coordinate system, completing the automatic calibration of the multi-camera system. The actual measurement process can now proceed. Large-size panels have widely distributed corner points, and a single camera has a limited field of view, making it impossible to capture all of them simultaneously. Therefore, a multi-camera system is required to capture each corner point by capturing the image in separate areas, ensuring that each corner point appears in the image of at least one camera.

[0094] In actual measurement, each camera captures a corresponding area of ​​the display panel to be tested, acquiring an area image encompassing the panel's edges and corners. Each camera's field of view covers a portion of the panel. The display panel's boundaries are identified from each camera's area image, its corners are located, and the corner coordinates (xc, yc) in the image coordinate system are generated. Each camera can extract one or more corner points, depending on the field of view coverage, which are ultimately integrated into a set of all corner points of the panel. For a rectangular panel, for example, four cameras capture the panel's top left, top right, bottom left, and bottom right areas, respectively, ensuring that the coordinates of all four corner points of the rectangular panel are recorded.

[0095] 105. Mapping the corner point coordinates to the coordinate system of the calibration plate through an affine transformation matrix to obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and calculating the size information of the display panel to be measured based on the target coordinates;

[0096] Because each camera in a multi-camera system captures a different area of ​​the panel, its corner coordinates are located in their own image coordinate systems, independent of each other and lacking physical meaning. However, using an affine transformation matrix, these corner points can be uniformly mapped to the calibration plate's coordinate system, thereby integrating the local data from multiple cameras into a unified physical coordinate framework. Specifically, using the affine transformation matrix established for each camera in step 103, the camera's corner coordinates (xc, yc) are transformed to the calibration plate's coordinate system to obtain the corresponding target coordinates (Xi, Yi). These target coordinates are the physical locations in the calibration plate's coordinate system. With the calibration plate's origin as a reference, they represent the real-world positions of the panel's corners, eliminating differences in view angles between cameras. By calculating the distances between these corner coordinates, the panel's dimensional information can be accurately derived. For example, the distances between adjacent corner points can be calculated to determine the panel's side length, or the diagonal distance can be used to verify geometric consistency. Ultimately, the dimensional information is output to meet high-precision inspection requirements.

[0097] See also Figure 4 , Figure 4 The size measurement diagram of the display panel is shown in Figure 2. Assume that the corresponding screen corners on the images of cameras C1, C2, C3, and C4 are 、 、 、 , corresponding to the affine transformation matrix 、 、 、 , we can get the actual coordinate position of the screen corner point on the calibration plate on the image as 、 、 、 . is the center point of the corresponding corner point on the edge, that is for and The center point of the edge corresponding to the two corner points, and so on. According to the coordinates of the center point of each edge combined with the Euclidean distance calculation formula, the actual size of the corresponding screen body can be obtained as follows:

[0098] .

[0099] In this embodiment, a calibration plate with row and column index characters is photographed by a camera, and the coordinate information on the calibration plate is automatically obtained by combining character recognition technology. The physical coordinates are calculated based on the actual size information of the calibration plate. Then, combined with the coordinates of the center of gravity of the circular area in the calibration image, an affine transformation matrix is ​​established between each camera and the calibration plate, realizing a fully automatic calibration process. Afterwards, using multiple cameras to work together, the corner coordinates of the display panel to be measured are mapped to the coordinate system of the calibration plate through the affine transformation matrix. The panel size can be calculated by combining the mapping results of multiple cameras. This method can fully cover large-size panels and meet the needs of high-precision measurement. This method does not require manual input of the calibration plate coordinate numbers, is easy to operate, greatly improves the calibration efficiency, and avoids the errors that may be introduced by manual input, ensuring the consistency and reliability of the calibration results. In addition, this method overcomes the defects of the limited field of view of traditional single-camera and the cumbersome calibration of multiple cameras, and can achieve efficient and accurate large-panel size detection, with strong versatility and practical value.

[0100] The following is a detailed description of the large panel size measurement method provided by this application using multiple cameras for automatic calibration. Figure 5 , Figure 5 Another embodiment of the large panel size measurement method provided by the present application using multiple cameras for joint automatic calibration includes:

[0101] 501. Obtain a calibration image of a corresponding area of ​​a calibration plate by photographing with at least two cameras, wherein a plurality of circular areas are regularly distributed on the calibration plate, and each circular area contains corresponding row and column index character information;

[0102] In this embodiment, step 501 is similar to step 101 in the aforementioned embodiment and will not be described again here.

[0103] 502. Binarize the calibration image, and perform hole filling and connected domain segmentation on the binarized calibration image.

[0104] Because the original calibration image often contains noise, uneven lighting, or background interference, direct processing may result in feature extraction failure. Therefore, the calibration image needs to be binarized, converting the pixel values ​​to 0 (black) or 255 (white) to highlight the black circular area on the calibration plate as white. Binarization is specifically based on a grayscale threshold, setting pixel values ​​below the threshold to 0 and those above or equal to the threshold to 255, resulting in a distinct black and white image.

[0105] Furthermore, in order to screen the circular area and repair isolated white pixels in the circular area caused by image noise, minor defects on the calibration plate surface, or uneven lighting, it is necessary to perform hole filling on the binary calibration image. That is, the binary calibration image is scanned, the black parts within the white area are identified, and then these holes are filled with white to ensure that the circular area is continuous and uninterrupted. Connected domain analysis is then performed on the filled binary image to identify all continuous white areas and mark them as independent connected domains, providing clear image data for subsequent circular area screening and character extraction. Figure 6 , Figure 6 Schematic diagram of binarization processing, small hole filling processing and connected domain segmentation processing.

[0106] 503. Calculate the circularity and area characteristics of each connected domain, and filter out valid circular areas in the calibration image based on a preset threshold;

[0107] Through binarization, hole filling and connected domain segmentation, a list of connected domains in the calibration image is generated, but these connected domains may include circular areas, noise areas or other unexpected areas (such as the edge of the calibration plate). To ensure that the subsequent steps only process the circular areas of the calibration plate, it is necessary to distinguish and filter out valid circular areas through geometric features. First, for each connected domain generated in step 502, its area and circularity are calculated as geometric feature quantities. The area is the total number of pixels in the connected domain, reflecting the size of the area in the image, and the circularity measures whether the area boundary is close to the ideal circle. Then, based on the preset thresholds of circularity and area, the connected domains that meet the circular area characteristics of the calibration plate are filtered out and marked as valid circular areas.

[0108] Considering that the pixel areas of the circular area of ​​the calibration plate in the image may be inconsistent due to differences in camera height, angle, or lens focal length in a multi-camera system, in order to adapt to the area changes under different camera shooting conditions, in some specific embodiments, a dynamic area threshold can be used to screen the valid circular areas:

[0109] The circularity and area characteristics of each connected domain are calculated, and the average area of ​​several connected domains with the top 10% area size is calculated; a dynamic area threshold is set based on the average area and a preset deviation parameter; and valid circular areas in the calibration image are screened out based on the dynamic area threshold.

[0110] Specifically, the area and circularity of each connected domain are calculated, and then the areas of all connected domains are sorted from large to small. The top 10% of connected domains are selected, and the average area of ​​these connected domains is calculated as a reference for the area size of the circular area. Based on this average area, the dynamic area threshold is calculated in combination with a preset deviation parameter (usually a coefficient less than 1, for example, set to 0.8). The dynamic area threshold is set to the product of the average area and the deviation parameter, and is usually lower than the average area to tolerate changes in the area of ​​the circular area. Finally, all connected domains are traversed, and connected domains with an area greater than or equal to the dynamic area threshold and a circularity that meets the preset conditions are retained and marked as valid circular areas. Since the dynamic area threshold is calculated based on the area distribution of the image itself, it can adapt to different camera shooting conditions, and there is no need to set a threshold separately for each camera.

[0111] 504. Extract the target character in the valid circular area and calculate the center of gravity coordinates of the valid circular area;

[0112] By screening for circularity and area, valid circular regions in the calibration image are identified. The white character content within each filtered valid circular region is then identified to obtain the target character. For each valid circular region, the centroid coordinates of its pixels must be calculated. This involves calculating the average coordinates of all pixels within the valid circular region to generate the coordinates (xi, yi) of the circular region in the corresponding image coordinate system. The centroid approximates the geometric center of the circular region. Because the circular regions of the calibration plate have a high degree of regularity after preprocessing, the centroid accuracy can reach sub-pixel levels.

[0113] Considering that when the camera captures the calibration plate, the circular area and characters in the calibration image may be rotated or perspectively distorted due to tilted angles or the plate not being placed completely horizontally, which will affect the accuracy of character recognition. In some specific embodiments, the calibration image is corrected through image rotation or perspective transformation so that the effective circular areas are located in the same horizontal direction; the target characters in the effective circular areas are extracted and segmented horizontally and vertically to obtain the corresponding individual characters; the individual characters are differentially compared with the standard characters to determine the character content of the individual characters.

[0114] Specifically, the geometric distribution of the calibration image containing the filtered valid circular area is analyzed to detect whether the valid circular area deviates from the horizontal direction. Then, image rotation or perspective transformation is applied to adjust the image so that the center (center of gravity) of the valid circular area is approximately on the same horizontal line, so that the character area presents a standard horizontal posture. In the corrected image, the white character area within each valid circular area is located. For example, the target character containing the row and column index "32,16" is extracted, and the character area is horizontally and vertically segmented in sequence, decomposing the target character into individual character units "3", "2", "1", and "6". The recognized individual characters are then differentially compared with standard characters. The standard characters are predefined character image templates that contain all possible characters that may appear on the calibration plate. The differential comparison specifically calculates the pixel difference between the individual character image and each standard character template, generates a similarity score, and selects the template with the highest similarity as the character content of the individual character.

[0115] Furthermore, in order to improve the calibration accuracy, all recognized character contents may be detected for common rows and columns, and abnormal character contents detected may be corrected.

[0116] Specifically, character recognition errors can cause the coordinates of the center of gravity to be paired with incorrect physical coordinates, generating an inaccurate affine transformation matrix. For the character content of all valid circular areas extracted, check whether the character content conforms to the row and column rules of the calibration plate grid. For example, the circular areas in the same row should have the same row number, and the column numbers of adjacent columns should increase continuously. Based on the results of the co-row and co-column detection, identify characters that do not conform to the grid rules, correct the erroneous characters, adjust them to the correct values ​​that conform to the grid rules, and output the corrected character data to ensure that the transformation matrix is ​​based on the correct coordinate pairing.

[0117] By detecting and correcting the common rows and columns of all recognized character contents, the regularity of the rows and columns of the calibration grid can be used to identify abnormal characters that do not conform to continuity, enhance the reliability of subsequent feature point pairing, and ensure the accuracy and robustness of the calibration process.

[0118] 505. Convert the target character into the physical coordinates of the circular area according to the distance between adjacent circle centers on the calibration plate;

[0119] Using the extracted target characters and the known center distance (CenterDistance) between adjacent circle centers on the calibration plate, the physical coordinates (Xi, Yi) of each circular area in the calibration plate coordinate system can be calculated. Since the distance between adjacent circle centers is a known parameter when the calibration plate is manufactured, it is usually a fixed value and has extremely high accuracy, which can directly determine the calculation accuracy of the physical coordinates. In addition, the distance is equal in the row and column directions of the calibration plate, which simplifies the coordinate calculation. For example, the circular area is 、 、 ,..., , the corresponding physical coordinates of the calibration plate are:

[0120] .

[0121] In this formula, is the vertical coordinate number corresponding to the circular area, that is, the column index character; is the horizontal coordinate number corresponding to the circular area, that is, the row index character.

[0122] 506. Pair the centroid coordinates of each circular area with the corresponding physical coordinates to form a point pair set;

[0123] Using the centroid coordinates (xi, yi) of each circular area extracted in step 504 and the corresponding physical coordinates (xi, yi) calculated in step 505, pair them to form point pairs (xi, yi, xi, yi). The pairing results for all valid circular areas are integrated to generate a point pair set. It should be noted that while the point pair set for each camera is generated independently, the physical coordinates (xi, yi) used are based on a unified calibration plate design, ensuring that the same character recognized by different cameras corresponds to the same physical coordinates. The point pair set provides independent calibration data for each camera, and ultimately the calibration results of multiple cameras are integrated using the coordinate system of the calibration plate.

[0124] 507. Calculate the affine transformation matrix between the camera and the calibration plate based on the point pair set, and iteratively optimize the affine transformation matrix using the RANSAC algorithm;

[0125] The above point pairs can be combined to calculate the affine transformation matrix between the camera and the calibration plate, that is, to obtain the coordinates of all the circle centers on the calibration image and the physical coordinates of the calibration plate at the corresponding circle centers for affine transformation to achieve the coordinates of the center of gravity of the circular area. 、 、 ,..., To the actual physical coordinates on the actual calibration plate 、 、 ,..., The affine transformation matrix corresponding to the camera image coordinate system to the real calibration plate is obtained. When calculating the affine transformation matrix, it is necessary to apply the RANSAC algorithm to iteratively analyze the point pair set, identify and eliminate abnormal point pairs, and then recalculate the affine transformation matrix based on the point pair set after eliminating abnormal point pairs to generate an optimized matrix. Taking the use of four cameras C1, C2, C3, and C4 for shooting as an example, let the corresponding affine transformation matrix be 、 、 、 A 2*3 matrix. In the field of view of C1, the corresponding affine transformation matrix is , so the Map to The image transformation formula corresponds to:

[0126] .

[0127] 508. Obtain a regional image of the display panel to be tested by photographing with a camera, and use a caliper tool to extract sub-pixel coordinate points of the screen edge to generate an edge point set;

[0128] In actual measurement, each camera is used to capture the corresponding area of ​​the display panel to be tested, and a regional image containing the panel edges and corners is obtained. The field of view of each camera covers a portion of the panel to ensure that the edge of the panel appears in the image of at least one camera. For each camera's regional image, the caliper tool is applied to analyze the grayscale changes at the edge of the panel and locate the sub-pixel coordinate points of the edge. All edge points are collected to form an edge point set, which represents the boundary features of the panel in the image coordinate system. The caliper tool is specifically an image processing algorithm that detects the edge position by scanning the grayscale changes in the image along a specified direction (for example, horizontally or vertically). Sub-pixel positioning is achieved through mathematical interpolation, which can capture slight changes in the edge. The position of the edge point in the image coordinate system is recorded as (xe,ye), which has better accuracy than integer pixels and can improve the precision of positioning.

[0129] 509. Fit the edge point set using the least squares method to obtain a fitting straight line corresponding to the edge of the screen;

[0130] 510. Determine the coordinates of the corner points of the display panel to be tested according to the intersection points of the fitted straight lines;

[0131] Because the extracted edge point set contains hundreds to thousands of sub-pixel coordinates, slight deviations may exist due to noise, edge discontinuities, or camera distortion. Therefore, the edge point set is grouped into subsets corresponding to different edges of the panel. Least squares analysis is then applied to each subset to fit a straight line representing the geometric characteristics of that edge. The intersection of these fitted lines can then be used to determine the coordinates of the corner points of the display panel under test, achieving precise positioning of the panel corners.

[0132] 511. Map the corner point coordinates to the coordinate system of the calibration plate through an affine transformation matrix to obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and calculate the size information of the display panel to be measured based on the target coordinates.

[0133] In this embodiment, step 511 is similar to step 105 in the aforementioned embodiment and will not be described again here.

[0134] In this embodiment, a fully automated calibration and measurement process, combined with sub-pixel edge detection, least squares fitting, and RANSAC optimization, effectively achieves high-precision detection of large panel sizes. Furthermore, the calibration process employs techniques such as dynamic area thresholding, image correction, and abnormal character correction to effectively address complex and diverse shooting conditions, ensuring robustness and reliability of calibration and detection. This method overcomes the limitations of traditional single-camera field of view and the cumbersome multi-camera calibration process, enabling efficient and accurate large-panel inspection and demonstrating strong versatility and practical value.

[0135] The following is a detailed description of the large panel size measurement system with multi-camera joint automatic calibration provided by this application. Figure 7 , Figure 7 An embodiment of a large panel size measurement system with multi-camera joint automatic calibration provided by the present application includes:

[0136] The first shooting unit 701 is used to obtain a calibration image of the corresponding area of ​​the calibration plate by shooting with at least two cameras. The calibration plate has a plurality of circular areas regularly distributed on it, and each circular area contains corresponding row and column index character information;

[0137] An extraction unit 702 is used to extract the target character in the calibration image and the centroid coordinates of the circular area in the calibration image;

[0138] The calibration unit 703 is used to calculate the physical coordinates of the calibration plate based on the actual size information of the target character and the calibration plate, and to establish an affine transformation matrix between the camera and the calibration plate according to the center of gravity coordinates and the physical coordinates;

[0139] The second shooting unit 704 is configured to obtain a regional image of the display panel to be tested by shooting with a camera, and extract the coordinates of the corner points of the display panel to be tested according to the regional image;

[0140] The detection unit 705 is used to map the corner point coordinates to the coordinate system of the calibration plate through the affine transformation matrix, obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and calculate the size information of the display panel to be measured based on the target coordinates.

[0141] Optionally, the extraction unit 702 is specifically configured to:

[0142] Binarize the calibration image, and perform hole filling and connected domain segmentation on the binarized calibration image;

[0143] Calculate the circularity and area characteristics of each connected domain, and filter out the valid circular areas in the calibration image based on the preset threshold;

[0144] The target character in the valid circular area is extracted, and the coordinates of the center of gravity of the valid circular area are calculated.

[0145] Optionally, the extraction unit 702 is further configured to:

[0146] Correct the calibration image by image rotation or perspective transformation so that the effective circular area is in the same horizontal direction;

[0147] Extract the target character in the valid circular area and perform horizontal and vertical segmentation on the target character to obtain the corresponding single character;

[0148] Perform differential comparison between a single character and a standard character to determine the character content of the single character.

[0149] Optionally, the extraction unit 702 is further configured to:

[0150] All recognized characters are checked for common rows and columns, and any abnormal characters detected are corrected.

[0151] Optionally, the extraction unit 702 is further configured to:

[0152] Calculate the circularity and area characteristics of each connected domain, and calculate the average area of ​​several connected domains with the top 10% area size;

[0153] Setting a dynamic area threshold based on the average area and a preset deviation parameter;

[0154] The valid circular area in the calibration image is screened out based on the dynamic area threshold.

[0155] Optionally, the calibration unit 703 is specifically configured to:

[0156] The target characters are converted into the physical coordinates of the circular area according to the distance between the adjacent circle centers on the calibration plate;

[0157] Pair the centroid coordinates of each circular area with the corresponding physical coordinates to form a point pair set;

[0158] The affine transformation matrix between the camera and the calibration plate is calculated based on the point pair set, and the affine transformation matrix is ​​iteratively optimized using the RANSAC algorithm.

[0159] Optionally, the second shooting unit 704 is specifically configured to:

[0160] The camera is used to obtain an image of the display panel to be tested, and a caliper tool is used to extract the sub-pixel coordinate points of the screen edge to generate an edge point set;

[0161] Fit the edge point set using the least squares method to obtain a fitting straight line corresponding to the edge of the screen;

[0162] The coordinates of the corner points of the display panel to be tested are determined according to the intersection points of the fitted straight lines.

[0163] In this embodiment, the functions of each unit are the same as those in the above Figure 1 or Figure 5 The steps in the method embodiment shown correspond to each other and will not be repeated here.

[0164] This application also provides a large panel size measurement device with multi-camera joint automatic calibration, see Figure 8 , Figure 8 An embodiment of a large panel size measurement device with multi-camera joint automatic calibration provided by the present application includes:

[0165] Processor 801, memory 802, input and output unit 803, bus 804;

[0166] The processor 801 is connected to the memory 802, the input and output unit 803 and the bus 804;

[0167] The memory 802 stores a program, and the processor 801 calls the program to execute any of the above methods for measuring the size of a large panel using multi-camera combined automatic calibration.

[0168] The present application also relates to a computer-readable storage medium having a program stored thereon. When the program is run on a computer, the computer is enabled to execute any of the above multi-camera joint automatic calibration methods for measuring the size of a large panel.

[0169] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0171] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0172] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0173] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A large panel size measurement method with multi-camera joint automatic calibration, characterized in that: The large panel size measurement method includes: A calibration image of a corresponding area of ​​a calibration plate is obtained by photographing with at least two cameras, wherein a plurality of circular areas are regularly distributed on the calibration plate, and each of the circular areas contains corresponding row and column index character information; Extracting the target character in the calibration image and the center of gravity coordinates of the circular area in the calibration image; Converting the target character into the physical coordinates of the circular area according to the distance between adjacent circle centers on the calibration plate; Pairing the center of gravity coordinates of each circular area with the corresponding physical coordinates to form a point pair set; Calculating an affine transformation matrix between the camera and the calibration plate based on the point pair set, and iteratively optimizing the affine transformation matrix using a RANSAC algorithm; Obtaining a regional image of the display panel to be tested by photographing with the camera, and extracting the coordinates of corner points of the display panel to be tested according to the regional image; The corner point coordinates are mapped to the coordinate system of the calibration plate through the affine transformation matrix to obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and the size information of the display panel to be measured is calculated based on the target coordinates.

2. The large panel size measurement method according to claim 1, characterized in that: The extracting the target character in the calibration image and the center of gravity coordinates of the circular area in the calibration image includes: Binarizing the calibration image, and performing hole filling and connected domain segmentation on the binarized calibration image; Calculating the circularity and area characteristics of each connected domain, and screening out valid circular areas in the calibration image based on a preset threshold; The target character in the effective circular area is extracted, and the coordinates of the center of gravity of the effective circular area are calculated.

3. The large panel size measurement method according to claim 2, characterized in that: Extracting the target character in the valid circular area includes: Correcting the calibration image by image rotation or perspective transformation so that the effective circular areas are located in the same horizontal direction; Extracting target characters in the effective circular area, and performing horizontal and vertical segmentation on the target characters to obtain corresponding single characters; The single character is differentially compared with a standard character to determine the character content of the single character.

4. The large panel size measurement method according to claim 3, characterized in that: After performing differential comparison between the single character and the standard character to determine the character content of the single character, the method further includes: All the identified character contents are detected for common rows and columns, and abnormal character contents detected are corrected.

5. The large panel size measurement method according to claim 2, characterized in that: The calculating of the circularity and area characteristics of each connected domain and screening out the valid circular area in the calibration image based on a preset threshold value includes: Calculate the circularity and area characteristics of each connected domain, and calculate the average area of ​​several connected domains with the top 10% area size; setting a dynamic area threshold based on the average area and a preset deviation parameter; A valid circular area in the calibration image is screened out based on the dynamic area threshold.

6. The large panel size measuring method according to any one of claims 1 to 5, characterized in that: The step of obtaining a regional image of the display panel to be tested by photographing the camera and extracting the coordinates of corner points of the display panel to be tested according to the regional image includes: The camera is used to obtain a regional image of the display panel to be tested, and a caliper tool is used to extract sub-pixel coordinate points of the edge of the screen to generate an edge point set; Fitting the edge point set by the least square method to obtain a fitting straight line corresponding to the edge of the screen; The coordinates of the corner points of the display panel to be tested are determined according to the intersection points of the fitted straight lines.

7. A large panel size measurement system with multi-camera joint automatic calibration, characterized in that: The large panel size measurement system includes: A first shooting unit is configured to obtain a calibration image of an area corresponding to a calibration plate by shooting with at least two cameras, wherein the calibration plate has a plurality of circular areas regularly distributed thereon, and each of the circular areas contains corresponding row and column index character information; an extraction unit, configured to extract the target character in the calibration image and the centroid coordinates of the circular area in the calibration image; a calibration unit, configured to convert the target character into the physical coordinates of the circular area according to the spacing between adjacent circle centers on the calibration plate; pair the centroid coordinates of each circular area with the corresponding physical coordinates to form a set of point pairs; calculate an affine transformation matrix between the camera and the calibration plate based on the set of point pairs, and iteratively optimize the affine transformation matrix using a RANSAC algorithm; a second shooting unit, configured to obtain a regional image of the display panel to be tested by shooting with the camera, and extract the coordinates of corner points of the display panel to be tested according to the regional image; The detection unit is used to map the corner point coordinates to the coordinate system of the calibration plate through the affine transformation matrix, obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and calculate the size information of the display panel to be measured based on the target coordinates.

8. A computer device, characterized in that: The computer device comprises: processor, memory, input and output units, and buses; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 6 is executed.

Images