Multi-camera joint automatic calibration large panel size measurement method and system

Abstract

The invention discloses a large panel size measurement method and system based on multi-camera joint automatic calibration. The large panel size measurement method and system are used for realizing efficient and accurate large panel size detection. The method comprises the following steps: shooting by at least two cameras to obtain a calibration image of a corresponding area of a calibration plate; extracting a target character in the calibration image and a barycentric coordinate of a circular area in the calibration image; calculating physical coordinates of the calibration plate based on the target character and the actual size information of the calibration plate, and establishing an affine transformation matrix between the camera and the calibration plate according to the barycentric coordinates and the physical coordinates; shooting through a camera to obtain a regional image of the display panel to be detected, and extracting angular point coordinates of the display panel to be detected according to the regional image; and mapping the angular point coordinates into a coordinate system of the calibration plate through an affine transformation matrix to obtain target coordinates of the angular points of the display panel in the coordinate system of the calibration plate, and calculating size information of the display panel to be measured according to the target coordinates.

Description

Technical Field

[0001] This application relates to the technical field of large panel size measurement with multi - camera joint automatic calibration, and particularly relates to a method and system for large panel size measurement with multi - camera joint automatic calibration. Background Art

[0002] With the rapid development of modern industrial manufacturing technology, large - size panels are increasingly widely used in fields such as displays, touchscreens, photovoltaic modules, and semiconductor substrates, and their market demand continues to grow. As the core component of these high - tech products, the size accuracy of large panels directly affects the performance, quality, and production efficiency of the products. For example, in the production of flat panel displays, a slight deviation in the panel size may lead to abnormal display effects or assembly difficulties.

[0003] In the field of large panel size detection, due to the high requirements for large panel size detection and the small error tolerance rate, when using a single camera for size detection, the accuracy often cannot meet the requirements. Therefore, in the prior art, a multi - camera joint calibration method is usually adopted for measurement to improve the measurement accuracy. However, most of the existing multi - camera calibration methods require manual input of the coordinate numbers on the calibration board, which is cumbersome and inefficient. At the same time, there are also differences in the results manually input by different personnel, and this human error directly affects the consistency and reliability of the calibration results, resulting in a decrease in the overall calibration accuracy and the stability of the measurement system. Summary of the Invention

[0004] This application provides a method and system for large panel size measurement with multi - camera joint automatic calibration, which is used to achieve efficient and accurate large panel size detection.

[0005] In the first aspect of this application, a method for large panel size measurement with multi - camera joint automatic calibration is provided, including: Taking calibration images of the corresponding area of the calibration board through at least two cameras, where a number of circular areas are regularly distributed on the calibration board, and each circular area contains corresponding row - column index character information; Extracting the target characters in the calibration image and the centroid coordinates of the circular areas in the calibration image; Calculating the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establishing an affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates; Taking an area image of the display panel to be measured through the camera, and extracting the corner coordinates of the display panel to be measured according to the area image; Map the corner coordinates to the coordinate system of the calibration board through the affine transformation matrix to obtain the target coordinates of the corners of the display panel in the coordinate system of the calibration board, and calculate the size information of the display panel to be measured according to the target coordinates.

[0006] Optionally, the extracting the target characters in the calibration image and the centroid coordinates of the circular region in the calibration image includes: Perform binarization processing on the calibration image, and perform small hole filling and connected component segmentation processing on the binarized calibration image; Calculate the roundness and area feature quantities of each connected component, and filter out the effective circular regions in the calibration image based on a preset threshold; Extract the target characters in the effective circular regions, and calculate the centroid coordinates of the effective circular regions.

[0007] Optionally, the extracting the target characters in the effective circular regions includes: Correct the calibration image through image rotation or perspective transformation so that the effective circular regions are located in the same horizontal direction; Extract the target characters in the effective circular regions, and perform horizontal and vertical segmentation on the target characters to obtain the corresponding single characters; Perform differential comparison between the single characters and the standard characters to determine the character content of the single characters.

[0008] Optionally, after the performing differential comparison between the single characters and the standard characters to determine the character content of the single characters, the method further includes: Perform co-row and co-column detection on all the recognized character contents, and correct the detected abnormal character contents.

[0009] Optionally, the calculating the roundness and area feature quantities of each connected component, and filtering out the effective circular regions in the calibration image based on a preset threshold includes: Calculate the roundness and area feature quantities of each connected component, and calculate the average area of a number of connected components with the top 10% in terms of area size; Set a dynamic area threshold based on the average area and a preset deviation parameter; Filter out the effective circular regions in the calibration image based on the dynamic area threshold.

[0010] Optionally, the calculating the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establishing the affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates includes: Convert the target character into the physical coordinates of the circular area according to the distance between adjacent circle centers on the calibration board; Pair the centroid coordinates of each circular area with the corresponding physical coordinates to form a set of point pairs; Calculate the affine transformation matrix between the camera and the calibration board based on the set of point pairs, and iteratively optimize the affine transformation matrix using the RANSAC algorithm.

[0011] Optionally, the step of obtaining the area image of the display panel to be measured by the camera and extracting the corner coordinates of the display panel to be measured from the area image includes: Obtain the area image of the display panel to be measured by the camera, and use a caliper tool to extract the sub-pixel coordinate points of the screen edge to generate an edge point set; Fit the edge point set by the least squares method to obtain a fitting line corresponding to the screen edge; Determine the corner coordinates of the display panel to be measured according to the intersection points of the fitting lines.

[0012] A large panel size measurement system for multi-camera joint automatic calibration provided in the second aspect of the present application includes: A first shooting unit for obtaining a calibration image of a corresponding area of the calibration board by at least two cameras, where a plurality of circular areas are regularly distributed on the calibration board, and each circular area contains corresponding row and column index character information; An extraction unit for extracting the target characters in the calibration image and the centroid coordinates of the circular areas in the calibration image; A calibration unit for calculating the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establishing an affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates; A second shooting unit for obtaining the area image of the display panel to be measured by the camera and extracting the corner coordinates of the display panel to be measured from the area image; A detection unit for mapping the corner coordinates to the coordinate system of the calibration board through the affine transformation matrix to obtain the target coordinates of the corners of the display panel in the coordinate system of the calibration board, and calculating the size information of the display panel to be measured according to the target coordinates.

[0013] Optionally, the extraction unit is specifically used for: Perform binarization processing on the calibration image, and perform small hole filling and connected component segmentation processing on the binarized calibration image; Calculate the roundness and area feature quantities of each connected component, and filter out the effective circular regions in the calibration image based on a preset threshold; Extract the target characters in the effective circular regions, and calculate the centroid coordinates of the effective circular regions.

[0014] Optionally, the extraction unit is further specifically configured to: Correct the calibration image through image rotation or perspective transformation so that the effective circular regions are in the same horizontal direction; Extract the target characters in the effective circular regions, and perform horizontal and vertical segmentation on the target characters to obtain corresponding single characters; Perform differential comparison between the single characters and standard characters to determine the character content of the single characters.

[0015] Optionally, the extraction unit is further specifically configured to: Perform co-row and co-column detection on all the recognized character contents, and correct the detected abnormal character contents.

[0016] Optionally, the extraction unit is further specifically configured to: Calculate the roundness and area feature quantities of each connected component, and calculate the average area of several connected components with the top 10% in terms of area size; Set a dynamic area threshold based on the average area and a preset deviation parameter; Filter out the effective circular regions in the calibration image based on the dynamic area threshold.

[0017] Optionally, the calibration unit is specifically configured to: Convert the target characters into physical coordinates of the circular regions according to the distance between adjacent circle centers on the calibration plate; Pair the centroid coordinates of each circular region with the corresponding physical coordinates to form a set of point pairs; Calculate the affine transformation matrix between the camera and the calibration plate based on the set of point pairs, and iteratively optimize the affine transformation matrix using the RANSAC algorithm.

[0018] Optionally, the second shooting unit is specifically configured to: Obtain a regional image of the display panel to be measured through the camera, and use a caliper tool to extract sub-pixel coordinate points on the screen edge to generate an edge point set; Perform fitting on the edge point set by the least squares method to obtain a fitting line corresponding to the screen edge; Determine the corner coordinates of the display panel to be measured according to the intersection points of the fitting lines.

[0019] In a third aspect of the present application, a large panel size measurement device for multi-camera joint automatic calibration is provided. The device includes: a processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the large panel size measurement method for multi-camera joint automatic calibration in the first aspect and any optional one of the first aspect.

[0020] In a fourth aspect of the present application, a computer-readable storage medium is provided. A program is stored on the computer-readable storage medium, and when the program is executed on a computer, it executes the large panel size measurement method for multi-camera joint automatic calibration in the first aspect and any optional one of the first aspect.

[0021] As can be seen from the above technical solutions, the present application has the following advantages: By using a camera to photograph a calibration board with row and column index characters, coordinate information on the calibration board is automatically obtained in combination with character recognition technology, and physical coordinates are calculated based on the actual size information of the calibration board. Then, in combination with the centroid coordinates of the circular area in the calibration image, an affine transformation matrix between each camera and the calibration board is established, realizing a fully automatic calibration process. After that, by using multiple cameras to work together, the corner coordinates of the display panel to be measured are mapped to the coordinate system of the calibration board through the affine transformation matrix, and the panel size can be obtained by comprehensively calculating the mapping results of multiple cameras, which can fully cover large-sized panels and meet the requirements of high-precision measurement.

[0022] This method does not require manual input of calibration board coordinate numbers, is easy to operate, greatly improves the calibration efficiency, and at the same time avoids errors that may be introduced by manual input, ensuring the consistency and reliability of the calibration results. And this method overcomes the defects of limited field of view of traditional single cameras and cumbersome calibration of multiple cameras, and can realize efficient and accurate detection of large panel sizes, with strong versatility and practical value. Description of the Drawings

[0023] In order to more clearly illustrate the technical solutions in the present application, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 It is a schematic flowchart of an embodiment of the large panel size measurement method for multi-camera joint automatic calibration provided by the present application; Figure 2Schematic diagram of multi-camera calibration with four cameras used for calibration in the actual application of this application; Figure 3 Calibration images obtained by taking pictures with four cameras used for calibration in the actual application of this application; Figure 4 Schematic diagram of the size measurement of the display panel in this application; Figure 5 Schematic flow chart of another embodiment of the large panel size measurement method for multi-camera joint automatic calibration provided by this application; Figure 6 Schematic diagrams of binarization processing, small hole filling processing, and connected component segmentation processing in this application; Figure 7 Schematic structural diagram of an embodiment of the large panel size measurement system for multi-camera joint automatic calibration provided by this application; Figure 8 Schematic structural diagram of an embodiment of the large panel size measurement device for multi-camera joint automatic calibration provided by this application. Detailed implementation manners

[0025] This application provides a large panel size measurement method and system for multi-camera joint automatic calibration, which is used to achieve efficient and accurate large panel size detection.

[0026] It should be noted that the large panel size measurement method for multi-camera joint automatic calibration provided by this application can be applied to a terminal or a server. For example, the terminal can be a smart phone, a computer, a tablet computer, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, this application takes the terminal as the execution subject for example.

[0027] Please refer to Figure 1 , Figure 1 An embodiment of the large panel size measurement method for multi-camera joint automatic calibration provided by this application, the method includes: 101. Obtain calibration images of corresponding areas of the calibration board by shooting with at least two cameras. There are several circular areas regularly distributed on the calibration board, and each circular area contains corresponding row and column index character information; The area of a large - size panel usually exceeds the field - of - view range of a single camera. A single camera cannot capture all key points at once, such as the four corner points of the panel. Therefore, in this embodiment, at least two cameras are used to photograph the calibration board from different angles to obtain several calibration images. These calibration images need to cover all or key areas of the calibration board to ensure that there are enough reference points for the subsequent calibration process. It should be noted that on the surface of the calibration board in this embodiment, there are several circular areas regularly distributed, usually arranged in a grid pattern. Each circular area is black, and a white character is embedded in the center of the circular area to represent the row - column index character information of the circular area on the calibration board. The specific row - column index form is "m,n", indicating the position of the circular area in the calibration board grid. For example, "3,4" represents the 3rd row and the 4th column.

[0028] When photographing the calibration images, first, fix the calibration board on a stable platform in the detection area to ensure its position is within the field - of - view of all cameras. Secondly, it is necessary to control the lighting conditions, using a uniform diffused light source (such as a ring - shaped LED lamp) to avoid shadows or highlights interfering with the quality of the calibration board images. Each camera synchronously or sequentially photographs the corresponding area of the calibration board to obtain calibration images. Please refer to Figure 2 and Figure 3 , Figure 2 Figure 9 shows a schematic diagram of multi - camera calibration using four cameras for calibration in practical applications, Figure 3 and Figure 10 shows the corresponding calibration images obtained by photographing. The actual size data of this calibration board is 36 * 50, the center - to - center distance CenterDistance = 1.5 cm, and the center diameter D = 1 cm.

[0029] 102. Extract the target characters in the calibration images and the centroid coordinates of the circular areas in the calibration images; The calibration images are the basic data source for establishing the transformation relationship between the camera and the calibration board coordinate systems. The circular areas distributed on the calibration board and the row - column index character information embedded in them can provide accurate geometric and position references. After obtaining the calibration images by photographing, key features need to be extracted from them, that is, to extract the target characters in the calibration images and the centroid coordinates of the circular areas in the calibration images, so as to associate the image coordinates with the physical coordinates of the calibration board.

[0030] Specifically, the row - column index characters (such as "3,4") embedded in each circular area on the calibration board represent the position of the circular area in the calibration board grid. Therefore, by performing character recognition on the calibration images to extract the target characters, the row - column identifiers corresponding to each circular area can be obtained, providing a basis for subsequent physical coordinate calculation. And the circular areas on the calibration board represent known physical positions. By calculating the centroid coordinates of the circular areas in the calibration images, the positions of these feature points in the camera images can be represented. This centroid coordinate is the central position of the circular area in the image coordinate system, representing the projection points of the feature points on the calibration board.

[0031] 103. Calculate the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establish an affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates; Since the target characters (such as the row-column indices "m,n") directly indicate the positions of the circular regions in the calibration board grid, combined with the actual size information of the calibration board, such as the center distance CenterDistance, the physical coordinates of these circular regions in the calibration board coordinate system can be calculated. That is, the physical coordinates (Xi, Yi) are the real positions of the circular regions on the calibration board, and can be calculated through the target characters and the center distance with the upper left corner (or other specified origin) of the calibration board as the reference. For example, for circular regions such as , , ,..., , the corresponding physical coordinates of the calibration board are: .

[0032] In this formula, is the ordinate number of the corresponding circular region, that is, the column index character; is the abscissa number of the corresponding circular region, that is, the row index character.

[0033] The physical coordinates (Xi, Yi) are the physical positions of the circular regions on the calibration board in the real world, and the centroid coordinates (xi, yi) identified in step 102 are the pixel positions in the image coordinate system, representing the projection centers of the circular regions in the camera image. By using the pairing of the centroid coordinates and the physical coordinates, an affine transformation matrix between the camera and the calibration board can be established to achieve an accurate mapping between the camera image coordinate system and the calibration board physical coordinate system, ensuring that the multi-camera system can work collaboratively in the calibration board coordinate system.

[0034] 104. Obtain the regional image of the display panel to be measured by camera shooting, and extract the corner coordinates of the display panel to be measured according to the regional image; In the foregoing steps, an affine transformation matrix between each camera and the calibration board coordinate system has been established through the calibration board, that is, the automatic calibration of the multi-camera system has been completed, and the actual measurement process can continue. The corner points of the large-size panel are widely distributed, and the field of view of a single camera is limited and cannot capture all the corner points at the same time. Therefore, a multi-camera system needs to be used to shoot by regions to ensure that each corner point appears in the image of at least one camera.

[0035] In actual measurement, each camera is used to capture the corresponding area of the display panel to be measured, and an area image containing the panel edges and corner points is obtained. The field of view of each camera covers a part of the panel. The boundaries of the display panel are identified from the area images of each camera, the corner point positions are located, and the corner point coordinates (xc, yc) in the image coordinate system are generated. Each camera can extract one or more corner points, specifically determined by the field of view coverage, and finally integrated into the set of all corner points of the panel. Taking a rectangular panel as an example, four cameras respectively capture the upper left, upper right, lower left, and lower right areas of the panel to ensure that the four corner point coordinates of the rectangular panel are recorded.

[0036] 105. Map the corner point coordinates to the coordinate system of the calibration board through the affine transformation matrix to obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration board, and calculate the size information of the display panel to be measured according to the target coordinates; Since each camera in the multi-camera system captures different areas of the panel, its corner point coordinates are located in their respective image coordinate systems, independent of each other and without physical meaning. However, through the affine transformation matrix, these corner points can be uniformly mapped to the coordinate system of the calibration board, thus integrating the local data of multiple cameras into a unified physical coordinate framework. Specifically, using the affine transformation matrix established for each camera in step 103, the corner point coordinates (xc, yc) of this camera are transformed to the coordinate system of the calibration board to obtain the corresponding target coordinates (Xi, Yi). This target coordinate is the physical position in the coordinate system of the calibration board, representing the real-world position of the panel corner point with reference to the origin of the calibration board, eliminating the perspective differences between cameras. At this time, by calculating the distances between these corner points, the size information of the panel can be accurately obtained. For example, calculating the distance between adjacent corner points gives the side length of the panel, or the diagonal distance is used to verify geometric consistency, and finally the size information is output to meet the high-precision detection requirements.

[0037] Please refer to Figure 4 , Figure 4 for the schematic diagram of the size measurement of the display panel. Let the corresponding screen corner points on the images of cameras C1, C2, C3, and C4 be 、 、 、 , and the corresponding affine transformation matrices are 、 、 、 . The actual coordinate positions of the screen corner points on the calibration board can be obtained as 、 、 、 . Then it is the center point of the corner points on the corresponding edge, that is, is and The center point of the edge corresponding to the two corner points, and so on. According to the coordinates of the center points of each edge combined with the Euclidean distance calculation formula, the actual size of the corresponding screen body can be obtained as follows: .

[0038] In this embodiment, the calibration plate with row and column index characters is photographed by the camera, and the coordinate information on the calibration plate is automatically obtained in combination with the character recognition technology, and the physical coordinates are calculated based on the actual size information of the calibration plate, and then the centroid coordinates of the circular area in the calibration image are combined to establish the affine transformation matrix between each camera and the calibration plate, thereby realizing a fully automatic calibration process. After that, the corner coordinates of the display panel to be tested are mapped to the coordinate system of the calibration plate through the affine transformation matrix by using multiple cameras to work together, and the panel size can be obtained by calculating the mapping results of multiple cameras, which can fully cover large-size panels and meet the needs of high-precision measurement. This method does not require manual input of the calibration plate coordinate numbers, is easy to operate, greatly improves the calibration efficiency, and avoids the errors that may be introduced by manual input, ensuring the consistency and reliability of the calibration results. In addition, this method overcomes the defects of the limited field of view of the traditional single camera and the cumbersome calibration of multiple cameras, and can achieve efficient and accurate large-panel size detection, with strong versatility and practical value.

[0039] The following is a detailed description of the large panel size measurement method provided by this application using multi-camera joint automatic calibration. Figure 5 , Figure 5 Another embodiment of the large panel size measurement method provided by the present application using multiple cameras for joint automatic calibration includes: 501. Obtain a calibration image of a corresponding area of a calibration plate by photographing with at least two cameras, wherein a plurality of circular areas are regularly distributed on the calibration plate, and each circular area contains corresponding row and column index character information; In this embodiment, step 501 is similar to step 101 in the aforementioned embodiment and will not be described in detail here.

[0040] 502. Binarize the calibration image, and perform hole filling and connected domain segmentation on the calibration image after the binarization process; Since the original calibration image usually contains noise, uneven lighting or background interference, direct processing may lead to feature extraction failure. Therefore, it is necessary to perform binarization on the calibration image and convert the pixel values to 0 (black) or 255 (white) to highlight the black circular area on the calibration plate as white. The binarization process is based on the grayscale threshold, which sets the pixel value below the threshold to 0 and the pixel value above or equal to the threshold to 255, forming a black and white image.

[0041] Further, in order to screen the circular region and repair the isolated white pixel points in the circular region caused by image noise, minute defects on the calibration plate surface, or uneven illumination, it is necessary to perform hole filling on the binary-calibrated image, that is, scan the binary-calibrated image, identify the black parts within the white region, and then fill these holes with white to ensure the continuity and non-interruption within the circular region. Then perform connected component analysis on the filled binary image to identify all continuous white regions and label them as independent connected components, providing clear image data for subsequent circular region screening and character extraction. Please refer to Figure 6 , Figure 6 which is a schematic diagram of binary processing, hole filling processing, and connected component segmentation processing.

[0042] 503. Calculate the roundness and area feature quantities of each connected component, and screen out the effective circular regions in the calibrated image based on a preset threshold; Through binaryization, hole filling, and connected component segmentation, a list of connected components in the calibrated image is generated. However, these connected components may include circular regions, noise regions, or other unexpected regions (such as the edges of the calibration plate). To ensure that subsequent steps only process the circular regions of the calibration plate, it is necessary to screen out the effective circular regions by distinguishing geometric features. First, calculate the area and roundness of each connected component generated in step 502 as geometric feature quantities. Among them, the area is the total number of pixels within the connected component, reflecting the size of the region in the image, and the roundness measures whether the region boundary is close to an ideal circle. Then, according to the preset thresholds of roundness and area, screen out the connected components that conform to the characteristics of the circular regions of the calibration plate and label them as effective circular regions.

[0043] Considering that in a multi-camera system, due to differences in camera height, angle, or lens focal length, the pixel areas of the circular regions of the calibration plate in the image are inconsistent. To adapt to the area changes under different camera shooting conditions, in some specific embodiments, a dynamic area threshold method can be used to screen the effective circular regions: Calculate the roundness and area feature quantities of each connected component, and calculate the average area of several connected components with the top 10% of the area sizes; set a dynamic area threshold based on the average area and a preset deviation parameter; screen out the effective circular regions in the calibrated image based on the dynamic area threshold.

[0044] Specifically, calculate the area and roundness of each connected component, then sort the areas of all connected components from largest to smallest, select the connected components with the top 10% in terms of area ranking, calculate the average area of these connected components, and use it as a reference for the area size of the circular region. Based on this average area, combined with a preset deviation parameter (usually a coefficient less than 1, such as set to 0.8), calculate the dynamic area threshold. The dynamic area threshold is set as the product of the average area and the deviation parameter, usually lower than the average area, to tolerate the area change of the circular region. Finally, traverse all connected components, retain the connected components with an area greater than or equal to the dynamic area threshold and a roundness that meets the preset conditions, and mark them as valid circular regions. Since the dynamic area threshold is calculated based on the area distribution of the image itself, it can adapt to different camera shooting conditions without the need to set a threshold separately for each camera.

[0045] 504. Extract the target characters in the valid circular region and calculate the centroid coordinates of the valid circular region; Through the screening of roundness and area, the valid circular regions in the calibration image are determined. At this time, for each screened valid circular region, identify the white character content inside it to obtain the target characters. For each valid circular region, it is also necessary to calculate the centroid coordinates of its pixels, that is, calculate the average value of the coordinates of all pixels within the valid circular region to generate the coordinates (xi, yi) of this circular region in the corresponding image coordinate system. The centroid approximates the geometric center of the circular region. Because the circular region of the calibration board has high regularity after preprocessing, the centroid accuracy can reach the sub-pixel level.

[0046] Considering that when the camera shoots the calibration board, due to the angle tilt or the calibration board not being placed completely horizontally, the circular region and characters in the calibration image may be rotated or perspectively distorted, which will affect the accuracy of character recognition. In some specific embodiments, the calibration image is corrected through image rotation or perspective transformation so that the valid circular regions are located in the same horizontal direction; extract the target characters in the valid circular regions, and perform horizontal and vertical segmentation on the target characters to obtain the corresponding single characters; compare the single characters with the standard characters by difference to determine the character content of the single characters.

[0047] Specifically, for the calibrated image where the effective circular regions are screened out, analyze their geometric distribution, detect whether the effective circular regions deviate from the horizontal direction, and then apply image rotation or perspective transformation to adjust the image so that the centers (centroids) of the effective circular regions are approximately on the same horizontal line, making the character regions present a standard horizontal posture. In the corrected image, locate the white character regions within each effective circular region. For example, if the target character containing the row and column indices "32, 16" is extracted, perform horizontal and vertical segmentation on the character region in sequence to decompose the target character into individual character units "3", "2", "1", "6". Then, perform differential comparison between the recognized individual characters and the standard characters. The standard characters are predefined character image templates that contain all the characters that may appear on the calibration board. The differential comparison specifically calculates the pixel differences between the individual character image and each standard character template to generate similarity scores, and selects the template with the highest similarity as the character content of the individual character.

[0048] Furthermore, to improve the calibration accuracy, co-row and co-column detection can also be performed on all the recognized character contents, and the detected abnormal character contents can be corrected.

[0049] Specifically, since character recognition errors can lead to the pairing of centroid coordinates with incorrect physical coordinates, generating an inaccurate affine transformation matrix. For the character contents of all the extracted effective circular regions, detect whether the character contents conform to the row and column rules of the calibration board grid. For example, the circular regions in the same row should have the same row number, and the column numbers of adjacent columns should increase continuously. Based on the results of the co-row and co-column detection, identify the characters that do not conform to the grid rules, correct the incorrect characters, adjust them to the correct values that conform to the grid rules, and output the corrected character data to ensure that the transformation matrix is based on the correct coordinate pairing.

[0050] By performing co-row and co-column detection and correction on all the recognized character contents, the row and column regularity of the calibration board grid can be utilized to identify abnormal characters that do not conform to continuity, enhance the reliability of subsequent feature point pairing, and ensure the accuracy and robustness of the calibration process.

[0051] 505. Convert the target characters into the physical coordinates of the circular regions according to the distance between adjacent circle centers on the calibration board; Using the extracted target characters, combined with the known center distance CenterDistance between adjacent circle centers on the calibration board, the physical coordinates (Xi, Yi) of each circular region in the calibration board coordinate system can be calculated. Since the distance between adjacent circle centers is a known parameter during the manufacturing of the calibration board, usually a fixed value and with extremely high precision, it can directly determine the calculation accuracy of the physical coordinates. And the distance is equal in the row and column directions of the calibration board, simplifying the coordinate calculation. For example, the circular region is 、 、 ,..., , the corresponding physical coordinates of the calibration plate are: .

[0052] In this formula, is the ordinate number corresponding to the circular area, i.e. the column index character; It is the horizontal coordinate number corresponding to the circular area, that is, the row index character.

[0053] 506. Pair the centroid coordinates of each circular area with the corresponding physical coordinates to form a point pair set; Using the centroid coordinates (xi, yi) of each circular area extracted in step 504 and the corresponding physical coordinates (Xi, Yi) calculated in step 505, pair the two to form a point pair (xi, yi, Xi, Yi). Integrate the pairing results of all valid circular areas to generate a point pair set. It should be noted that the point pair set of each camera is generated independently, but the physical coordinates (Xi, Yi) used are based on the unified calibration plate design to ensure that the same character recognized by different cameras corresponds to the same physical coordinates. The point pair set can provide independent calibration data for each camera, and finally integrate the calibration results of multiple cameras through the coordinate system of the calibration plate.

[0054] 507. Calculate the affine transformation matrix between the camera and the calibration plate based on the point pair set, and iteratively optimize the affine transformation matrix using the RANSAC algorithm; The above point pairs can be combined to calculate the affine transformation matrix between the camera and the calibration plate, that is, to obtain the coordinates of all the circle centers on the calibration image and the physical coordinates of the calibration plate at the corresponding circle center for affine transformation to achieve the coordinates of the center of the circular area. , , ,..., To the actual physical coordinates on the actual calibration plate , , ,..., The affine transformation matrix corresponding to the camera image coordinate system to the real calibration plate is obtained by transforming. When calculating the affine transformation matrix, it is necessary to apply the RANSAC algorithm to iteratively analyze the point pair set, identify and remove abnormal point pairs, and then recalculate the affine transformation matrix based on the point pair set after removing the abnormal point pairs to generate an optimized matrix. Taking the use of four cameras C1, C2, C3, and C4 for shooting as an example, the corresponding affine transformation matrix is , , , A 2*3 matrix. In the field of view of C1, the corresponding affine transformation matrix is , so Map to The image transformation formula corresponds to: .

[0055] 508. Obtain a regional image of the display panel to be tested by photographing with a camera, and use a caliper tool to extract sub-pixel coordinate points of the edge of the screen to generate an edge point set; In actual measurement, each camera is used to capture the corresponding area of the display panel to be tested, and a regional image containing the panel edge and corner points is obtained. The field of view of each camera covers a portion of the panel to ensure that the edge of the panel appears in the image of at least one camera. For the regional image of each camera, the caliper tool is applied to analyze the grayscale changes at the edge of the panel and locate the sub-pixel coordinate points of the edge. All edge points are collected to form an edge point set, which represents the boundary features of the panel in the image coordinate system. The caliper tool is specifically an image processing algorithm that detects the edge position by scanning the grayscale changes in the image along a specified direction (for example, horizontally or vertically). Sub-pixel positioning is achieved through mathematical interpolation, which can capture slight changes in the edge. The position of the edge point in the image coordinate system is recorded as (xe,ye), which has better accuracy than integer pixels and can improve the precision of positioning.

[0056] 509. Fit the edge point set by the least square method to obtain a fitting straight line corresponding to the edge of the screen; 510. Determine the coordinates of the corner points of the display panel to be tested according to the intersection points of the fitted straight lines; Since the extracted edge point set contains hundreds to thousands of sub-pixel coordinate points, there may be slight deviations due to noise, edge discontinuity or camera distortion. Therefore, the edge point set can be grouped into subsets corresponding to different edges of the panel. The least squares method is applied to each subset to fit a straight line to represent the geometric features of the edge. The coordinates of the corner points of the display panel to be tested can be determined based on the intersection of the fitted straight line, thus achieving accurate positioning of the panel corner points.

[0057] 511. Map the corner point coordinates to the coordinate system of the calibration plate through an affine transformation matrix to obtain the target coordinates of the corner points of the display panel in the coordinate system of the calibration plate, and calculate the size information of the display panel to be measured according to the target coordinates.

[0058] In this embodiment, step 511 is similar to step 105 in the aforementioned embodiment and will not be described again here.

[0059] In this embodiment, through a fully automated calibration and measurement process, combined with sub-pixel edge detection, least squares fitting, and RANSAC optimization, high-precision detection of large panel sizes can be effectively achieved. At the same time, techniques such as dynamic area thresholding, image correction, and abnormal character correction are adopted during the calibration process to effectively cope with complex and diverse shooting conditions, ensuring the robustness and reliability of calibration and detection. This method overcomes the defects of limited field of view of traditional single cameras and cumbersome calibration of multiple cameras, and can achieve efficient and accurate detection of large panel sizes, with strong versatility and practical value.

[0060] The following provides a detailed description of the large panel size measurement system with joint automatic calibration of multiple cameras according to the present application. Please refer to Figure 7 , Figure 7 which is an embodiment of the large panel size measurement system with joint automatic calibration of multiple cameras provided by the present application. The system includes: A first shooting unit 701, configured to obtain calibration images of corresponding areas of a calibration board by shooting with at least two cameras. A number of circular areas are regularly distributed on the calibration board, and each circular area contains corresponding row and column index character information; An extraction unit 702, configured to extract target characters in the calibration image and the centroid coordinates of the circular areas in the calibration image; A calibration unit 703, configured to calculate the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establish an affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates; A second shooting unit 704, configured to obtain an area image of a display panel to be measured by shooting with a camera, and extract the corner coordinates of the display panel to be measured according to the area image; A detection unit 705, configured to map the corner coordinates to the coordinate system of the calibration board through the affine transformation matrix to obtain the target coordinates of the corners of the display panel in the coordinate system of the calibration board, and calculate the size information of the display panel to be measured according to the target coordinates.

[0061] Optionally, the extraction unit 702 is specifically configured to: Perform binarization processing on the calibration image, and perform small hole filling and connected component segmentation processing on the binarized calibration image; Calculate the roundness and area feature quantities of each connected component, and filter out valid circular areas in the calibration image based on a preset threshold; Extract target characters in the valid circular areas, and calculate the centroid coordinates of the valid circular areas.

[0062] Optionally, the extraction unit 702 is further specifically configured to: Correct the calibration image through image rotation or perspective transformation so that the valid circular areas are located in the same horizontal direction; Extract the target characters in the effective circular region, and perform horizontal and vertical splitting on the target characters to obtain the corresponding single characters; Perform differential comparison between the single characters and the standard characters to determine the character content of the single characters.

[0063] Optionally, the extraction unit 702 is further specifically configured to: Perform co-line and co-column detection on all recognized character contents, and correct the detected abnormal character contents.

[0064] Optionally, the extraction unit 702 is further specifically configured to: Calculate the roundness and area feature quantities of each connected component, and calculate the average area of a number of connected components with the top 10% in terms of area size; Set a dynamic area threshold based on the average area and a preset deviation parameter; Filter out the effective circular regions in the calibration image based on the dynamic area threshold.

[0065] Optionally, the calibration unit 703 is specifically configured to: Convert the target characters into physical coordinates of circular regions according to the spacing between adjacent center points on the calibration plate; Pair the centroid coordinates of each circular region with the corresponding physical coordinates to form a set of point pairs; Calculate the affine transformation matrix between the camera and the calibration plate based on the set of point pairs, and iteratively optimize the affine transformation matrix using the RANSAC algorithm.

[0066] Optionally, the second photographing unit 704 is specifically configured to: Obtain a regional image of the display panel to be measured by camera shooting, and use a caliper tool to extract the sub-pixel coordinate points at the screen edge to generate an edge point set; Fit the edge point set by the least squares method to obtain a fitting line corresponding to the screen edge; Determine the corner coordinates of the display panel to be measured according to the intersection points of the fitting lines.

[0067] In the system of this embodiment, the functions of each unit correspond to the steps in the foregoing Figure 1 or Figure 5 shown method embodiment, and will not be elaborated here.

[0068] This application further provides a large panel size measurement device for multi-camera joint automatic calibration. Please refer to Figure 8 , Figure 8 which is an embodiment of the large panel size measurement device for multi-camera joint automatic calibration provided by this application. The device includes: A processor 801, a memory 802, an input / output unit 803, and a bus 804; The processor 801 is connected to the memory 802, the input / output unit 803, and the bus 804; The memory 802 stores a program, and the processor 801 invokes the program to execute any of the above-described large panel size measurement methods for multi-camera joint automatic calibration.

[0069] This application also relates to a computer-readable storage medium that stores a program. When the program runs on a computer, the computer is caused to execute any of the above-described large panel size measurement methods for multi-camera joint automatic calibration.

[0070] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, indirect couplings or communication connections of devices or units, and can be in electrical, mechanical, or other forms.

[0072] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0073] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0074] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, read-only memory), random access memories (RAM, random access memory), magnetic disks, or optical discs.

Claims

1. A large panel size measurement method for multi-camera joint automatic calibration, characterized in that The large panel size measurement method includes: Obtaining calibration images of corresponding areas of a calibration board by shooting with at least two cameras. A number of circular areas are regularly distributed on the calibration board, and each circular area contains corresponding row and column index character information; Extracting the target characters in the calibration images and the centroid coordinates of the circular areas in the calibration images; Calculating the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establishing an affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates; Obtaining an area image of the display panel to be measured by shooting with the camera, and extracting the corner coordinates of the display panel to be measured from the area image; Mapping the corner coordinates to the coordinate system of the calibration board through the affine transformation matrix to obtain the target coordinates of the corners of the display panel in the coordinate system of the calibration board, and calculating the size information of the display panel to be measured according to the target coordinates.

2. The large panel size measurement method according to claim 1, characterized in that, The extracting the target characters in the calibration images and the centroid coordinates of the circular areas in the calibration images includes: Performing binarization processing on the calibration images, and performing small hole filling and connected component segmentation processing on the binarized calibration images; Calculating the roundness and area feature quantities of each connected component, and screening out the effective circular areas in the calibration images based on a preset threshold; Extracting the target characters in the effective circular areas, and calculating the centroid coordinates of the effective circular areas.

3. The large panel size measurement method according to claim 2, characterized in that The extracting the target characters in the effective circular areas includes: Correcting the calibration images through image rotation or perspective transformation so that the effective circular areas are located in the same horizontal direction; Extracting the target characters in the effective circular areas, and performing horizontal segmentation and vertical segmentation on the target characters to obtain corresponding single characters; Performing differential comparison between the single characters and standard characters to determine the character content of the single characters.

4. The large panel size measurement method according to claim 3, wherein After performing the differential comparison between the single characters and standard characters to determine the character content of the single characters, the method further includes: Performing co-row and co-column detection on all the recognized character contents, and correcting the detected abnormal character contents.

5. The large panel size measurement method according to claim 2, wherein The calculating the roundness and area feature quantities of each connected component, and screening out the effective circular areas in the calibration images based on a preset threshold includes: Calculating the roundness and area feature quantities of each connected component, and calculating the average area of a number of connected components with the top 10% in terms of area size; Setting a dynamic area threshold based on the average area and a preset deviation parameter; Screening out the effective circular areas in the calibration images based on the dynamic area threshold.

6. The large panel size measurement method according to claim 1, characterized in that The calculating the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establishing an affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates includes: Converting the target characters into the physical coordinates of the circular areas according to the distance between adjacent circle centers on the calibration board; Pair the centroid coordinates of each of the circular regions with the corresponding physical coordinates to form a set of point pairs; Calculate the affine transformation matrix between the camera and the calibration board based on the set of point pairs, and iteratively optimize the affine transformation matrix using the RANSAC algorithm.

7. The large panel size measurement method according to any one of claims 1 to 6, characterized in that, The obtaining the regional image of the display panel to be measured by the camera, and extracting the corner coordinates of the display panel to be measured according to the regional image, includes: Obtain the regional image of the display panel to be measured by the camera, and use a caliper tool to extract the sub-pixel coordinate points of the screen edge to generate an edge point set; Fit the edge point set by the least squares method to obtain a fitted line corresponding to the screen edge; Determine the corner coordinates of the display panel to be measured according to the intersection points of the fitted lines.

8. A large panel size measurement system for multi-camera joint automatic calibration, characterized in that The large panel size measurement system includes: A first photographing unit, configured to obtain a calibration image of a corresponding area of a calibration board by photographing with at least two cameras, wherein a plurality of circular regions are regularly distributed on the calibration board, and each of the circular regions contains corresponding row and column index character information; An extraction unit, configured to extract target characters in the calibration image and the centroid coordinates of the circular regions in the calibration image; A calibration unit, configured to calculate the physical coordinates of the calibration board based on the target characters and the actual size information of the calibration board, and establish an affine transformation matrix between the camera and the calibration board according to the centroid coordinates and the physical coordinates; A second photographing unit, configured to obtain a regional image of the display panel to be measured by the camera, and extract the corner coordinates of the display panel to be measured according to the regional image; A detection unit, configured to map the corner coordinates to the coordinate system of the calibration board through the affine transformation matrix to obtain the target coordinates of the corners of the display panel in the coordinate system of the calibration board, and calculate the size information of the display panel to be measured according to the target coordinates.

9. A large panel size measuring device for multi-camera joint automatic calibration, characterized in that The large panel size measurement device includes: A processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored on the computer-readable storage medium, and when the program is executed on a computer, the method according to any one of claims 1 to 7 is executed.

Images