An automatic calibration method based on a QR code calibration board

Abstract

This invention relates to the field of image processing technology, and more particularly to an automatic calibration method based on a QR code calibration board. The method includes acquiring a calibration board image and preprocessing it; fitting and locating the contours of the Mark point targets, obtaining the scaling ratio in the XY directions and the target angle, determining the calibration board orientation based on the positional relationship between the two Mark point targets, rotating the image using an affine transformation to complete the orientation correction of the calibration board image; correcting tangential distortion in the calibration board image; reading QR code information from the tangentially corrected calibration board image; and correcting radial distortion in the calibration board image. This invention corrects tangential and radial distortion in calibration board images of Mark targets and QR codes to achieve camera calibration and image calibration; based on the calibration board images of Mark targets and QR codes, it automatically acquires relevant information stored on the calibration board and automatically completes camera calibration and image calibration.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an automatic calibration method based on a QR code calibration board. Background Technology

[0002] In machine vision equipment or components, calibration boards are used to correct camera lens distortion, determine the ratio between pixel size and actual size, and calculate the orientation of the imaging system in the world coordinate system. By taking pictures of the calibration board with special images through the imaging system and performing algorithmic calculations on a series of images, an abstract model of the imaging system can be obtained, thereby completing operations such as image distortion correction and pixel size calibration.

[0003] Currently used checkerboard or dot array calibration boards have a simple structure. They require manual input of the calibration board's specifications, model, and other relevant information before calibration can be performed. This lack of automation cannot meet the needs of current vision equipment for fully unmanned calibration processes. Summary of the Invention

[0004] To address the shortcomings of existing methods, this invention corrects tangential and radial distortion in the calibration board images of the Mark target and QR code, thereby achieving camera calibration and image calibration. Based on the calibration board images of the Mark target and QR code, it automatically acquires relevant information stored on the calibration board and automatically completes camera calibration and image calibration.

[0005] The technical solution adopted in this invention is: an automatic calibration method based on a QR code calibration board, comprising the following steps:

[0006] Step 1: Obtain an image of a calibration board containing four Mark points and a QR code, and preprocess the calibration board image;

[0007] In a preferred embodiment of the present invention, the preprocessing includes:

[0008] Step 11: Crop the image containing the calibration board area;

[0009] Step 12: Binarize the cropped image to extract the black area; separate the black image from the white background using a grayscale threshold; fill the holes in the black area; fill and trim the pits and bumps around the black area; fill the grayscale value of the filled and trimmed image with 0; fill the grayscale value of pixels that are not filled with 0 with 255.

[0010] Step 2: Fit and locate the contour of the Mark point target, obtain the scaling ratio and target angle in the XY direction, determine the orientation of the calibration board based on the positional relationship between the two Mark point targets, and use affine transformation to rotate the image to complete the orientation correction of the calibration board image.

[0011] In a preferred embodiment of the present invention, step two specifically includes:

[0012] Step 21: Using the contour features of the Mark point target as a template, search for four Mark point targets within the calibration board image;

[0013] Step 22: Obtain the scaling ratio, target angle, and minimum unit of the calibration board image in the XY direction based on contour matching;

[0014] In a preferred embodiment of the present invention, the smallest unit is the width of the black Mark ring, the length and width of the cross shape, and the side length of the black QR code block.

[0015] Step 23: Rotate the target with the white dot at the center of the target outline to the upper left corner of the calibration board image.

[0016] Step 3: Perform tangential distortion correction on the calibration plate image;

[0017] In a preferred embodiment of the present invention, step three specifically includes:

[0018] Step 31: Divide the actual center coordinates of the four Mark targets diagonally to obtain the coordinates of the correction center point;

[0019] Step 32: Obtain the theoretical center coordinates of the four Mark targets based on the coordinates of the correction center point and the distance between the actual center point and the correction center point of the Mark target;

[0020] Step 33: Obtain the homogeneous projective transformation matrix using the actual coordinates of the four Mark targets and their corresponding theoretical center coordinates;

[0021] Step 34: Perform a projection transformation on the calibration plate image according to the homogeneous projective transformation matrix to obtain the tangentially corrected calibration plate image.

[0022] Step 4: Read the QR code information from the calibration board image after tangential correction;

[0023] In a preferred embodiment of the present invention, step four specifically includes:

[0024] Step 41: Divide the QR code on the calibration board image after tangential correction into several regions and label them;

[0025] Step 42: Using the top left corner of a QR code with a certain number as the starting coordinate and the smallest pixel unit of the graphic as the side length, generate a square reading block; calculate the grayscale average value of each reading block, and mark the reading block as "0" when the grayscale average value is higher than the judgment threshold, otherwise mark it as "1".

[0026] Step 43: Perform an XOR operation on the bitwise AND mask of the reading block to decode the first character of the ASCII code string. Multiply the first character of the ASCII code string by the number of bits 8 to obtain the valid information of the calibration board in the reading block; other bits that are outside the range of multiplying the first character of the ASCII code string by the number of bits 8 are invalid information.

[0027] Step 44: Obtain the scaling factor between pixel coordinates and world coordinates based on the smallest physical unit and the smallest pixel unit of the calibration board image.

[0028] In a preferred embodiment of the present invention, the scaling factor = minimum pixel unit / minimum physical unit.

[0029] Step 5: Correct radial distortion of the calibration plate image;

[0030] In a preferred embodiment of the present invention, step five specifically includes:

[0031] By transforming the relationships between the camera coordinate system, image coordinate system, and pixel coordinate system, a relationship model between camera coordinates and pixel coordinates is established; homogeneous coordinates are introduced, and the intrinsic parameter matrix is ​​calculated using the corner coordinates of N black squares in the QR code of the calibration board image; the intrinsic parameter matrix is ​​then used for radial distortion correction.

[0032] As a preferred embodiment of the present invention, an automatic calibration system based on a QR code calibration board includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement an automatic calibration method based on a QR code calibration board.

[0033] In a preferred embodiment of the present invention, a computer-readable medium storing computer program code implements an automatic calibration method based on a QR code calibration board when executed by a processor.

[0034] The beneficial effects of this invention are:

[0035] 1. The entire calibration process is automated. The QR code on the calibration board can store relevant information about the calibration board required during the calibration process, such as the actual physical size of the Mark point target and the QR code, the model of the calibration board, the serial number, etc. No manual input or participation in the calibration process is required, realizing fully automated calibration and recognition.

[0036] 2. The calibration board image combines a checkerboard pattern with a QR code. This calibration board image not only has the function of storing information using a QR code, but also has the function of calibrating a checkerboard pattern image.

[0037] 3. The special Mark target design, with a large proportion of Mark points composed of both circles and rectangles, results in faster positioning speed, higher positioning accuracy, and a wider range of applications, especially when the calibration board occupies a small area of ​​the field of view; the Mark point target can obtain the smallest image unit size for segmenting image regions.

[0038] 4. The Mark point targets are located at the four corners, which is more conducive to correcting tangential distortion; two types of targets, one with a small white circle feature and the other without, can be used to indicate the orientation of the calibration plate. Attached Figure Description

[0039] Figure 1 This is a flowchart of the automatic calibration method based on a QR code calibration board according to the present invention;

[0040] Figure 2 The red area is a schematic diagram of the cut-out calibration plate area;

[0041] Figure 3 This is a local image showing the effect of threshold segmentation and extraction of the black region;

[0042] Figure 4 This is a partial rendering of the holes filled in the black area;

[0043] Figure 5 This is a partial effect image showing the filling and trimming of the pits and bumps around the black area;

[0044] Figure 6 This is a partial image showing the effect of filling and cropping the image and then filling its grayscale values ​​to 0.

[0045] Figure 7 This is a partial image showing the effect of filling and cropping the image and then filling the grayscale value to 255.

[0046] Figure 8 This is a schematic diagram of the outline features of the Mark point target;

[0047] Figure 9 This is a schematic diagram of the target contour matching found;

[0048] Figure 10 This is a schematic diagram showing the location coordinates of the four targets;

[0049] Figure 11 This is a diagram showing how the image was rotated so that the Mark target with a small circle in the center is located in the upper left corner.

[0050] Figure 12 This is a schematic diagram showing the intersection of the diagonal lines generated at the center of the Mark target;

[0051] Figure 13 This is a schematic diagram before tangential correction;

[0052] Figure 14 This is a schematic diagram after tangential correction;

[0053] Figure 15 The QR code information is divided into five areas;

[0054] Figure 16 This is a diagram illustrating the QR code reading area;

[0055] Figure 17 This is a schematic diagram of the black square corner detection results;

[0056] Figure 18 This is a diagram illustrating coordinate system transformation;

[0057] Figure 19 This is a schematic diagram showing the subtraction of images before and after radial distortion correction. Detailed Implementation

[0058] The present invention will be further described below with reference to the accompanying drawings and embodiments. The drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0059] like Figure 1 As shown, an automatic calibration method based on a QR code calibration board includes the following steps:

[0060] Step 1: Image preprocessing;

[0061] It mainly consists of two parts: one part is to crop out the region containing the calibration plate from the whole image; the other part is to perform binarization processing on the image, thereby reducing background interference, reducing the amount of computation, and improving calibration speed and robustness.

[0062] Step 11, First, as Figure 2 As shown, a coarse localization operation is performed on the image, and the calibration image is cut out from the whole image through threshold segmentation;

[0063] Step 12, Secondly, as Figure 3 As shown, the binarization operation includes threshold segmentation to extract the black region, and the black image is segmented from the white background using a grayscale threshold.

[0064] Secondly, such as Figure 4 As shown, holes in the black area are filled to prevent interference and foreign objects from affecting the black image.

[0065] Secondly, such as Figure 5As shown, morphological processing fills and trims the pits and bumps around the black graphic's outline, improving graphic consistency. The image areas after morphological processing are filled with 0 grayscale values ​​to make the black blocks more complete and uniform. Pixels with grayscale values ​​not filled to 0 are filled with 255 grayscale values, resulting in stronger black-and-white contrast and a cleaner image. Figure 6 and Figure 7 The images show partial effects of filling and cropping the images with grayscale values ​​of 0 and 255, respectively.

[0066] Step 2: Correct the orientation of the calibration plate;

[0067] Step 21: By fitting and locating the contour of the Mark point target, obtain the scaling ratio in the XY direction of the image and the target angle. Determine the calibration plate orientation based on the positional relationship between the two Mark point targets, and use affine transformation to rotate the image to the specified angle; the contour features of the Mark point target are as follows: Figure 8 As shown, using contour features as a template, a search is performed within the calibration board image, such as... Figure 9 This is a schematic diagram of the target contour matching found; the positioning coordinates of the four targets are obtained. If none of the four targets are found, the calibration plate image does not meet the requirements and is discarded. Figure 10 This is a schematic diagram showing the positioning coordinates of the four targets.

[0068] Step 22: Based on the contour matching results, obtain the scaling ratio of the calibration board image in the XY direction, the target angle, and the smallest unit of the calibration image; the smallest unit of the image is the width of the black circle of the Mark, the length and width of the cross shape, and the side length of the black QR code block. The shape of the entire calibration image is an integer multiple of the smallest unit of the image.

[0069] Step 23: Based on the found target outline center, determine whether there is a small white circle to divide the targets into two types. Mark targets with a small circle in their center should be located in the upper left corner of the calibration board image. If a Mark target with a small circle in its center is not in the upper left corner, rotate the image to position it in the upper left corner. Figure 11 As shown, the image is rotated so that the Mark target with a small circle in the center is located in the upper left corner; the two targets are used to distinguish the orientation of the calibration plate.

[0070] Step 3: Perform tangential distortion correction on the calibration plate image;

[0071] Step 31, as Figure 12 Based on the actual center coordinates of the four Mark targets before angle correction, i.e. the center coordinates of the targets are the coordinates of the center point of the cross, the intersection of the diagonals is generated, and the intersection of the two diagonals is taken as the correction center point.

[0072] Step 32: Calculate the theoretical center coordinates of the four Mark targets in integer multiples of the smallest pixel unit of the graphic; these are the anchor point coordinates. For example, assuming the correction center point coordinates are (x, y), the distance between the actual center point of the four Mark targets and the correction center point is z smallest units. Taking z = 10 as an example, the theoretical coordinates of the four Mark targets are (x-10n, y+10n), (x+10n, y+10n), (x+10n, y-10n), (x-10n, y-10n), where n is the smallest pixel unit of the graphic.

[0073] Step 33: Locate the correspondence between the actual center coordinates and the theoretical center coordinates of the mark target through image recognition, and calculate the homogeneous projective transformation matrix;

[0074] Step 34: Next, perform a projection transformation on the original image according to the homogeneous projective transformation matrix to obtain the calibration plate image after tangential correction; for example... Figure 13 and Figure 14 The images are before and after tangential correction. Before correction, the line connecting the centers of the mark targets is clearly trapezoidal, indicating that the image has obvious distortion. After correction, the red line becomes a rectangle, indicating that the correction is complete.

[0075] Step 4: Read the QR code information on the calibration board image;

[0076] Step 41: Obtain the stored calibration board information by parsing the information in the calibration board image; firstly, crop the QR code reading area according to the tangentially corrected Mark target position, and divide the QR code reading area into five positions, such as... Figure 15 The QR code is divided into five positions, with the center position numbered 1, and the left, right, top, and bottom positions numbered 2 to 5 respectively.

[0077] Step 42: Read sequentially according to the label order; during reading, use the top left corner of each position as the starting coordinate and the smallest pixel unit of the image as the side length to generate a square recognition block. Calculate the grayscale mean of each recognition block. When the grayscale mean is higher than the judgment threshold, the recognition block is marked as "0", otherwise it is marked as "1". Record the label values ​​of each recognition block in order from left to right and from top to bottom, such as... Figure 16 As shown.

[0078] Step 43: After all the tag values ​​of the reading blocks have been read, perform a bitwise XOR operation with the mask. The mask is used to prevent large areas of identical black and white blocks from appearing, which is not conducive to calibration. After the bitwise XOR operation, perform the decoding action to convert the binary into an ASCII code string. The first digit of the ASCII code string is the number of valid data bits. Extract the string according to the number of valid data bits to obtain the calibration board information stored in the QR code.

[0079] by Figure 16The QR code reading area labeled 1 is, for example, the first 8 bits shown in Table 1. The ASCII code parsing yields 6 effective bits and 8 bits. Therefore, the calibration board information is stored in the first 6*8=48 reading blocks. The calibration board information stores information including: the actual physical size of the Mark target and QR code, the calibration board model, serial number, and the smallest physical unit of the calibration board image.

[0080] Table 1 shows the conversion of binary tag values ​​to ASCII codes.

[0081]

[0082] Step 44: By reading the QR code information, the smallest physical unit (mm) of the calibration board image is obtained. Based on the smallest pixel unit (pixel) of the calibration board image identified by the image, the actual size corresponding to each pixel can be calculated, thereby accurately obtaining the scaling factor β between pixel coordinates and world coordinates = smallest pixel unit / smallest physical unit = pixel / mm.

[0083] β is used in actual product testing to convert pixel values ​​in the image coordinate system to physical coordinate values ​​in the world coordinate system, and is used for the relevant testing of the product to be tested by the equipment.

[0084] Step 5: Correct radial distortion of the calibration plate image;

[0085] Based on a pinhole camera model, a relationship model between camera coordinates and pixel coordinates is established through the transformation between the camera coordinate system, image coordinate system, and pixel coordinate system, and homogeneous coordinates are introduced; for each calibration board image, the corner coordinates of the black squares are identified, such as... Figure 17 As shown, the coordinates of N corner points (N≥4) are obtained. Combined with the information from the calibration plate, the intrinsic parameter matrix of the vision system is calculated by substituting it into the pinhole camera model. The radial distortion correction of the image can be completed using the intrinsic parameter matrix.

[0086] Corner points are black and white intersections, so the contrast of corner point coordinates is the strongest. Using angular coordinates to calculate the intrinsic parameter matrix has the advantages of high accuracy and strong stability.

[0087] like Figure 18 First, the pinhole camera model is the process of mapping coordinate points in the three-dimensional world to the two-dimensional image plane, using the camera coordinate system O. c (X c ,Y c Z c Convert to image coordinate system O i (x, y), according to the similarity relationship of triangles:

[0088]

[0089] Among them, Xc It is the projection of point P onto point O along the X-axis. c The distance, Y c It is the projection of point P onto point O along the Y-axis. c The distance Z c It is the projection of point P onto point O along the Z-axis. c The distance, f is the focal length (f = 0). i O c ), x is P i The projection of the point onto the X-axis onto O c The distance y is P i The projection of the point on the Y-axis onto O c The distance, P i It is a point in the image coordinate system corresponding to point P.

[0090] From image coordinate system O i (x,y) Transform to pixel coordinate system O p (u,v) requires scaling and translation:

[0091]

[0092] Where, d x and d y This refers to the size of a single pixel on a camera sensor, expressed in pixels per mm; c x and c y The center of the image, in pixels; f nx and f ny W represents the normalized focal length in the X and Y directions. sensor and H sensor The physical dimensions of the camera sensor are in mm; W image and H image This represents the width and height of the image.

[0093] The relationship between the camera coordinate system and the pixel coordinate system is obtained as follows:

[0094]

[0095] Introducing secondary coordinates:

[0096]

[0097] in, The intrinsic parameter matrix of the visual system required to correct distortion.

[0098] Radial distortion is caused by the manufacturing process of the lens elements. Radial distortion is minimal at the center of the lens and increases with distance from the center. It can be expressed as f(r) 2 The first few terms of the Taylor series expansion of ) approximate the radial distortion, and the normalized coordinates of the radial distortion are:

[0099]

[0100] Where k1 and k2 are radial distortion correction coefficients, x distorted and y distorted These are the actual coordinate values ​​affected by distortion.

[0101] In summary, integrating the above formulas, we get:

[0102]

[0103] Where (u,v) are the theoretical coordinates of the corner point calculated by reading the calibration plate information, (u distorted v distorted ) represents the actual coordinates of the corner points identified from the actual image, and c represents the actual coordinates of the corner points identified from the actual image. x and c y The coordinates of the correction center point generated for the four Mark points, in pixels.

[0104] Fifteen images of the calibration board were captured, with the board completely unobstructed in the images and positioned at different locations and angles. Black squares in a specified area were identified, and the corner coordinates of the black squares were obtained. After removing outliers, the pixel coordinates of the same corner points in different images were selected. The corner coordinates were then substituted into the model in Equation 6. Based on the "Zhang Zhengyou calibration method", the intrinsic parameter matrix and radial distortion correction coefficients of the vision system can be obtained, thus completing the radial distortion correction of the image.

[0105] like Figure 19 This is a pixel-by-pixel subtraction diagram before and after radial distortion correction, illustrating the effect before and after distortion.

[0106] This invention proposes an automatic calibration method based on a QR code calibration board. Based on the image of a customized special image QR code calibration board, the method automatically obtains the relevant information stored on the calibration board and automatically completes the camera calibration and image calibration functions.

[0107] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. An automatic calibration method based on a QR code calibration board, characterized in that, Includes the following steps: Step 1: Obtain an image of the calibration board containing four Mark points and a QR code, and preprocess the calibration board image. Step 2: Fit and locate the contour of the Mark point target, obtain the scaling ratio and target angle in the XY direction, determine the orientation of the calibration board based on the positional relationship between the two Mark point targets, and use affine transformation to rotate the image to complete the orientation correction of the calibration board image. Step two specifically includes: Step 21: Using the contour features of the Mark point target as a template, search for four Mark point targets within the calibration board image; Step 22: Obtain the scaling ratio, target angle, and minimum unit of the calibration board image in the XY direction based on contour matching; Step 23: Rotate the target with the white dot at the center of the target outline to the upper left corner of the calibration board image; Step 3: Perform tangential distortion correction on the calibration plate image; Step 4: Read the QR code information from the calibration board image after tangential correction; Step four specifically includes: Step 41: Divide the QR code on the calibration board image after tangential correction into several regions and label them; Step 42: Using the top left corner of a QR code with a certain number as the starting coordinate and the smallest pixel unit of the graphic as the side length, generate a square reading block; calculate the grayscale average value of each reading block, and mark the reading block as "0" when the grayscale average value is higher than the judgment threshold, otherwise mark it as "1"; Step 43: Perform a bitwise XOR operation on the read block and the mask to calculate the first and second digits of the ASCII code string. Multiply the first and second digits of the ASCII code string by the number of bits to obtain the calibration board information in the read block. Step 44: Obtain the scaling factor between pixel coordinates and world coordinates based on the smallest physical unit and the smallest pixel unit of the calibration board image; Scale factor = minimum pixel unit / minimum physical unit; Step 5: Perform radial distortion correction on the calibration plate image.

2. The automatic calibration method based on a QR code calibration board according to claim 1, characterized in that, Preprocessing includes: Step 11: Crop the image containing the calibration board area; Step 12: Binarize the cropped image to extract the black area; separate the black image from the white background using a grayscale threshold; fill the holes in the black area; fill and trim the pits and bumps around the black area; fill the grayscale value of the filled and trimmed image with 0; fill the grayscale value of pixels that are not filled with 0 with 255.

3. The automatic calibration method based on a QR code calibration board according to claim 2, characterized in that, The smallest unit is the width of the black circle of the Mark, the length and width of the cross shape, and the side length of the black block of the QR code.

4. The automatic calibration method based on a QR code calibration board according to claim 1, characterized in that, Step three specifically includes: Step 31: Divide the actual center coordinates of the four Mark targets diagonally to obtain the coordinates of the correction center point; Step 32: Obtain the theoretical center coordinates of the four Mark targets based on the coordinates of the correction center point and the distance between the actual center point and the correction center point of the Mark target; Step 33: Obtain the homogeneous projective transformation matrix using the actual coordinates of the four Mark targets and their corresponding theoretical center coordinates; Step 34: Perform a projection transformation on the calibration plate image according to the homogeneous projective transformation matrix to obtain the tangentially corrected calibration plate image.

5. The automatic calibration method based on a QR code calibration board according to claim 1, characterized in that, Step five specifically includes: By transforming the relationships between camera coordinates, image coordinates, and pixel coordinates, a relationship model between camera coordinates and pixel coordinates is established; homogeneous coordinates are introduced, and the intrinsic parameter matrix is ​​calculated using the corner coordinates of several black squares in the QR code of the calibration board image; the intrinsic parameter matrix is ​​then used for radial distortion correction.

6. An automatic calibration system based on a QR code calibration board, characterized in that, include: Memory is used to store instructions that can be executed by the processor; A processor for executing instructions to implement the automatic calibration method based on a QR code calibration board as described in any one of claims 1-5.

7. A computer-readable medium storing computer program code, characterized in that, The computer program code, when executed by a processor, implements the automatic calibration method based on a QR code calibration board as described in any one of claims 1-5.

Images