Automatic calibration method based on two-dimensional code calibration board

Abstract

The invention relates to the technical field of image processing, in particular to an automatic calibration method based on a two-dimensional code calibration board, and the method comprises the steps: obtaining a calibration board image, and carrying out the preprocessing of the calibration board image; fitting and positioning the contour of the Mark point target, obtaining the scaling and target angle in the X and Y directions, determining the direction of the calibration plate according to the position relation of the two Mark point targets, rotating the image by using affine transformation, and completing the correction of the direction of the image of the calibration plate; carrying out tangential distortion correction on the calibration plate image; reading two-dimensional code information of the calibration board image after tangential correction; and carrying out radial distortion correction on the calibration plate image. Tangential distortion and radial distortion correction is performed on calibration plate images of a Mark target and a two-dimensional code, and camera calibration and image calibration are realized; based on a Mark target and a calibration board image of a two-dimensional code, related information stored in the calibration board is automatically obtained, and camera calibration and image calibration are automatically completed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an automatic calibration method based on a two-dimensional code calibration plate. Background Art

[0002] Calibration plates are used in machine vision devices or components to correct camera lens distortion, determine the proportional relationship between pixel size and actual size, and calculate the posture of the imaging system in the world coordinate system. By taking pictures of the calibration plate with a special image through the imaging system and performing algorithmic calculations on a series of pictures, an abstract model of the imaging system can be obtained, thereby completing operations such as image distortion correction and pixel size calibration.

[0003] The commonly used checkerboard calibration plates or dot array calibration plates currently have a single structure. When used, they need to manually enter the specifications, models and other relevant information of the calibration plates before calibration can be performed. The degree of automation is insufficient and cannot meet the current visual equipment and full-process unmanned calibration usage scenarios. Summary of the invention

[0004] In view of the shortcomings of the existing methods, the present invention corrects the tangential distortion and radial distortion of the Mark target and the calibration plate image of the QR code to achieve camera calibration and image calibration; based on the Mark target and the calibration plate image of the QR code, it realizes automatic acquisition of relevant information stored in the calibration plate, and automatically completes the camera calibration and image calibration.

[0005] The technical solution adopted by the present invention is: an automatic calibration method based on a two-dimensional code calibration plate comprises the following steps:

[0006] Step 1: Obtain an image of a calibration plate containing four Mark point targets and a QR code, and preprocess the calibration plate image;

[0007] As a preferred embodiment of the present invention, the pretreatment comprises:

[0008] Step 11, cropping the image containing the calibration plate area;

[0009] Step 12: Binarize the cropped image and extract the black area in the image; segment the black image and the white background using a grayscale threshold; fill the holes in the black area; fill and trim the pits and bumps around the black area; fill the grayscale value of the filled and trimmed image with 0; and fill the grayscale value of pixels that are not filled with 0 with 255.

[0010] Step 2: Fit and locate the contour of the Mark point target, obtain the scaling ratio and target angle in the XY direction, determine the direction of the calibration plate according to the positional relationship between the two Mark point targets, rotate the image using affine transformation, and complete the direction correction of the calibration plate image;

[0011] As a preferred embodiment of the present invention, step 2 specifically includes:

[0012] Step 21: Using the contour features of the Mark point targets as a template, search for four Mark point targets in the calibration plate image;

[0013] Step 22, obtaining the scaling ratio, target angle and minimum unit of the calibration plate image in the XY direction according to the contour matching;

[0014] As a preferred implementation of the present invention, the minimum unit is the width of the black circle of the Mark, the length and width of the cross, and the side length of the black block of the QR code.

[0015] Step 23: Set the Mark target with a white dot at the center of the target outline at the upper left corner of the calibration plate image by rotating.

[0016] Step 3: Correct the tangential distortion of the calibration plate image;

[0017] As a preferred embodiment of the present invention, step three specifically includes:

[0018] Step 31, diagonally intersect the actual center coordinates of the four Mark targets to obtain the corrected center point coordinates;

[0019] Step 32, obtaining the theoretical center coordinates of the four Mark targets according to the coordinates of the correction center point and the distance between the actual center point of the Mark target and the correction center point;

[0020] Step 33, obtaining a homogeneous projective transformation matrix through the actual coordinates of the four Mark targets and the corresponding theoretical center coordinates;

[0021] Step 34: Perform a projection transformation on the calibration plate image according to the homogeneous projection transformation matrix to obtain a tangentially corrected calibration plate image.

[0022] Step 4: Read the QR code information of the calibration plate image after tangential correction;

[0023] As a preferred embodiment of the present invention, step 4 specifically includes:

[0024] Step 41, dividing the QR code of the calibration plate image after tangential correction into a plurality of regions and labeling them;

[0025] Step 42: Take the upper left corner of a numbered QR code as the starting coordinate and the minimum pixel unit of the graphic as the side length to generate a square reading block; calculate the grayscale mean value for each reading block, and mark the reading block as "0" when the grayscale mean value is higher than the judgment threshold, otherwise mark it as "1";

[0026] Step 43, perform an XOR operation on the reading block bit by bit with the mask, decode the first character of the ASCII code string, and obtain the valid information of the calibration plate in the reading block by multiplying the first character of the ASCII code string by the bit number 8; other bits beyond the range of the multiplication of the first character of the ASCII code string and the bit number 8 are invalid information;

[0027] Step 44 : Obtain a proportionality coefficient between pixel coordinates and world coordinates according to the minimum physical unit of the image of the calibration plate and the minimum pixel unit of the image of the calibration plate.

[0028] As a preferred implementation manner of the present invention, the proportionality factor=minimum pixel unit / minimum physical unit.

[0029] Step 5: Correct the radial distortion of the calibration plate image;

[0030] As a preferred implementation of the present invention, step five specifically includes:

[0031] Through the relationship transformation between the camera coordinate system, image coordinate system and pixel coordinate system, a relationship model between camera coordinates and pixel coordinates is established; homogeneous coordinates are introduced, and the intrinsic parameter matrix is ​​calculated using the corner point coordinates of N groups of black squares in the QR code of the calibration plate image; the radial distortion correction is performed using the intrinsic parameter matrix.

[0032] As a preferred embodiment of the present invention, an automatic calibration system based on a two-dimensional code calibration plate includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement an automatic calibration method based on a two-dimensional code calibration plate.

[0033] As a preferred embodiment of the present invention, a computer readable medium stores a computer program code, and when the computer program code is executed by a processor, an automatic calibration method based on a two-dimensional code calibration plate is implemented.

[0034] Beneficial effects of the present invention:

[0035] 1. The calibration is completed automatically throughout the entire process. The QR code in the calibration plate can store the relevant information of the calibration plate required during the calibration process, such as the actual physical size of the Mark point target and the QR code, the model and serial number of the calibration plate, etc.; no manual input or participation in the calibration process is required, and the entire process is automated for calibration and identification.

[0036] 2. The calibration plate image combines the chessboard with the QR code. The calibration plate image not only has the function of storing information of the QR code, but also has the function of calibrating the chessboard image.

[0037] 3. Special Mark target design. The Mark point target has a large proportion and is composed of a circle center and a rectangle. Therefore, the positioning speed is faster, the positioning accuracy is higher, and the application range is wider, especially when the calibration plate occupies a small proportion of the field of view. The minimum image unit size can be obtained in the Mark point target, which is used to segment the image area.

[0038] 4. The mark point targets are located at the four corners, which is more conducive to correcting tangential distortion; two types of targets, targets with white circle features and targets without white circle features, can be used to indicate the direction of the calibration plate. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the automatic calibration method based on the two-dimensional code calibration plate of the present invention;

[0040] Figure 2 The red area is a schematic diagram of the cut-out calibration plate area;

[0041] Figure 3 It is the local effect map of the black area extracted by threshold segmentation;

[0042] Figure 4 This is a local rendering of the holes in the black area filled;

[0043] Figure 5 It is a local effect diagram of filling and trimming the pits and bulges around the black area;

[0044] Figure 6 This is the local effect image after the gray value of the filled and trimmed image is filled with 0;

[0045] Figure 7 This is the local effect image after the grayscale value of the filled and trimmed image is filled with 255;

[0046] Figure 8 It is a schematic diagram of the contour features of the Mark point target;

[0047] Fig. 9 is a schematic diagram of the searched target contour matching;

[0048] Fig.10 It is a schematic diagram of the positioning coordinates of the four targets;

[0049] Fig.11 It is a schematic diagram of rotating the image so that the Mark target with a small circle in the center is located in the upper left corner;

[0050] Fig.12 This is a schematic diagram of the intersection of diagonal lines generated by the center of the Mark target;

[0051] Fig.13 is a schematic diagram before tangential correction;

[0052] Fig.14 is a schematic diagram after tangential correction;

[0053] Fig.15 The QR code information is divided into five areas;

[0054] Fig.16 It is a schematic diagram of QR code area reading;

[0055] Fig.17 This is a schematic diagram of the black square corner point recognition result;

[0056] Fig.18 It is a schematic diagram of coordinate system transformation;

[0057] Fig.19 It is a schematic diagram of image subtraction before and after radial distortion correction. DETAILED DESCRIPTION

[0058] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0059] like Figure 1 As shown, an automatic calibration method based on a two-dimensional code calibration plate includes the following steps:

[0060] Step 1: Image preprocessing;

[0061] It mainly includes two parts. One part is to crop the area containing the calibration plate from the entire image; the other part is to binarize the image to reduce background interference, reduce the amount of calculation, and improve the calibration speed and robustness.

[0062] Step 11: First, Figure 2 As shown, a rough positioning operation is performed on the image, and the calibration image is cut out from the entire image through threshold segmentation;

[0063] Step 12: Next, Figure 3 As shown, the binarization operation includes threshold segmentation to extract the black area and segmenting the black image from the white background by grayscale threshold;

[0064] Secondly, if Figure 4 As shown, the holes in the black area are filled to prevent non-black influences such as interference and foreign matter inside the black image;

[0065] Secondly, if Figure 5As shown in the figure, the pits and protrusions at the periphery of the black figure are filled and trimmed through morphological processing to make the figure more consistent, and the grayscale value of the image area after morphological processing is filled with 0 to make the black block more complete and unified, and the grayscale value of the pixels in the image that are not filled with 0 is filled with 255 to make the black and white contrast stronger and the picture purer; Figure 6 and Figure 7 The local effect graph of the filled and trimmed image with grayscale value filled with 0 and the local effect graph of the filled and trimmed image with grayscale value filled with 255 are respectively shown.

[0066] Step 2: Correct the direction of the calibration plate;

[0067] Step 21, by fitting the contour of the Mark point target, obtain the image XY direction scaling ratio, target angle, determine the calibration plate direction according to the positional relationship between the two Mark point targets, and use affine transformation to rotate the image to the specified angle; the contour features of the Mark point target are as follows: Figure 8 As shown in the figure, the contour feature is used as a template to search in the calibration plate image, such as Fig. 9 is a schematic diagram of the searched target contour matching; the positioning coordinates of the four targets are obtained. If the four targets are not found, the calibration plate image does not meet the requirements and is discarded; Fig.10 Schematic diagram of the positioning coordinates of the four targets.

[0068] Step 22: Obtain the scaling ratio of the calibration plate image in the XY direction, the target angle, and the minimum unit of the calibration image according to the result of contour matching; the minimum unit of the image is the width of the black circle of the Mark, the length and width of the cross, and the side length of the black block of the QR code, and the shape of the entire calibration image is an integer multiple of the minimum unit of the image;

[0069] Step 23: Based on whether there is a small white circle in the center of the target contour, the target is divided into two types. The Mark target with a small circle in the center of the target should be located in the upper left corner of the calibration plate image. If the Mark target with a small circle in the center is not in the upper left corner, the image is rotated so that it is located in the upper left corner. Fig.11 As shown, the image is rotated so that the Mark target with a small circle in the center is located in the upper left corner; the two targets are used to distinguish the direction of the calibration plate.

[0070] Step 3: Correct the tangential distortion of the calibration plate image;

[0071] Step 31: Fig.12 , based on the actual center coordinates of the four Mark targets before angle correction, that is, the center coordinates of the target are the coordinates of the cross center point, the intersection of the diagonals is generated, and the intersection of the two diagonals is taken as the correction center point;

[0072] Step 32, calculate the theoretical center coordinates of the four Mark targets as integer multiples of the minimum pixel unit of the graphic, that is, the anchor point coordinates; for example, assuming the coordinates of the corrected center point are (x, y), the distance between the actual center points of the four Mark targets and the corrected center point is z minimum units; taking z=10 as an example, the theoretical coordinates of the four Mark targets are (x-10n, y+10n), (x+10n, y+10n), (x+10n, y-10n), (x-10n, y-10n), and n is the minimum pixel unit of the graphic, pixel;

[0073] Step 33, locate the correspondence between the actual center coordinates of the mark target and the theoretical center coordinates through image recognition, and calculate the homogeneous projective transformation matrix;

[0074] Step 34: Perform a projection transformation on the original image according to the homogeneous projection transformation matrix to obtain a tangentially corrected calibration plate image; Fig.13 and Fig.14 These are the images before and after tangential correction. Before correction, the line connecting the centers of the mark targets is obviously a trapezoid, indicating that the image has obvious distortion. After correction, the red line becomes a rectangle, indicating that the correction is completed.

[0075] Step 4: Read the QR code information of the calibration plate image;

[0076] Step 41, obtain the stored calibration plate information by parsing the information in the calibration plate image; firstly, cut the QR code reading area according to the Mark target position after tangential correction, and divide the QR code reading area into five positions, such as Fig.15 The diagram shows the five positions of the QR code, with the center numbered 1, and the left, right, top and bottom numbers 2 to 5;

[0077] Step 42, read in sequence according to the label order; when reading, take the upper left corner of each position as the starting coordinate, take the minimum pixel unit of the graphic as the side length, generate a square reading block, calculate the grayscale mean of each reading block, when the grayscale mean is higher than the judgment threshold, the reading block is marked as "0", otherwise it is marked as "1"; record the mark value of each reading block in order from left to right and from top to bottom, such as Fig.16 shown.

[0078] Step 43, after the tag values ​​of all the reading blocks are read, an XOR operation is performed on the bit by bit with the mask. The function of the mask is to prevent the appearance of large areas of identical black and white blocks, which is not conducive to calibration. After the bitwise XOR operation, a decoding operation is performed to convert the binary into an ASCII code string. The first digit of the ASCII code string is the number of valid bits of the data. The character string is intercepted according to the number of valid bits to obtain the calibration board information stored in the QR code.

[0079] by Fig.16The QR code reading area labeled 1 is the first 8 bits as shown in Table 1. The ASCII code is parsed to obtain 6 valid digits and 8 bits. The calibration plate information is stored in the first 6*8=48 reading blocks. The calibration plate information includes: the actual physical size of the Mark point target and the QR code, the model and serial number of the calibration plate, the smallest physical unit of the calibration plate image and other information.

[0080] Table 1 Converts binary tag values ​​to ASCII codes

[0081]

[0082] Step 44, by reading the QR code information, the minimum physical unit (mm) of the image of the calibration plate is known. According to the minimum pixel unit (pixel) of the calibration plate image recognized by the image, the actual size corresponding to each pixel can be calculated, thereby accurately obtaining the proportional coefficient β between the pixel coordinate and the world coordinate = minimum pixel unit / minimum physical unit = pixel / mm.

[0083] β is used in actual product testing to convert pixel values ​​in the image coordinate system into physical coordinate values ​​in the world coordinate system, which is used for related testing of the product to be tested by the equipment.

[0084] Step 5: Correct the radial distortion of the calibration plate image;

[0085] Based on the pinhole camera model, the relationship model between camera coordinates and pixel coordinates is established by converting the relationship between the camera coordinate system, image coordinate system, and pixel coordinate system, and homogeneous coordinates are introduced; for each calibration plate image, the coordinates of the corner points of the black square are identified, such as Fig.17 As shown in the figure, N corner point coordinates (N≥4) are obtained, and combined with the read calibration plate information, they are brought into the pinhole camera model to calculate the intrinsic parameter matrix of the visual system; the radial distortion correction of the image can be completed using the intrinsic parameter matrix.

[0086] Corner points are the intersection points of black and white, so the contrast of corner point coordinates is the strongest. Using angle coordinates to calculate the intrinsic parameter matrix has the advantages of high accuracy and strong stability.

[0087] like Fig.18 First of all, the pinhole camera model is the process of mapping the coordinate points of the three-dimensional world to the two-dimensional image plane. c (X c ,Y c ,Z c ) is converted to the image coordinate system O i (x,y), according to the triangle similarity relationship:

[0088]

[0089] Among them, Xc is the projection of point P on the X axis to O c The distance, Y c is the projection of point P on the Y axis to O c The distance, Z c is the projection of point P on the Z axis to O c distance, f is the focal length (f = O i O c ), x is P i The projection of the point on the X axis to O c The distance, y is P i The projection of the point on the Y axis to O c The distance, P i It is a point in the image coordinate system corresponding to point P.

[0090] From the image coordinate system O i (x,y) to pixel coordinate system O p (u,v), we need to add zoom and translation:

[0091]

[0092] Among them, d x and d y is the size of the unit pixel of the camera sensor, in pixel / mm; c x and c y is the center of the image, in pixels; f nx and f ny is the normalized focal length in the X and Y directions; W sensor and H sensor is the physical size of the camera sensor in mm; W image and H image is the width and height of the image.

[0093] The relationship between the camera coordinate system and the pixel coordinate system is obtained as follows:

[0094]

[0095] Introduce the secondary coordinates:

[0096]

[0097] in, The intrinsic parameter matrix of the visual system required to correct distortion.

[0098] Radial distortion is caused by the manufacturing process of the lens. The radial distortion is smallest at the center of the lens and becomes more severe as it moves away from the center. 2 ) The first few terms of the Taylor series expansion of ) approximately describe the radial distortion, and the normalized coordinates of the radial distortion are:

[0099]

[0100] Among them, k1 and k2 are radial distortion correction coefficients, x distorted and distorted is the actual coordinate value affected by the distortion.

[0101] In summary, integrating the above formulas gives:

[0102]

[0103] Among them, (u, v) is the theoretical coordinate value of the corner point calculated by reading the calibration plate information, (u distorted , v distorted ) is the actual coordinate value of the corner point identified in the actual image, c x and c y The coordinates of the correction center point generated by the four mark points, in pixels.

[0104] Take 15 images of the calibration plate. The calibration plate is complete and unobstructed in the image and is located at different positions and angles in the image. Identify the black squares in the specified area, obtain the corner coordinates of the black squares, remove abnormal data, and filter out the pixel coordinates of the same corner points in different images. Substitute the corner point coordinates into the model of Formula 6. According to the "Zhang Zhengyou calibration method", the intrinsic parameter matrix and radial distortion correction coefficient of the visual system can be obtained, thereby completing the radial distortion correction of the image.

[0105] like Fig.19 This is a schematic diagram of pixel-by-pixel subtraction before and after radial distortion correction, showing the effects before and after distortion.

[0106] The present invention proposes an automatic calibration method based on a two-dimensional code calibration plate. Based on the image of a customized special image two-dimensional code calibration plate, the relevant information stored in the calibration plate is automatically obtained to automatically complete the calibration and image calibration functions of the camera.

[0107] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. An automatic calibration method based on a two-dimensional code calibration plate, characterized in that: The following steps are involved: Step 1: Obtain a calibration plate image containing four Mark point targets and a QR code, and preprocess the calibration plate image; Step 2: Fit and locate the contour of the Mark point target, obtain the scaling ratio and target angle in the XY direction, determine the direction of the calibration plate according to the positional relationship between the two Mark point targets, rotate the image using affine transformation, and complete the direction correction of the calibration plate image; Step 3: Correct the tangential distortion of the calibration plate image; Step 4: Read the QR code information of the calibration plate image after tangential correction; Step 5: Correct the radial distortion of the calibration plate image.

2. The automatic calibration method based on the two-dimensional code calibration plate according to claim 1 is characterized in that: Preprocessing includes: Step 11, cropping the image containing the calibration plate area; Step 12: Binarize the cropped image and extract the black area in the image; segment the black image and the white background using a grayscale threshold; fill the holes in the black area; fill and trim the pits and bumps around the black area; fill the grayscale value of the filled and trimmed image with 0; and fill the grayscale value of pixels that are not filled with 0 with 255.

3. The automatic calibration method based on the two-dimensional code calibration plate according to claim 1 is characterized in that: Step 2 specifically includes: Step 21: Using the contour features of the Mark point targets as a template, search for four Mark point targets in the calibration plate image; Step 22, obtaining the scaling ratio, target angle and minimum unit of the calibration plate image in the XY direction according to the contour matching; Step 23: Set the Mark target with a white dot at the center of the target outline at the upper left corner of the calibration plate image by rotating.

4. The automatic calibration method based on the two-dimensional code calibration plate according to claim 3 is characterized in that: The minimum unit is the width of the black circle of the Mark, the length and width of the cross, and the side length of the black block of the QR code.

5. The automatic calibration method based on the two-dimensional code calibration plate according to claim 1, characterized in that: Step three specifically includes: Step 31, diagonally intersect the actual center coordinates of the four Mark targets to obtain the corrected center point coordinates; Step 32, obtaining the theoretical center coordinates of the four Mark targets according to the coordinates of the correction center point and the distance between the actual center point of the Mark target and the correction center point; Step 33, obtaining a homogeneous projective transformation matrix through the actual coordinates of the four Mark targets and the corresponding theoretical center coordinates; Step 34: Perform a projection transformation on the calibration plate image according to the homogeneous projection transformation matrix to obtain a tangentially corrected calibration plate image.

6. The automatic calibration method based on the two-dimensional code calibration plate according to claim 1, characterized in that: Step 4 specifically includes: Step 41, dividing the QR code of the calibration plate image after tangential correction into a plurality of regions and labeling them; Step 42: Take the upper left corner of a numbered QR code as the starting coordinate and the minimum pixel unit of the graphic as the side length to generate a square reading block; calculate the grayscale mean value for each reading block, and mark the reading block as "0" when the grayscale mean value is higher than the judgment threshold, otherwise it is marked as "1"; Step 43, perform an XOR operation on the reading block and the mask bit by bit, calculate the first bit of the ASCII code string, and obtain the calibration plate information in the reading block by multiplying the first bit of the ASCII code string by the bit number; Step 44 : Obtain a proportionality coefficient between pixel coordinates and world coordinates according to the minimum physical unit of the image of the calibration plate and the minimum pixel unit of the image of the calibration plate.

7. The automatic calibration method based on the two-dimensional code calibration plate according to claim 6, characterized in that: Scale factor = minimum pixel unit / minimum physical unit.

8. The automatic calibration method based on the two-dimensional code calibration plate according to claim 1, characterized in that: Step 5 specifically includes: By transforming the relationship between the camera coordinate system, image coordinate system and pixel coordinate system, a relationship model between camera coordinates and pixel coordinates is established; homogeneous coordinates are introduced, and the coordinates of the corner points of several black squares in the QR code of the calibration plate image are used to calculate the intrinsic parameter matrix; the intrinsic parameter matrix is ​​used to correct radial distortion.

9. The automatic calibration system based on the QR code calibration plate is characterized by: include: a memory for storing instructions executable by a processor; A processor, used to execute instructions to implement the automatic calibration method based on the two-dimensional code calibration plate as described in any one of claims 1-8.

10. A computer readable medium storing computer program code, characterized in that: When the computer program code is executed by a processor, the automatic calibration method based on the two-dimensional code calibration plate as described in any one of claims 1 to 8 is implemented.

Images