Method for extracting angular points in checkerboard calibration object image

Abstract

The invention discloses a method for extracting angular points in a checkerboard calibration object image. Firstly, rough positions of angular points of a checkerboard calibration object are obtained; converting the rough position of the angular point into a supersampled image coordinate system, and intercepting an image window at the rough position of the angular point; calculating a response value of a pixel in the central area of the window, and generating a response diagram; gaussian blur is carried out on the response diagram to enable the response diagram to be smooth, and an area near an angular point in the blur response diagram is fitted into an elliptical paraboloid; calculating the sub-pixel extreme value position of the bright spot according to the parameters of the elliptical paraboloid; and finally, carrying out coordinate conversion to obtain the accurate position of the angular point. Compared with a checkerboard calibration object angular point position extraction method based on an angular point boundary, the method provided by the invention can realize higher angular point position extraction precision, thereby improving the precision of a camera calibration result, and facilitating the accurate realization of computer vision application.

Description

Technical Field

[0001] The invention belongs to the technical field of machine vision and is a method for processing a chessboard calibration object image, and in particular relates to a method for accurately extracting corner points in a chessboard calibration object image. Background Art

[0002] Camera calibration is the process of determining the geometric relationship between the world coordinate system and the image coordinate system by calculating the camera parameters. It is a necessary step for many 3D vision applications and has a key impact on the implementation of vision applications. Camera calibration usually requires the use of a calibration object with a set of key points with a known geometric arrangement. First, images of the calibration object are taken from multiple perspectives, and then the camera parameters are solved based on the known positions of the key points of the calibration object in the world coordinate system (object points) and their positions in the image coordinate system (image points) obtained by extraction.

[0003] Among various planar calibration objects, checkerboard calibration objects naturally contain key points such as checkerboard corners, and have a simple structure and are easy to produce with high precision, so they are widely used. The method of extracting corner point positions from checkerboard calibration object images based on image processing is usually divided into three steps. First, use a corner point or feature detector to obtain all possible corner point positions from the image; second, select the corner points formed by the intersection of the checkerboard grid dividing lines from the detection results of the previous step, which can be achieved with the help of the geometric rules of the checkerboard pattern; finally, the positions of the checkerboard corner points are further optimized to improve the accuracy of the corner point positions.

[0004] The last step of the above process can be achieved through a variety of different methods. The method uses the characteristic that the corner points of the chessboard are located at the intersection of the black and white grid dividing lines of the chessboard, and uses the gradient at the chessboard dividing line to extract the exact position of the corner points. This method requires that the black and white grid dividing lines in the image are clear and accurate, and has poor robustness to image noise. In practical applications, especially in non-ideal shooting conditions such as outdoor calibration, the accuracy is low. Summary of the invention

[0005] In order to solve the problems and needs existing in the background technology, the present invention proposes a method for accurately extracting corner points in a checkerboard calibration object image. The present invention is based on the centerline model of the corner point, focusing on the straight lines in all directions passing through the corner point, rather than only considering the dividing line of the black and white squares. The present invention uses a response value to estimate the distance between a point near the corner point and the exact position of the corner point. The larger the response value, the closer the point is to the exact position of the corner point. The present invention first obtains the rough position of the checkerboard corner point, and then calculates the response values ​​of all pixels in a square window near the position to form a response map. According to the relationship between the above-mentioned response value size and the distance from the corner point position, the sub-pixel brightness maximum position of the response map is the exact position of the corner point. The present invention estimates the sub-pixel brightness maximum position of the response map by fitting the response map to an elliptical parabola, thereby realizing the extraction of the sub-pixel position of the corner point.

[0006] The technical solution of the present invention is as follows:

[0007] 1. A method for extracting corner points from a checkerboard calibration object image

[0008] S10: Use a checkerboard calibration object with alternating black and white squares. The number of squares in each row and column is known, so the number and arrangement of the internal corner points of the checkerboard calibration object are known. The camera captures the checkerboard calibration object to obtain an image of the checkerboard calibration object, which should contain all the internal corner points of the checkerboard calibration object. Use existing methods to obtain a rough estimate of the position of each corner point from the checkerboard calibration object image;

[0009] S20: After supersampling the checkerboard calibration object image, a supersampled image is obtained, the rough estimated position of the corner point obtained in S10 is converted to the image coordinate system after supersampling, and then the preset image window is used to intercept the corresponding original corner point image at each corner point, thereby obtaining the original corner point image corresponding to each corner point; wherein the preset image window is a square image window, and the distance between the center of the window and the rough position of the corner point in the x and y directions is no more than 0.5 pixels. Supersampling is implemented by interpolation, such as bilinear interpolation.

[0010] S30: generating a response map corresponding to each original corner point image, and then extracting the exact position of the corner point according to the response map;

[0011] In S30, a response map corresponding to each original corner point image is generated, including:

[0012] For each pixel position in the central area of ​​the original corner point image, the response value of the pixel position is calculated according to the multi-directional line integral result of each pixel position, and all pixel positions in the central area are traversed to calculate the response value of each pixel position, thereby generating a response map, and the image center of the response map coincides with the image center of the original corner point image.

[0013] The size of the original corner image 2A×2A, the size of the response image 2B×2B and the scale of the line integral satisfy: A=B+l B , 2l B +1=L, A is half the length / width of the original corner image, B is half the length / width of the response image, l B is the half scale of the line integral, L is the total length of the line integral, A, B, l B All are positive integers.

[0014] The response value of each pixel position is calculated based on the multi-directional line integral results of the pixel position, including:

[0015] A circle with a radius of l is formed with each pixel position as the center. B The integrating circle of D The diameter is divided into two equal parts, and the direction of each diameter is recorded as a direction. The line integral result of the direction is obtained after the line integral of the sampling points on each diameter is performed. After traversing and calculating each diameter, the line integral results of multiple directions are obtained, and the variance of the line integral results of multiple directions is used as the response value of the current pixel position. The specific calculation method of the line integral is to take equally spaced (2l B +1) sampling points, the central sampling point is located at the current pixel position, and the line segment contains l on both sides of the current pixel position. B Sampling points, the distance between any adjacent sampling points is 1 pixel, the sum of the pixel values ​​at all sampling points is calculated and averaged as the directional integral value;

[0016] If the above sampling points are located at sub-pixel positions, their pixel values ​​are obtained by bilinear interpolation.

[0017] In S30, the exact position of the corner point is extracted according to the response graph, including:

[0018] First, the current response map is Gaussian blurred to obtain the blurred response map. Then, the central bright spot of the blurred response map is fitted as an elliptical parabola represented by a general equation. The sub-pixel extreme position of the bright spot is calculated according to the elliptical parabola equation obtained by fitting. Finally, the sub-pixel extreme position of the bright spot is converted from the image coordinate system of the blurred response map to the image coordinate system after supersampling, and then converted to the image coordinate system of the checkerboard calibration object image to obtain the accurate position of the corresponding corner point of the response map.

[0019] After Gaussian blurring the current response graph, the blurred response graph is obtained, including:

[0020] Use size (2l G +1,2l G+1) is used to perform Gaussian blur on the response image of size 2B×2B to obtain a blurred response image of size 2C×2C. The image center of the blurred response image coincides with the image center of the response image. The size of the response image, the size of the blurred response image and the size of the Gaussian kernel satisfy: B=C+l G , B is half the length / width of the response map, C is half the length / width of the blurred response map, l G is the half-side length of the Gaussian kernel, B, C, l G are all positive integers, and the standard deviation of the Gaussian kernel in both directions is equal, that is, σ x =σ y .

[0021] The sub-pixel extreme position of the bright spot is calculated according to the elliptical parabola equation obtained by fitting, including:

[0022] According to the elliptical parabola equation obtained by fitting, the projection point of the parabola maximum point in the image plane is calculated and used as the sub-pixel extreme position of the bright spot, that is, the exact position of the corner point in the response image.

[0023]

[0024] The superscript M represents the blurred response image, and the subscript P represents the exact position of the corner point.

[0025] The exact position of the corner point in the blurred response image Transform back to the super-sampled image coordinate system to obtain the exact position of the corner point in the super-sampled image coordinate system Wherein, the superscript S represents the super-sampled image of the checkerboard calibration object;

[0026] The exact position of the corner points expressed in the supersampled image coordinate system Transform back to the original image coordinate system to get the exact position of the corner point in the original image The superscript O represents the original image of the checkerboard calibration object.

[0027] S40: Repeat S30 to accurately extract the corner point positions of the remaining original corner point images, thereby obtaining the accurate positions of all corner points.

[0028] 2. A storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method for extracting corner points in a chessboard calibration object image is implemented.

[0029] The beneficial effects of the present invention are:

[0030] The present invention improves the accuracy of the camera calibration process by realizing high-precision extraction of the corner position of the chessboard calibration object, and ultimately improves the implementation effect of computer vision applications. In the embodiment, the error between the corner position extraction function cornerSubPix based on image gradient of OpenCV and the method of the present invention is compared through synthetic image testing and camera calibration experiments, proving that the method of the present invention has higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is the overall flow chart of the method of the present invention;

[0032] Figure 2 It is a schematic diagram of the relationship between a straight line passing through a point near a corner point and the black and white areas of the checkerboard calibration object, where (a) is a straight line passing through a corner point that only passes through the black or white area, and (b) is a straight line passing through a point outside the corner point that is partially located in the black area and the other part in the white area.

[0033] Figure 3 This is the method for selecting directional integration sampling points when calculating pixel response values, taking the case where the total number of integration directions is 8 and the number of sampling points in each direction is 25 as an example;

[0034] Figure 4 It is a schematic diagram of the image window, response map, range of the blurred response map, and the relationship between various parameters when extracting the exact position of a single corner point;

[0035] Figure 5 It is the response map of a single corner point and the response map after blur, where (a) is the response map of a single corner point and (b) is the response map after blur.

[0036] Figure 6 is a schematic diagram of a corner point composite image in Example 1, wherein (a) is an initial corner point image, and (b) is a corner point image after transformation;

[0037] Figure 7 is the reprojection error of the two groups of calibration experiments in Example 2, where (a) is the reprojection error of the first group of calibration experiments, and (b) is the reprojection error of the second group of calibration experiments. DETAILED DESCRIPTION

[0038] The present invention provides a method for accurately extracting corner points in a checkerboard calibration object image. In order to explain the implementation process of the method and its effectiveness in detail, two embodiments are described below. Among them, embodiment 1 illustrates how to use the method of the present invention to extract the corner point positions of the checkerboard calibration object in the synthetic image, and compare the error with the corner point position extraction function cornerSubPix based on the image gradient of OpenCV; embodiment 2 illustrates how to use the method of the present invention to extract the corner point positions in the checkerboard calibration object image taken by the camera, and then use these position information to calibrate the camera, and then calculate the reprojection error, and compare it with the reprojection error when using the corner point position extraction function cornerSubPix based on the image gradient of OpenCV.

[0039] Example 1

[0040] In this embodiment, a composite image I of a single corner point of a 24×24 chessboard calibration object is U Rotate around the corner point and scale with the corner point as the anchor point to obtain the intermediate composite image I after rotation and scaling RS , and then rotate and scale the intermediate composite image I RS Sub-pixel shifts are performed in the horizontal and vertical directions to obtain a synthetic image I similar to the corner image of the checkerboard calibration object taken by the camera. O The method proposed in the present invention is then used to extract the exact position of the corner point. The advantage is that the true value of the position of the corner point in the synthetic image is known, so the position extraction error of the corner point can be calculated to verify the accuracy of the algorithm.

[0041] In the following description, each image coordinate system is defined as: the center of the upper left corner pixel of the image is the origin of the image coordinate system, the vertical downward is the positive direction of the x-axis, and the horizontal rightward is the positive direction of the y-axis. The following symbols are used to describe the transformation of the image: the counterclockwise rotation angle of the image around the corner point is θ; the scaling factor of the image along the x direction with the corner point as the anchor point is s x , the scaling factor along the y direction is s y ; The sub-pixel offset of the image is

[0042] Synthetic Image I O The generation of is carried out in two steps. First, the synthetic image I U Rotate around the corner point and scale with the corner point as the anchor point to obtain the intermediate composite image I RS . The superscript RS represents the intermediate composite image after rotation and scaling. Image I U Hit the mark Its transformed position The relationship between them is: RS =S(s x ,sy )·rot(θ)·(Pd)+d, where is the scaling matrix, is the rotation matrix, d is the rotation matrix from image I U The vector from the origin of the coordinate system to the corner point, that is, the center of the image. Generate image I RS In fact, it is based on the integer pixel position P RS Inversely, it corresponds to image I U The sub-pixel position P in U , the relationship is: For example, when image I U with I RS The size is 24×24, the rotation angle is 30°, and the scaling factors in the x and y directions are s respectively. x =1,s y = 0.5, image I RS Integer position P RS The pixel value of image I U The corresponding sub-pixel position P in U The same, the relationship between the two is Image I is obtained by bilinear interpolation U Medium U The pixel value at position is taken as image I RS Located in P RS The pixel value at . RS The above operation is performed on each pixel in the image to generate an intermediate composite image I RS .

[0043] Secondly, for image I RS Sub-pixel shifts are performed in the x and y directions, with the shift amounts ranging from 0.00 to 1.00 pixels, to obtain the composite image I O Image I O Position P in O With image I RS The corresponding position P in RS The relationship between them is: RS =P O -t, where t is a vector representing the sub-pixel offset. For example, when the offsets in the x and y directions are 0.20 and 0.40 pixels respectively, the above formula is For image I O Each integer pixel position P in O , using the above relationship to calculate its RS The corresponding sub-pixel position P in RS , the gray value at that position is obtained by bilinear interpolation as image I O Located in P OThe pixel value of the pixel at , thereby generating a synthetic image I O Untransformed synthetic image I U ,like Figure 6 As shown in (a), I U The synthetic image I after rotation around the corner point, scaling with the corner point as the anchor point, and sub-pixel shifting O ,like Figure 6 as shown in (b).

[0044] like Figure 1 As shown, the exact position of the corner points in the synthetic image is extracted through the following steps:

[0045] S10: Obtain a rough estimated position of each corner point from the chessboard calibration object image;

[0046] Since the offset of the corner point relative to the center of the image in the synthetic image used in this embodiment is known, and the offset in the x and y directions does not exceed 1 pixel, the original position of the corner point, that is, the center position of the synthetic image, is directly used as the rough estimated position of the corner point. The subscript R represents the rough position of the corner point.

[0047] S21: Image supersampling. Take the supersampling magnification n = 2, and perform the synthesis of the image I O Super sampling is performed, and bilinear interpolation is used as the image interpolation method. The synthetic image after super sampling is recorded as I S , size is 48×48.

[0048] S22: The rough position coordinates of the corner points are transformed to the super-sampled coordinate system. O The rough position of the mid-corner point is located at the center of the image, and the corresponding super-sampled synthetic image I S The mid-corner point is roughly located at the center of the image, so The rough position of the corner point after supersampling can also be calculated using the formula: n=2 The same results can be obtained.

[0049] S23: Using the preset image window, capture each corner point to obtain the corresponding original corner point image. S The rough position of the midpoint is The nearest integer pixel position to its lower right is The subscript BR indicates the lower right corner. S The size is 48×48, which is already small, so we will use image I SThe whole is used as an image window for subsequent processing, that is, the window size is 2A×2A=48×48, the half length of the image window is A=48 / 2=24; the x and y ranges of the window are and

[0050] S24: Calculate the response value of the central area of ​​the window and generate a response graph, such as Figure 4 As shown. The half side length of the image window is A = 48 / 2 = 24. The method for calculating the response value of a pixel position is to draw 8 directions of equal angle line segments centered on the pixel position and integrate the number of directions n D =8, if vertical downward is 0° and horizontal rightward is 90°, then the integral direction is That is, 0°, 22.5°, ..., 157.5°, the length of the line segment is 2l B +1 = 25 pixels, i.e. l B =12. Figure 3 As shown, line integrals are performed along 8 line segments. Specifically, 25 sampling points are taken on the line segments, the central sampling point is the center of the pixel position, and l sampling points are taken on both sides of the line segment separated by the pixel position. B = 12 sampling points, and the distance between any two adjacent sampling points is 1 pixel. The average value of the pixel values ​​at all sampling points on a line segment is taken as the line integral value in that direction. If the sampling point is located at a sub-pixel position, its pixel value is obtained by bilinear interpolation. The variance of the line integral results in each direction is taken as the response value of the pixel position. In order to avoid the integration range exceeding the image window, only the response value of the pixels in the central square area of ​​the window is calculated. The half side length of the central area is B = Al B = 12. Therefore, the size of the generated response map is 2B×2B=24×24.

[0051] like Figure 2 As shown in (a), due to the pattern of the checkerboard calibration object, the straight line passing through the corner point is always completely located in the white area or the black area of ​​the checkerboard, so the variance of each line integral result is large; while the straight line passing through a point next to the corner point always has part of it located in the white area and the other part in the black area, as shown in Figure 2 As shown in (b), the variance of each line integral result is small. Therefore, the response value is the largest at the exact position of the corner point, and the local maximum position of the response graph can be used as the exact position of the corner point.

[0052] S31: Gaussian blur the central area of ​​the response map. Use a size of (2l G +1)×(2l G +1)=9×9, standard deviation in x and y directions σ x =σ y= 9 Gaussian kernel, Gaussian blur the central area of ​​the response graph to make it smooth. The subscript G represents Gaussian blur. The half-side length of the Gaussian kernel is l G =(9-1) / 2=4. In order to prevent the Gaussian kernel from exceeding the range of the response map, only the central square area of ​​the response map is Gaussian blurred, and the half side length of the central area is C=Bl G = 8. The response map of a single corner point and the response map after blurring are as follows: Figure 5 (a) and Figure 5 as shown in (b).

[0053] S32: Fit the area near the corner point in the response map to an elliptical parabola. Take a sampling window of size 5×5 with the pixel with the largest pixel value in the blurred response map as the center, and fit the light spot to the general equation z=ax according to the pixel position (x, y coordinates, expressed in the coordinate system of the blurred response map) and the pixel value (z coordinate). 2 +bxy+cy 2 +dx+ey+f represents an elliptical paraboloid, and the parameters a, b, c, d, e, and f in the equation are estimated using the least squares method. For example, the rotation angle is 30°, and the image scaling factors in the x and y directions are s respectively. x =0.5,s y =1, and the offsets in the x and y directions are 0.20 and 0.40 pixels respectively, the coordinates and pixel values ​​of the pixels in the sampling window are shown in Table 1:

[0054] Table 1 shows the coordinates and pixel values ​​of the pixels in the sampling window

[0055]

[0056] The parameters of the elliptical parabola obtained by least squares fitting are shown in Table 2:

[0057] Table 2 is the parameter table of elliptical parabola

[0058] a b c d e f -60.09 15.20 -38.84 821.21 524.19 -2446.89

[0059] S33: Calculate the exact position of the corner point according to the elliptical parabola parameters. After obtaining the elliptical parabola equation parameters, the projection point of its maximum value point in the image plane can be calculated as:

[0060]

[0061] The superscript M represents the image coordinate system of the blurred response image, and the subscript P represents the exact position of the corner point. This point is the exact position of the corner point in the image coordinate system of the blurred response image estimated by fitting the elliptical parabola.

[0062] Will Transform back to the super-sampled image coordinate system to obtain the exact position of the corner point represented in this coordinate system The transformation relationship is:

[0063]

[0064] S34: The accurate position coordinates of the corner points are transformed back to the original image coordinate system. According to the equation:

[0065]

[0066] The corner point position is transformed from the super-sampled image coordinate system to the synthetic image coordinate system as the algorithm input, and the corner point is obtained in the synthetic image I O The exact location in Since in generating the synthetic image I O When the x and y direction offsets are 0.20 and 0.40 pixels respectively, the original coordinates of the corner point are Therefore, the composite image I O The true value of the midpoint position should be The subscript GT represents the true value. So the extraction error (pixel) of the corner point position is:

[0067]

[0068] In order to verify the effectiveness of the algorithm, a variety of parameter combinations are used to generate synthetic images I O , the true values ​​of the sub-pixel positions of the corner points are known. The OpenCV image gradient-based method and the method proposed in the present invention are used to extract the exact positions of the corner points from the image, and the errors of the two are compared.

[0069] In order to simulate the corner points in the camera image of the checkerboard calibration object, and considering the symmetry, the synthetic image I of the corner points is generated. O The parameters used are: the rotation angle range is 0-90°, the rotation angle step is 5°; the scaling factor in the x direction is 1 times, and the scaling factor in the y direction is 0.5 times; the x-direction offset range of the corner point is 0.00-0.50 pixels, the y-direction offset range is 0.00-1.00 pixels, and the x- and y-direction offset steps are both 0.01 pixels. Therefore, there are 19×51×101=97869 parameter combinations, that is, the total number of synthetic images used is 97869. According to the above steps, the errors of the method of the present invention and the corner point position extraction method of OpenCV are calculated and compared, as shown in Table 3:

[0070] Table 3 shows the error results of the method of the present invention and the method of OpenCV

[0071]

[0072]

[0073] It can be seen from Table 3 that the maximum error and average error of the corner point position obtained by the method of the present invention are both smaller than the corner point position extraction function cornerSubPix based on image gradient of OpenCV.

[0074] Example 2

[0075] This embodiment uses a camera to take a photo of the actual object of the checkerboard calibration object, uses the corner point detection method of OpenCV to obtain the rough position of the corner point in the image, and then uses the method proposed in the present invention to extract the exact position of the corner point (image point), and combines the known position of the corner point in the world coordinate system (object point) to calibrate the camera to obtain its internal and external parameters and distortion parameters. According to these parameters, the object point can be reprojected into the image coordinate system. The difference between the position of the reprojected point and the previously extracted image point is called the reprojection error, which can reflect the accuracy of the corner point extraction result. This embodiment compares the accuracy of the method of the present invention and the corner point position extraction function cornerSubPix based on image gradient of OpenCV by calculating the reprojection error.

[0076] In order to obtain more accurate camera calibration results, when taking a physical image of the checkerboard calibration object, the calibration object should fill the image as much as possible. Although it is allowed that there are large areas in some images that are not covered by the calibration pattern, when considering all images, the calibration pattern should cover the entire area of ​​the image, especially the edges and corners of the image. The pose of the calibration object in different images should be sufficiently varied, but extreme shooting angles should not be used or the distance between the calibration plate and the camera should not be greatly changed.

[0077] In this embodiment, a fixed-focus network camera is used to capture a checkerboard calibration object image with a grid number of 10×7 and an image size of 1280×720.

[0078] The meanings of the superscripts and subscripts of the variables in the calculation process of this embodiment are similar to those in Embodiment 1.

[0079] like Figure 1 As shown, the exact position of the corner points in the camera image is extracted through the following steps:

[0080] S10: Obtaining the rough position of the corner points. O Establish the image coordinate system C O . Use OpenCV's findChessboardCorners function to detect the original image I O The checkerboard calibration object in the image is used to obtain the rough position of the corner points in the image. OThere are multiple corner points in the original image. For the sake of convenience, we will take one of the corner points as an example. Although the specific coordinates of the other corner points are different, the methods used to extract the positions of the corner points are the same. The rough position of this corner point in the original image coordinate system is

[0081] S21: Image supersampling. Take the supersampling multiple as n = 2, and O Supersampling is performed and bilinear interpolation is used as the image interpolation method.

[0082] S22: The rough position coordinates of the corner points are transformed to the super-sampled coordinate system. The coordinates of the rough position of the corner points in the super-sampled image Its coordinates in the original calibration image The relationship is:

[0083]

[0084] S23: Cut the image window near the rough position of the corner point. S Located in The nearest integer pixel coordinate at the bottom right is The size of the window is 48×48 pixels, that is, the half length of the window is A=24, and the x and y ranges of the window are and

[0085] S24: Calculate the response value of the central area of ​​the window and generate a response map. The method for calculating the response value of a pixel is to draw equal-angle line segments in multiple directions centered on the pixel, and integrate the number of directions n. D =8, if vertical downward is 0° and horizontal rightward is 90°, then the integral direction is That is, 0°, 22.5°, ..., 157.5°, the line segment length is 25 pixels, and the half-length of the line integral is l B =(25-1) / 2=12. Perform line integration along the 8 line segments respectively. Specifically, take 25 sampling points with a spacing of 1 pixel on the line segment. The central sampling point is the center of the pixel, and there are 12 sampling points on both sides of the pixel center on the line segment. The average value of the pixel values ​​at all sampling points on a line segment is taken as the line integral value. If the sampling point is located at a sub-pixel position, its pixel value is obtained by bilinear interpolation. The variance of each line integral result is taken as the response value of the pixel. In order to avoid the integration range exceeding the image window, only the response values ​​of the pixels in the central square area of ​​the window are calculated. The half-side length of the central area is B=Al B = 12. Therefore, the size of the generated response map is 2B×2B=24×24.

[0086] S31: Gaussian blur the central area of ​​the response map. Use a size of (2l G +1)×(2l G +1)=9×9, the standard deviation in the x and y directions is σ x =σ y = 9 Gaussian kernel, Gaussian blur the central area of ​​the response graph to make it smooth. The half-side length of the Gaussian kernel is l G =4. In order to prevent the Gaussian kernel from exceeding the range of the response map, only the central square area of ​​the response map is Gaussian blurred, and the half side length of the central area is C=Bl G =8.

[0087] S32: Fit the area near the corner point in the response map to an elliptical parabola. Establish the image coordinate system C in the blurred response map M . Take the pixel with the largest pixel value in the blurred response map as the center and take a sampling window of size 7×7. According to the position of the pixel (x, y coordinates, in C M The light spot is fitted as follows: 2 +bxy+cy 2 +dx+ey+f represents an elliptic paraboloid, where the parameters a, b, c, d, e, and f are estimated using the least squares method.

[0088] S33: Calculate the exact position of the corner point according to the elliptical parabola parameters. After obtaining the elliptical parabola equation parameters, the projection point of its maximum value point in the image plane can be calculated as:

[0089]

[0090] This point is the position of the local maximum value of the response graph expressed in the blurred response graph coordinate system, which is estimated by fitting the elliptical parabola, and is taken as the accurate position of the corner point in the coordinate system.

[0091] S34: Convert the exact position of the corner point back to the original image coordinate system. First, convert the exact position of the corner point in the blurred response image coordinate system to the image I S The coordinate system C S In, it is recorded as The conversion relationship is:

[0092]

[0093] Secondly, transform it to the original image coordinate system C O In, it is recorded as The conversion relationship is:

[0094]

[0095] Through the above steps, the exact position (image point) of the corner point can be extracted from the calibration plate image taken by a set of cameras. Combined with the world coordinates (object point) of the corner point in the known calibration plate, the camera can be calibrated to obtain the internal and external parameters and distortion parameters of the camera. When modeling camera distortion, the parameters included include 6 mirror distortion parameters k1, ..., k6, 2 tangential distortion parameters p1, p2 and 4 thin prism parameters s1, ..., s4. Using these parameters, the object point can be reprojected into the image plane. The difference between the position of the reprojected point and the previously extracted image point is called the reprojection error, which can reflect the accuracy of the corner point extraction result. This embodiment compares the accuracy of the method of the present invention and the corner point position extraction function cornerSubPix based on image gradient of OpenCV by calculating the reprojection error.

[0096] Two groups of camera calibration experiments are performed using two groups of calibration plate images taken by the camera. The rough positions of the corners in the image are obtained by the findChessboardCorners function of OpenCV, and the accurate positions of the corners are obtained using the method described in this embodiment and the image gradient-based corner position extraction function cornerSubPix of OpenCV. After the calibration is completed, the sum of the corner reprojection errors in each image in each group of calibration experiments is calculated. The accuracy of the cornerSubPix function is related to the window size search size, and the total reprojection error is minimized by adjusting the parameter value. The reprojection errors of each image in the two groups of calibration experiments are shown in Figure 2. Figure 7 (a) and Figure 7 As shown in (b), when the method described in this embodiment is used, the sum of the reprojection errors of each image in each calibration experiment is smaller than the result when the corner point position extraction function cornerSubPix based on image gradient of OpenCV is used. Therefore, the total reprojection error of each calibration experiment is also smaller, which proves that the corner point extraction method described in the embodiment of the present invention has higher accuracy.

[0097] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for extracting corner points in a checkerboard calibration object image, characterized in that: include: S10: Obtain a rough estimated position of each corner point from the chessboard calibration object image; S20: After supersampling the checkerboard calibration object image, a supersampled image is obtained, the rough estimated position of the corner point obtained in S10 is converted into the image coordinate system after supersampling, and then the corresponding original corner point image is intercepted at each corner point using a preset image window, thereby obtaining the original corner point image corresponding to each corner point; S30: generating a response map corresponding to each original corner point image, and then extracting the exact position of the corner point according to the response map; S40: Repeat S30 to extract the corner point positions of the remaining original corner point images, so as to obtain the accurate positions of all corner points.

2. The method for extracting corner points in a checkerboard calibration object image according to claim 1, characterized in that: In S20, the preset image window is a square image window, and the distances between the center of the window and the rough position of the corner point in the x and y directions are not more than 0.5 pixels respectively.

3. The method for extracting corner points in a checkerboard calibration object image according to claim 1, characterized in that: In S30, generating a response map corresponding to each original corner point image includes: For each pixel position in the central area of ​​the original corner point image, the response value of the pixel position is calculated according to the multi-directional line integral result of each pixel position, and all pixel positions in the central area are traversed to calculate the response value of each pixel position, thereby generating a response map.

4. The method for extracting corner points in a checkerboard calibration object image according to claim 1, characterized in that: In S30, the size of the original corner image 2A×2A, the size of the response map 2B×2B and the scale of the line integral satisfy: A=B+l B , 2l B +1=L, A is half the length / width of the original corner image, B is half the length / width of the response image, l B is the half-scale of the line integral, and L is the total length of the line integral.

5. The method for extracting corner points in a checkerboard calibration object image according to claim 3, characterized in that: The step of calculating the response value of each pixel position according to the multi-directional line integral result of the pixel position includes: A circle with a radius of l is formed with each pixel position as the center. B The integrating circle of D The diameter is obtained and the integral circle is divided into equal parts. The direction of each diameter is recorded as a direction. The line integral result of the direction is obtained after line integration of the sampling points on each diameter. After traversing and calculating each diameter, the line integral results of multiple directions are obtained, and the variance of the line integral results of multiple directions is used as the response value of the current pixel position.

6. The method for extracting corner points in a checkerboard calibration object image according to claim 1, characterized in that: In S30, extracting the exact position of the corner point according to the response graph includes: First, the current response map is Gaussian blurred to obtain the blurred response map. Then, the central bright spot of the blurred response map is fitted as an elliptical parabola represented by a general equation. The sub-pixel extreme position of the bright spot is calculated according to the elliptical parabola equation obtained by fitting. Finally, the sub-pixel extreme position of the bright spot is converted from the image coordinate system of the blurred response map to the image coordinate system of the oversampled image, and then converted to the image coordinate system of the checkerboard calibration object image to obtain the accurate position of the corresponding corner point of the response map.

7. The method for extracting corner points in a checkerboard calibration object image according to claim 6, characterized in that: The step of performing Gaussian blurring on the current response graph to obtain a blurred response graph includes: Use size (2l G +1,2l G +1) is used to perform Gaussian blur on the response image of size 2B×2B to obtain a blurred response image of size 2C×2C. The image center of the blurred response image coincides with the image center of the response image. The size of the response image, the size of the blurred response image and the size of the Gaussian kernel satisfy: B=C+l G , B is half the length / width of the response map, C is half the length / width of the blurred response map, l G is the half-side length of the Gaussian kernel.

8. The method for extracting corner points in a checkerboard calibration object image according to claim 6, characterized in that: The step of calculating the sub-pixel extreme position of the bright spot according to the elliptical parabola equation obtained by fitting includes: The projection point of the maximum value of the parabola in the blurred response image plane is calculated according to the elliptical parabola equation obtained by fitting and is used as the sub-pixel extreme value position of the bright spot.

9. The method for extracting corner points in a checkerboard calibration object image according to claim 1, characterized in that: In S20, the supersampling is implemented by interpolation.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for extracting corner points in a chessboard calibration object image as described in any one of claims 1 to 9 is implemented.

Images