A method for extracting corner points in a chessboard calibration object image

Abstract

The application discloses a kind of extraction methods of corner point in checkerboard calibration object image.The application first obtains the rough position of checkerboard calibration object corner point;Then the rough position of corner point is converted to the image coordinate system of supersampling and the image window is intercepted at the rough position of corner point;Then the response value of window center area pixel is calculated, and the response graph is generated;Response graph is blurred to make it smooth, and the area near the corner point in the blurred response graph is fitted as elliptic paraboloid;Then the subpixel extreme position of bright spot is calculated according to the elliptic paraboloid parameter;Finally, the accurate position of corner point is obtained after coordinate conversion.Compared with the corner point boundary line-based checkerboard calibration object corner point position extraction method, the method proposed by the application can achieve higher corner point position extraction accuracy, thereby improving the accuracy of camera calibration result, and is beneficial to the accurate implementation of computer vision application.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of machine vision, and particularly relates to a method for accurately extracting corner points in a checkerboard calibration object image. BACKGROUND

[0002] Camera calibration is a process of determining the geometric relationship between the world coordinate system and the image coordinate system by calculating camera parameters. It is a necessary step for many three-dimensional vision applications and has a key influence on the implementation effect of vision applications. Camera calibration usually needs the help of a calibration object containing a set of key points arranged in a known geometry. First, multiple view images of the calibration object are shot, and then the known positions of the key points in the world coordinate system (object points) and the positions of the key points in the image coordinate system (image points) obtained by extraction are used to solve the camera parameters.

[0003] Among various planar calibration objects, the checkerboard calibration object naturally contains the key point of the checkerboard corner, and has a simple structure and is easy to manufacture with high precision, so it is widely used. The method for extracting the position of the corner point from the checkerboard calibration object image based on image processing is usually carried out in three steps. First, use a corner point or feature detector to obtain all possible corner point positions from the image; second, filter the corner points formed by the intersection of the division lines of the checkerboard from the detection results of the previous step, which can be achieved by using the geometric rules of the checkerboard pattern; and third, further optimize the position of the checkerboard corner point to improve the accuracy of the position of the corner point.

[0004] For the last step of the above process, it can be achieved by various different methods. The method uses the characteristics that the checkerboard corner point is located at the intersection of the division lines of the black and white squares, and uses the gradient at the division line of the checkerboard to extract the accurate position of the corner point. This method requires that the division lines of the black and white squares in the image be clear and accurate, and has poor robustness to image noise. In actual application, especially under non-ideal shooting conditions such as outdoor calibration, the accuracy is low. SUMMARY

[0005] In order to solve the problems and needs in the background art, the present application provides an accurate extraction method of corner points in a checkerboard calibration object image. The present application is based on a centerline model of a corner point, and focuses on straight lines in each direction passing through the corner point, rather than only considering the boundary lines of black and white squares. The present application uses a response value to estimate the distance between a point near the corner point and the accurate position of the corner point, and the greater the response value, the closer the point is to the accurate position of the corner point. The present application first obtains the rough position of the checkerboard corner point, then calculates the response value of all pixels in a square window near the position, and forms a response map. According to the relationship between the response value and the distance from the corner point position, the sub-pixel brightness maximum position of the response map is the accurate position of the corner point. The present application estimates the sub-pixel brightness maximum position of the response map by fitting the response map into an elliptic paraboloid, and realizes the extraction of the sub-pixel position of the corner point.

[0006] The technical scheme of the present application is as follows:

[0007] One kind of extraction method of corner points in a checkerboard calibration object image

[0008] S10: A checkerboard calibration object with black and white squares arranged alternately is used, the number of squares in each row and each column is known, so the number and arrangement of internal corner points of the checkerboard calibration object are known. After the camera captures the checkerboard calibration object, a checkerboard calibration object image is obtained, which should contain all the internal corner points of the checkerboard calibration object. The rough estimated position of each corner point is obtained from the checkerboard calibration object image using an existing method;

[0009] S20: After super-sampling the checkerboard calibration object image, a super-sampled image is obtained, the rough estimated position of the corner point obtained in S10 is converted to the image coordinate system after super-sampling, and then a pre-set image window is used to capture the corresponding original corner point image at each corner point, so as to obtain the original corner point image corresponding to each corner point; wherein the pre-set image window is a square image window, and the distance between the center of the window and the x, y directions of the rough position of the corner point is not more than 0.5 pixels. Super-sampling is realized by interpolation, for example, bilinear interpolation.

[0010] S30: A response map corresponding to each original corner point image is generated, and then the accurate position of the corner point is extracted according to the response map;

[0011] In S30, the response map corresponding to each original corner point image is generated, including:

[0012] For each pixel position in the central region of the original corner point image, the response value of each pixel position is calculated according to the multi-direction line integral result of each pixel position, all pixel positions in the central region are traversed, and the response value of each pixel position is calculated to generate a response map, and the image center of the response map coincides with the image center of the original corner point image.

[0013] The size 2A x 2A of the original corner point image, the size 2B x 2B of the response map and the scale of the line integral satisfy: A = B + l B , 2l B + 1 = L, A is half of the length / width of the original corner point image, B is half of the length / width of the response map, l B is half of the scale of the line integral, and L is the total length of the line integral, A, B, l B are all positive integers.

[0014] The response value of each pixel position is calculated according to the multi-direction line integral result of the pixel position, comprising:

[0015] An integral circle with a radius of l B is formed with each pixel position as the center, n D strips of diameters are selected and the integral circle is equally divided, the direction of each diameter is recorded as a direction, the line integral result of each diameter is obtained by performing line integral on the sampling points on each diameter, and the line integral result of the multi-direction is obtained after traversing calculation of each diameter, and the variance of the line integral result of the multi-direction is taken as the response value of the current pixel position. The specific calculation method of line integral is that (2l B + 1) equally spaced sampling points are taken on the above line segment, the central sampling point is located at the current pixel position, the line segment includes l B sampling points on both sides of the current pixel position, the distance between any adjacent sampling points is 1 pixel, and the sum of the pixel values at all sampling points is calculated and averaged as the direction integral value.

[0016] If the above sampling point is located at a sub-pixel position, the pixel value is obtained by bilinear interpolation.

[0017] In S30, the accurate position of the corner point is extracted according to the response map, comprising:

[0018] First, the current response map is blurred to obtain a blurred response map, then the central bright spot of the blurred response map is fitted as an elliptic paraboloid represented by a general equation, then the sub-pixel extreme position of the bright spot is calculated according to the elliptic paraboloid equation obtained by fitting, and finally the sub-pixel extreme position of the bright spot is converted from the image coordinate system of the blurred response map to the image coordinate system of the super-sampled image, and then to the image coordinate system of the checkerboard calibration object image, to obtain the accurate position of the corner point corresponding to the response map.

[0019] The current response map is blurred to obtain a blurred response map, comprising:

[0020] A integral circle with a radius of 2l G + 1, 2l G+1) is Gaussian blurred on the response map of size 2B×2B to obtain a blurred response map of size 2C×2C. The image center of the blurred response map coincides with the image center of the response map. The size of the response map, the size of the blurred response map and the size of the Gaussian kernel satisfy the following relationship: B=C+l G , B is half the length / width of the response map, C is half the length / width of the blurred response map, l G is the half side length of the Gaussian kernel, B, C, l G are all positive integers, and the standard deviation of the Gaussian kernel in both directions is equal, that is, σ x =σ y .

[0021] The sub-pixel extreme position of the bright spot is calculated based on the fitted elliptical paraboloid equation, including:

[0022] According to the elliptical parabola equation obtained by fitting, the projection point of the parabola maximum point in the image plane is calculated and used as the sub-pixel extreme position of the bright spot, that is, the exact position of the corner point in the response map.

[0023]

[0024] Among them, the superscript M represents the blurred response map, and the subscript P represents the exact position of the corner point.

[0025] The exact position of the corner point in the blurred response image Transform back to the super-sampled image coordinate system to obtain the exact position of the corner point in the super-sampled image coordinate system Wherein, the superscript S represents the supersampled image of the checkerboard calibration object;

[0026] The exact position of the corner points expressed in the supersampled image coordinate system Transform back to the original image coordinate system to get the exact position of the corner points in the original image The superscript O represents the original image of the checkerboard calibration object.

[0027] S40: Repeat S30 to accurately extract the corner point positions of the remaining original corner point images, thereby obtaining the accurate positions of all corner points.

[0028] 2. A storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the method for extracting corner points in a checkerboard calibration object image is implemented.

[0029] The beneficial effects of the present invention are:

[0030] The present application improves the accuracy of camera calibration process by extracting high-precision chessboard corner points, and ultimately improves the implementation effect of computer vision applications. In the embodiment, through the synthetic image test and camera calibration experiment, the error of the corner point position extraction function cornerSubPix based on image gradient of OpenCV and the method of the present application is compared, which proves that the method of the present application has higher accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is the overall flowchart of the method of the present application;

[0032] Figure 2 is a schematic diagram of the relationship between a straight line passing through a corner point and the black and white areas of the chessboard calibration object, wherein (a) is a straight line passing through a corner point only through the black or white area, (b) is a straight line passing through a point outside the corner point, part of which is located in the black area and part of which is located in the white area.

[0033] Figure 3 is the selection method of the direction integral sampling point when calculating the pixel response value, taking the case of 8 as the total number of integral directions and 25 as the number of sampling points in each direction as an example;

[0034] Figure 4 is a schematic diagram of the range of the image window, the response map, the blurred response map and the relationship between the parameters when extracting the accurate position of a single corner point;

[0035] Figure 5 is the response map of a single corner point and the blurred response map, wherein (a) is the response map of a single corner point, (b) is the blurred response map.

[0036] Figure 6 is a schematic diagram of the corner point synthetic image in embodiment 1, wherein (a) is the initial corner point image, (b) is the corner point image after applying the transformation;

[0037] Figure 7 is the re-projection error of two sets of calibration experiments in embodiment 2, wherein (a) is the re-projection error of the first set of calibration experiments, (b) is the re-projection error of the second set of calibration experiments. DETAILED DESCRIPTION

[0038] The application provides an accurate extraction method of corner points in a checkerboard calibration object image. In order to describe the implementation process and effectiveness of the method in detail, two embodiments will be described below. In the embodiments, the first embodiment describes how to extract the corner point position of a synthetic image using the method of the application, and compares the error with the corner point position extraction function cornerSubPix based on image gradient of OpenCV; the second embodiment describes how to extract the corner point position of a checkerboard calibration object image taken by a camera using the method of the application, and then uses the position information to calibrate the camera, and then calculates the re-projection error, and compares the re-projection error with that when the corner point position extraction function cornerSubPix based on image gradient of OpenCV is used.

[0039] Embodiment 1

[0040] In this embodiment, a synthetic image I U of a single corner point of a checkerboard calibration object with a size of 24x24 is obtained by rotating the image around the corner point and scaling the image with the corner point as an anchor point to obtain an intermediate synthetic image I RS after rotation and scaling. RS Then, sub-pixel offset is performed on the intermediate synthetic image I O after rotation and scaling along the horizontal and vertical directions to obtain a synthetic image I x similar to the corner point image of the checkerboard calibration object taken by the camera. Then, the accurate position of the corner point is extracted using the method proposed in the application. The advantage is that the true value of the position of the corner point in the synthetic image is known, so that the position extraction error of the corner point can be calculated to verify the accuracy of the algorithm.

[0041] In the following description, each image coordinate system is defined as follows: the center of the upper left pixel of the image is taken as the origin of the image coordinate system, the vertical downward direction is taken as the positive direction of the x-axis, and the horizontal right direction is taken as the positive direction of the y-axis. The following symbols are used to describe the transformation of the image: let the image be rotated counterclockwise around the center of the corner point by an angle of θ; let the scaling factor of the image along the x-direction with the corner point as an anchor point be s x , and the scaling factor along the y-direction be s y ; let the sub-pixel offset of the image be

[0042] The generation of the synthetic image I O is performed in two steps. First, rotation around the corner point and scaling with the corner point as an anchor point are performed on the synthetic image I U to obtain an intermediate synthetic image I RS after rotation and scaling. The superscript RS indicates the intermediate synthetic image after rotation and scaling. The relationship between a point P U in the image I RS and its transformed position P x is as follows: P y =S(s U ,sy ) • rot(θ) • (P - d) + d, where is a scaling matrix, is a rotation matrix, and d is a vector from the origin of the coordinate system of image I U to the corner point, i.e. the image center. Generating image I RS is actually according to the integer pixel position P RS in image I U , which corresponds to the sub-pixel position P U in image I U , with the relationship: For example, when the size of image I RS is 24 x 24, the rotation angle is 30°, and the scaling coefficients in x and y directions are s x = 1 and s y = 0.5, respectively, the pixel value of integer position P RS in image I RS is the same as that of the corresponding sub-pixel position P U in image I U , with the relationship: The pixel value at position P U in image I U is obtained by bilinear interpolation, as the pixel value of image I RS at position P RS . The above operation is performed for each pixel in image I RS , thus generating intermediate composite image I RS .

[0043] Secondly, image I RS is sub-pixel shifted in x and y directions, with the shift amount in the range of 0.00-1.00 pixel, thus obtaining composite image I O . The relationship between position P O in image I O and the corresponding position P RS in image I RS is P RS = P O - t, where t is a vector representing the sub-pixel shift. For example, when the shift amounts in x and y directions are 0.20 and 0.40 pixel, respectively, the above equation is: For each integer pixel position P O in image I O , its corresponding sub-pixel position P RS in image I RS is calculated using the above relationship, and the gray value at this position is obtained by bilinear interpolation, as the pixel value of image I O at position P Opixel value of the pixel at the pixel, thereby generating a composite image I O . The composite image I U , without transformation Figure 6 (a) of FIG. 1 U , after rotation around the corner point, scaling with the corner point as the anchor point, and sub-pixel offset O (b) of FIG. 1 Figure 6 .

[0044] As shown in FIG. 1 Figure 1 , the accurate position of the corner point in the composite image is extracted by the following steps:

[0045] S10: Obtain a rough estimated position of each corner point from the checkerboard calibration object image.

[0046] Since the offset of the corner point in the composite image relative to the center of the image is known in this embodiment, and the offset in the x and y directions is not more than 1 pixel, the original position of the corner point, i.e., the center position of the composite image, is directly taken as the rough estimated position of the corner point wherein the subscript R represents the rough position of the corner point. For this embodiment

[0047] S21: Image super-sampling. Take a super-sampling magnification n = 2, super-sample the composite image I O , and use bilinear interpolation as the image interpolation method. Denote the super-sampled composite image as I S , and the size is 48 x 48.

[0048] S22: Transform the coordinates of the rough position of the corner point to the coordinate system after super-sampling. In the composite image I O , the rough position of the corner point is located at the center of the image. Correspondingly, in the super-sampled composite image I S , the rough position of the corner point is also located at the center of the image, so the rough position of the corner point after super-sampling can also be calculated using the formula, and n = 2 is brought into the same result can be obtained.

[0049] S23: Obtain the corresponding original corner point image by using a preset image window to cut at each corner point. The rough position of the corner point in the super-sampled image I S is the nearest integer pixel position located at the lower right of which is wherein the subscript BR represents the lower right corner. Since the size of the image I S is 48 x 48, which is already small, the image I SThe whole is processed as an image window, that is, the window size is 2A x 2A = 48 x 48, and the half side length of the image window is A = 48 / 2 = 24; the x and y ranges of the window are respectively With

[0050] S24: Calculate the response value of the center area of the window, and generate a response map as shown in Figure 4 The half side length of the image window is A = 48 / 2 = 24. The method for calculating the response value of a pixel position is to draw 8-directional equal-angle line segments with the pixel position as the center, and integrate the number of directions n D = 8. If the vertical downward direction is 0° and the horizontal right direction is 90°, the integration directions are 0°, 22.5°, …, 157.5°, and the lengths of the line segments are all 2l B +1 = 25 pixels, that is, l B = 12. As shown in Figure 3 , line integration is performed along the 8 line segments respectively. Specifically, 25 sampling points are taken on the line segment, the middle sampling point is the center of the pixel position, l B = 12 sampling points are taken on the two sides of the line segment divided by the pixel position, and the distance between any two adjacent sampling points is 1 pixel. The average value of the pixel values at all sampling points on a line segment is taken as the line integration value of the direction, and if the sampling point is at a sub-pixel position, the pixel value is obtained by bilinear interpolation. The variance of the line integration results of each direction is taken as the response value of the pixel position. In order to avoid the integral range exceeding the image window, only the response values of the pixels in the center square region of the window are calculated, and the half side length of the center region is B = A-l B = 12. Therefore, the size of the generated response map is 2B x 2B = 24 x 24.

[0051] As shown in (a) of Figure 2 , due to the regular pattern of the checkerboard calibration object, the straight line passing through the corner point is always completely located in the white or black area of the checkerboard, so the variance of each line integration result is large; and the straight line passing through a point next to the corner point is always partially located in the white area and partially located in the black area, as shown in (b) of Figure 2 , so the variance of each line integration result is small. Therefore, the response value at the accurate position of the corner point is the largest, and the local maximum position of the response map can be taken as the accurate position of the corner point.

[0052] S31: Gaussian blur is performed on the center region of the response map. A Gaussian blur filter with a size of (2l G +1) x (2l G +1) = 9 x 9 and an x, y direction standard deviation σ x = σ y= 9 Gaussian kernel, Gaussian blur the central area of ​​the response graph to make it smooth. Wherein, the subscript G represents Gaussian blur. The half side length of the Gaussian kernel is l G = (9-1) / 2 = 4. In order to prevent the Gaussian kernel from exceeding the range of the response map, only the central square area of ​​the response map is Gaussian blurred, and the half side length of the central area is C = Bl G = 8. The response map of a single corner point and the response map after blurring are as follows: Figure 5 (a) and Figure 5 As shown in (b).

[0053] S32: Fit the area near the corner point of the response map to an elliptical paraboloid. Take a sampling window of size 5×5 with the pixel with the largest pixel value in the blurred response map as the center, and fit the spot to the general equation z=ax according to the pixel position (x, y coordinates, expressed in the coordinate system of the blurred response map) and the pixel value (z coordinate). 2 +bxy+cy 2 +dx+ey+f represents an elliptical paraboloid, and the parameters a, b, c, d, e, and f in the equation are estimated using the least squares method. For example, if the rotation angle is 30°, the image scaling factors in the x and y directions are s respectively. x =0.5,s y =1, and the x- and y-direction offsets are 0.20 and 0.40 pixels respectively, the coordinates and pixel values ​​of the pixels in the sampling window are shown in Table 1:

[0054] Table 1 shows the coordinates and pixel values ​​of the pixels in the sampling window

[0055]

[0056] The parameters of the elliptical paraboloid obtained by least squares fitting are shown in Table 2:

[0057] Table 2 is the parameter table of elliptical paraboloid

[0058] a b c d e f -60.09 15.20 -38.84 821.21 524.19 -2446.89

[0059] S33: Calculate the exact position of the corner point based on the elliptical paraboloid parameters. After obtaining the elliptical paraboloid equation parameters, the projection point of its maximum point in the image plane can be calculated as:

[0060]

[0061] The superscript M represents the image coordinate system of the blurred response map, and the subscript P represents the exact position of the corner point. This point is the exact position of the corner point in the image coordinate system of the blurred response map estimated by fitting the elliptical paraboloid.

[0062] Will Transforming back to the super-sampled image coordinate system, the accurate position of the corner point in the coordinate system is obtained The transformation relationship is:

[0063]

[0064] S34: Transform the accurate position coordinate of the corner point back to the original image coordinate system. According to the equation:

[0065]

[0066] Transform the corner point position from the super-sampled image coordinate system to the composite image coordinate system as the algorithm input, and obtain the accurate position of the corner point in the composite image I O Since the x and y direction offset amounts are 0.20 and 0.40 pixels respectively when the composite image I O is generated, and the original coordinate of the corner point is , the true value of the corner point position in the composite image I O should be , wherein the subscript GT represents the true value. Therefore, the extraction error (pixels) of the corner point position at this time is:

[0067]

[0068] In order to verify the effectiveness of the algorithm, a plurality of parameter combinations are used to generate the composite image I O , and the true value of the sub-pixel position of the corner point is known. The accurate position of the corner point is extracted from the picture by using the OpenCV method based on image gradient and the method proposed in the application respectively, and the errors of the two are compared.

[0069] In order to simulate the corner points in the camera image of the chessboard calibration object, and considering the symmetry, when the composite image I O of the corner point is generated, the parameters used are: the range of the rotation angle is 0-90°, the rotation angle step is 5°; the scaling coefficient in the x direction is 1 times, and the scaling coefficient in the y direction is 0.5 times; the x direction offset amount of the corner point ranges from 0.00 to 0.50 pixels, the y direction offset amount ranges from 0.00 to 1.00 pixels, and the x and y direction offset amount steps are both 0.01 pixels. Therefore, the parameter combination is 19x51x101=97869, that is, the total number of composite images used is 97869. According to the above steps, the errors of the corner point position extraction method of the application and OpenCV are calculated, as shown in Table 3:

[0070] Table 3 is the error result of the method of the application and the method of OpenCV

[0071]

[0072] ​

[0073] As can be seen from Table 3, the maximum error and average error of the corner point position obtained by the method of the present invention are both smaller than the corner point position extraction function cornerSubPix based on image gradient of OpenCV.

[0074] Example 2

[0075] This embodiment uses a camera to take a photo of a checkerboard calibration object, uses OpenCV's corner detection method to obtain the rough position of the corner points in the image, and then uses the method proposed in this invention to extract the exact position of the corner points (image points). Combined with the known position of the corner points in the world coordinate system (object points), the camera is calibrated to obtain its internal and external parameters and distortion parameters. Based on these parameters, the object points can be reprojected into the image coordinate system. The difference between the position of the reprojected point and the previously extracted image point is called the reprojection error, which can reflect the accuracy of the corner point extraction result. This embodiment compares the accuracy of the method of the present invention with OpenCV's image gradient-based corner point position extraction function cornerSubPix by calculating the reprojection error.

[0076] To obtain more accurate camera calibration results, when capturing images of the physical checkerboard calibration object, ensure that the object fills the image as fully as possible. While some images may have large areas not covered by the calibration pattern, the calibration pattern should cover the entire image, especially the edges and corners, when considering all images. The pose of the calibration object should vary significantly between images, but extreme shooting angles or drastically changing the distance between the calibration board and the camera should not be used.

[0077] In this embodiment, a fixed-focus network camera is used to capture an image of a calibration object with a checkerboard grid having 10×7 grid squares, and the image size is 1280×720.

[0078] The meanings of the superscripts and subscripts of variables in the calculation process of this embodiment are similar to those in Example 1.

[0079] like Figure 1 As shown in the figure, the exact position of the corner points in the camera image is extracted through the following steps:

[0080] S10: Get the rough position of the corner points. O Establish the image coordinate system C O Use OpenCV's findChessboardCorners function to detect the original image I O The checkerboard calibration object in the image is used to obtain the rough position of the corner points in the image. OThe middle point contains multiple corner points, for the sake of description, one of the corner points is taken as an example, although the specific coordinates of other corner points are different, the method used in extracting the position of each corner point is the same. The rough position of this corner point in the original image coordinate system is recorded as

[0081] S21: Image super-sampling. Take the super-sampling factor n = 2, super-sample the original calibration object image I O , and use bilinear interpolation as the image interpolation method.

[0082] S22: Transform the corner point rough position coordinates to the super-sampled coordinate system. The coordinates of the corner point rough position in the super-sampled image are , and the relationship between the coordinates in the original calibration object image is:

[0083]

[0084] S23: Take the image window near the corner point rough position. Set the nearest integer pixel coordinates in the image I S below the right as , and take the window size as 48x48 pixels, that is, the half side length of the window is A = 24, and the x, y range of the window is and

[0085] S24: Calculate the response value of the window center area and generate the response map. The method for calculating the response value of a pixel is to draw multiple equal-angle lines with the pixel as the center, and the integral direction number n D = 8. If the vertical downward direction is 0° and the horizontal right direction is 90°, the integral directions are , that is, 0°, 22.5°, …, 157.5°, and the line segment length is 25 pixels, and the half length of the line integral is l B = (25-1) / 2 = 12. The line integral is performed along the 8 lines, and the specific method is to take 25 sampling points on the line segment with an interval of 1 pixel, and the central sampling point is the pixel center, and there are 12 sampling points on both sides of the pixel center on the line segment. The average value of the pixel values of all sampling points on a line segment is taken as the line integral value, and if the sampling point is located at a sub-pixel position, the pixel value is obtained by bilinear interpolation. The variance of each line integral result is taken as the response value of the pixel. In order to avoid the integral range exceeding the image window, only the response values of the pixels in the center square region of the window are calculated, and the half side length of the center region is B = A-l B = 12. Therefore, the size of the generated response map is 2Bx2B = 24x24. ​

[0086] S31: Gaussian blur is performed on the center region of the response map. A Gaussian kernel with size (2l G +1)×(2l G +1)=9×9 and standard deviation σ x =σ y =9 is used to smooth the center region of the response map. The half side length of the Gaussian kernel is l G =4. In order to avoid the Gaussian kernel exceeding the range of the response map, only the center square region of the response map is blurred, and the half side length of the center region is C=B-l G =8.

[0087] S32: The region around the corner point in the response map is fitted as an elliptic paraboloid. An image coordinate system C M is established in the blurred response map. A sampling window with size 7×7 is taken with the pixel with the maximum value in the blurred response map as the center, and according to the positions (x, y coordinates, expressed in C M ) and values (z coordinates) of the pixels in the window, the light spot is fitted as an elliptic paraboloid expressed by the general equation z=ax 2 +bxy+cy 2 +dx+ey+f, and the parameters a, b, c, d, e, and f in the equation are estimated using the least square method.

[0088] S33: The accurate position of the corner point is calculated according to the parameters of the elliptic paraboloid. After the parameters of the elliptic paraboloid equation are obtained, the projection point of the maximum value point of the elliptic paraboloid in the image plane is calculated as:

[0089]

[0090] The point is the local maximum value position of the response map estimated by fitting the elliptic paraboloid, expressed in the coordinate system of the blurred response map, and it is taken as the accurate position of the corner point in the coordinate system.

[0091] S34: The coordinate of the accurate position of the corner point is converted back to the coordinate system of the original image. First, the accurate position of the corner point in the coordinate system of the blurred response map is converted to the coordinate system C S of the image I S , denoted as The conversion relationship is:

[0092]

[0093] Second, it is converted to the coordinate system C O of the original image, denoted as The conversion relationship is:

[0094]

[0095] Through the above steps, the exact position of the corner point (image point) can be extracted from the calibration plate image taken by a set of cameras. Combined with the world coordinates (object point) of the corner point in the known calibration plate, the camera calibration can be performed to obtain the internal and external parameters and distortion parameters of the camera. When modeling camera distortion, the parameters included include 6 mirror distortion parameters k1, ..., k6, 2 tangential distortion parameters p1, p2 and 4 thin prism parameters s1, ..., s4. Using these parameters, the object point can be reprojected into the image plane. The difference between the position of the reprojected point and the previously extracted image point is called the reprojection error, which can reflect the accuracy of the corner point extraction result. This embodiment compares the accuracy of the method of the present invention and OpenCV's image gradient-based corner point position extraction function cornerSubPix by calculating the reprojection error.

[0096] Two sets of camera calibration experiments were performed using two sets of calibration plate images taken by the camera. The rough positions of the corners in the image are obtained by the findChessboardCorners function of OpenCV, and the accurate positions of the corners are obtained using the method described in this embodiment and the image gradient-based corner position extraction function cornerSubPix of OpenCV. After the calibration is completed, the sum of the corner reprojection errors in each image in each set of calibration experiments is calculated. The accuracy of the cornerSubPix function is related to the window size search size, and the total reprojection error is minimized by adjusting the parameter value. The reprojection errors of each image in the two sets of calibration experiments are shown as follows: Figure 7 (a) and Figure 7 As shown in (b), when the method described in this embodiment is used, the sum of the reprojection errors of each image in each calibration experiment is smaller than the result when OpenCV's image gradient-based corner position extraction function cornerSubPix is ​​used. Therefore, the total reprojection error in each calibration experiment is also smaller, demonstrating that the corner extraction method described in this embodiment of the present invention has higher accuracy.

[0097] Finally, it should be noted that the above embodiments and explanations are intended only to illustrate the technical solutions of the present invention and are not intended to limit the present invention. It should be understood by those skilled in the art that modifications or equivalent substitutions to the technical solutions of the present invention may be made without departing from the spirit and scope of the technical solutions disclosed herein, and all such modifications or equivalent substitutions shall be encompassed within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting corner points in a checkerboard calibration object image, characterized in that: include: S10: Obtain a rough estimated position of each corner point from the checkerboard calibration object image; S20: After supersampling the checkerboard calibration object image, a supersampled image is obtained, the rough estimated positions of the corner points obtained in S10 are converted to the supersampled image coordinate system, and then a preset image window is used to intercept each corner point to obtain its corresponding original corner point image, thereby obtaining the original corner point image corresponding to each corner point; S30: Generate a response map corresponding to each original corner point image, and then extract the exact position of the corner point according to the response map; In S30, generating a response map corresponding to each original corner image includes: For each pixel position in the central area of ​​the original corner point image, the response value of the pixel position is calculated according to the multi-directional line integral result of each pixel position, and all pixel positions in the central area are traversed to calculate the response value of each pixel position, thereby generating a response map; Calculating the response value of each pixel position according to the multi-directional line integral result of the pixel position includes: A circle with a radius of l is formed with each pixel position as the center B Integral circle, choose n D The diameter of each line is divided into two equal parts, and the direction of each diameter is recorded as a direction. The line integral result of the direction is obtained after the line integral of the sampling points on each diameter is performed. After traversing and calculating each diameter, the line integral results of multiple directions are obtained, and the variance of the line integral results of multiple directions is used as the response value of the current pixel position; In S30, extracting the exact position of the corner point according to the response graph includes: First, the current response map is Gaussian blurred to obtain the blurred response map. Then, the central bright spot of the blurred response map is fitted into an elliptical paraboloid represented by a general equation. Then, the sub-pixel extreme position of the bright spot is calculated according to the elliptical paraboloid equation obtained by fitting. Finally, the sub-pixel extreme position of the bright spot is converted from the image coordinate system of the blurred response map to the image coordinate system of the super-sampled image, and then to the image coordinate system of the checkerboard calibration object image to obtain the accurate position of the corresponding corner point of the response map. The calculating of the sub-pixel extreme position of the bright spot according to the elliptical parabola equation obtained by fitting includes: According to the elliptical parabola equation obtained by fitting, the projection point of the parabola maximum point in the blurred response image plane is calculated and used as the sub-pixel extreme position of the bright spot; S40: Repeat S30 to extract the corner point positions of the remaining original corner point images, thereby obtaining the accurate positions of all corner points.

2. The method for extracting corner points in a checkerboard calibration object image according to claim 1, characterized in that: In the above S20, the preset image window is a square image window, and the distances between the center of the window and the rough positions of the corner points in the x and y directions are no more than 0.5 pixels.

3. The method for extracting corner points in a checkerboard calibration object image according to claim 1, wherein: In the above S30, the size of the original corner image (2A×2A), the size of the response map (2B×2B), and the scale of the line integral satisfy: A=B+l B , 2l B +1=L, A is half of the length / width of the original corner image, B is half of the length / width of the response image, l B is the half-scale of the line integral, and L is the total length of the line integral.

4. The method for extracting corner points in a checkerboard calibration object image according to claim 1, wherein: The step of performing Gaussian blurring on the current response graph to obtain a blurred response graph includes: Use size (2l G +1,2l G +1) is Gaussian blurred on the response map of size 2B×2B to obtain a blurred response map of size 2C×2C. The image center of the blurred response map coincides with the image center of the response map. The size of the response map, the size of the blurred response map and the size of the Gaussian kernel satisfy the following relationship: B=C+l G , B is half the length / width of the response map, C is half the length / width of the blurred response map, l G is the half-side length of the Gaussian kernel.

5. The method for extracting corner points in a checkerboard calibration object image according to claim 1, wherein: In S20, the supersampling is implemented by interpolation.

6. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for extracting corner points in a checkerboard calibration object image as described in any one of claims 1 to 5 is implemented.

Images