A method, device and storage medium for automatic detection and sorting of chessboard corner points

Through the Geiger corner point detection template and the minimum convex hull arrangement principle, the corner points of the checkerboard grid are extracted and sorted, and the problem of corner point sorting depends on pose selection in the existing technology is solved, and high-precision and robust corner point detection are achieved. Suitable for environments where light changes and image rotation.

CN115131575BActive Publication Date: 2024-11-12NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202210665983.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-14
Publication Date
2024-11-12
Estimated Expiration
2042-06-14

AI Technical Summary

Technical Problem

The prior art relies more on image pose selection when automatically sorting the corners of the checkerboard during camera calibration, and it is easy to introduce pseudo-corner points, resulting in insufficient detection accuracy and robustness.

Method used

The Geiger corner point detection template is used to extract pixel-level candidate corner points, and the number of corner points is refined and subpixel positioned, and the four poles of the checkerboard grid are extracted using the principle of minimum convex hull arrangement, and the corner point sorting is completed based on the direction vector between the four poles.

Benefits of technology

The accuracy and robustness of corner point detection are improved, the occurrence of pseudo-corner points is reduced, and the adaptability to light changes and image rotation is enhanced. The detection accuracy is improved by 61.87%, and the impact on image brightness and rotation is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115131575B_ABST
    Figure CN115131575B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, device and storage medium for automatically detecting and sorting checkerboard corner points; the method comprises: obtaining a checkerboard grayscale image, extracting pixel-level candidate corner points in the grayscale image using a Geiger corner point detection template; refining the number of corner points of the candidate corner points, thereby filtering out pseudo corner points in the candidate corner points, and determining the candidate corner points retained after filtering as the checkerboard corner points; performing sub-pixel positioning on the checkerboard corner points to obtain sub-pixel corner points; extracting checkerboard quadrupoles from the sub-pixel corner points by the minimum convex hull arrangement principle; determining the starting point of the checkerboard according to the interval distance between the checkerboard quadrupoles; and completing the checkerboard corner point sorting based on the starting point of the checkerboard using the direction vectors between the checkerboard quadrupoles. The present invention can still detect the precise positions of all valid corner points under conditions such as illumination changes, image rotation and image distortion, thereby improving the accuracy of checkerboard corner point detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, a device and a storage medium for automatically detecting and sorting chessboard corner points. Background Art

[0002] At present, according to the degree of automation of the detection algorithm, the checkerboard corner detection method is divided into two categories: manual interaction method and automatic detection method; the former requires human-computer interaction to select four corner points outside the checkerboard in sequence to realize corner point detection and sorting. This method is more dependent on the operator's experience and cannot automatically extract corner points, which reduces the efficiency of camera calibration; the latter uses the gradient changes in the area around the corner points and the corner point detection operator to calculate the corner point coordinates. This method can automatically extract corner points and realize the automation of camera calibration, but it is easy to introduce pseudo corner points.

[0003] Commonly used checkerboard automatic corner detection templates include Smallest Univalue Segment Assimilating Nucleus (SUSAN) template, Harris template, Hessian template, etc. Among them, the SUSAN template uses the symmetry of the checkerboard and the circular template to extract corners, and has the advantages of high detection accuracy, fast speed, and rotation invariance, but there are still pseudo corners when distinguishing the edges and corners of the checkerboard, and it is also sensitive to noise; the Harris template is relatively simple and easy to implement, and can accurately detect corners under conditions such as image rotation and noise interference, but this method has low detection efficiency for overexposed images; in addition, based on the local characteristics of the checkerboard corners, the corner detection algorithm that uses the Hessian matrix to filter the image has high detection accuracy, simple principle, and is easy to implement, but the algorithm has the problem of failure to detect corners in local areas, and is more dependent on the choice of threshold.

[0004] The corner point extraction accuracy of the above template is high, but the order of the corner points is messy and cannot be used directly for camera calibration; therefore, how to improve the automation of the corner point detection and sorting algorithm is also the key to realizing the automation of camera calibration. Before sorting the corner points, the four poles of the chessboard, that is, the four vertices of the chessboard, are first identified from the detected sub-pixel corner points. Commonly used quadruple point recognition algorithms include the grid method and the coordinate extreme value method. Among them, the grid method requires the user to select four corner points on the image with the mouse, which is not conducive to the automatic sorting of corner points. When the image distortion is small, no matter how the chessboard rotates, the horizontal and vertical coordinates of the quadruple points are always between the horizontal and vertical extreme values ​​of the corner points; the coordinate extreme value method uses this feature to identify the quadruple points, but when there is a certain distortion in the image, the coordinate extreme value method is more dependent on the selection of the calibration plate posture to identify the chessboard quadruple points, and its robustness is poor. Currently, the open source computer vision library OpenCV provides a checkerboard corner point detection and sorting method. This method uses an adaptive threshold to process uneven brightness, and combines morphological processing and quadrilateral detection algorithms to extract corner points. The checkerboard corner point detection method based on OpenCV improves the detection efficiency, but the number of corner points in the checkerboard to be detected must be known in advance. When the image is blurred, distorted, or the calibration plate is tilted too much, the detection will fail. In addition, when the image brightness is too bright or the number of corner points increases, the corner point detection speed will be greatly reduced. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device and storage medium for automatic detection and sorting of checkerboard corner points, so as to solve the technical problem that the automatic sorting of checkerboard corner points in the prior art during the camera calibration process is more dependent on image pose selection.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a method for automatically detecting and sorting chessboard corner points, the method comprising:

[0008] Obtain a checkerboard grayscale image, and extract pixel-level candidate corner points in the grayscale image using a Geiger corner detection template;

[0009] Refining the number of the candidate corner points, thereby filtering out pseudo corner points in the candidate corner points, and determining the candidate corner points retained after filtering as the checkerboard corner points;

[0010] Performing sub-pixel positioning on the chessboard corner points to obtain sub-pixel corner points;

[0011] Extracting chessboard quadrupole points from the sub-pixel corner points by using the minimum convex hull arrangement principle;

[0012] Determine the starting point of the chessboard according to the interval distance between the four points of the chessboard;

[0013] Based on the starting point of the chessboard, the chessboard corner point sorting is completed using the direction vectors between the four poles of the chessboard.

[0014] In combination with the first aspect, preferably, the step of extracting pixel-level candidate corner points in the grayscale image using a Geiger corner detection template includes:

[0015] Performing convolution filtering on the grayscale image using a Geiger corner detection template to obtain a corner likelihood map;

[0016] The pseudo corner points in the corner likelihood map are removed by using a non-maximum suppression algorithm with an adaptive threshold, so as to extract pixel-level candidate corner points from the corner likelihood map after the pseudo corner points are removed.

[0017] In combination with the first aspect, preferably, the step of refining the number of the candidate corner points includes:

[0018] Calculate a 32-bit weighted gradient direction histogram for each candidate corner point region, wherein the histogram uses the gradient angle as the abscissa and the gradient value as the ordinate;

[0019] Using a mean shift algorithm to find the gradient angles corresponding to the two maximum peaks in the histogram;

[0020] The two gradient angles are used as the angles of two lines passing through the candidate corner point. If the difference between the two gradient angles is greater than a preset threshold, the candidate corner point is retained, otherwise the candidate point is eliminated.

[0021] In combination with the first aspect, preferably, the step of performing sub-pixel positioning on the checkerboard corner points includes:

[0022] According to the gradient characteristics of the corner points in the chessboard, the sub-pixel position of the corner points of the chessboard is calculated by formula (1):

[0023]

[0024] Where c is the calculated sub-pixel corner point, is the neighborhood of the sub-pixel corner point c and p is the neighborhood A point inside, g p is the gradient vector of point p, and T is the vector transpose sign.

[0025] In combination with the first aspect, preferably, the step of extracting checkerboard quadrupole points from the sub-pixel corner points by using the minimum convex hull arrangement principle includes:

[0026] According to the position relationship, search for the corner point located at the maximum value of the horizontal coordinate of the image as the initial corner point;

[0027] According to the convex hull arrangement principle, the minimum convex hull point is extracted from the initial corner point in the sub-pixel corner point in a clockwise direction;

[0028] The minimum convex hull points are traversed, and four points with the largest corners are extracted as the chessboard quadrupoles.

[0029] In combination with the first aspect, preferably, the step of using the direction vectors between the four points of the chessboard to complete the ordering of the chessboard corner points comprises the following steps:

[0030] The four points of the chessboard are named o1, o2, o3, and o4 in clockwise direction. o1 is the starting point of the chessboard. The corner points on the straight line o1o2 and the straight line o1o4 are respectively confirmed by formula (2);

[0031]

[0032] Wherein, au+bv+c=0 is the general equation of the straight line, a, b, c are the equation coefficients, and f is the control threshold; u and v represent the horizontal and vertical coordinate values ​​of the corner point to be confirmed relative to the starting point o1 of the chessboard, respectively. If the distance from the corner point to be confirmed to the straight line satisfies not greater than the control threshold f, then the corner point to be confirmed is a point on the straight line;

[0033] Based on the confirmed corner points on the straight line o1o2 and the straight line o1o4, the initial values ​​of the horizontal and vertical direction vectors between the corner points adjacent to the corner points on the straight line o1o2 and the straight line o1o4 are calculated by formula (3):

[0034]

[0035] In the formula, m is the number of corner points on the straight line o1o2, and n is the number of corner points on the straight line o1o4; is the horizontal direction vector, is the vertical direction vector;

[0036] Combine the initial values ​​of the horizontal and vertical vectors of adjacent corner points The adjacent virtual corner points C in the horizontal direction and vertical direction of the current adjacent corner point C0 are obtained by formula (4) x , C y :

[0037]

[0038] According to the adjacent virtual corner points C in the horizontal and vertical directions of the adjacent corner point C0 x , Cy , detect the point closest to the adjacent virtual corner point from the unsorted corner point set U, and update the current corner point and the direction vector of the current corner point in turn by formula (5) and formula (6) respectively until all corner points are sorted:

[0039]

[0040]

[0041] In the formula, C i is the point in the unsorted corner point set U that is closest to the virtual corner point, C x ′ and C y ′ are the updated current corner points in the horizontal and vertical directions of the chessboard, and It is the updated direction vector of the current corner point.

[0042] In a second aspect, the present invention provides a device for automatically detecting and sorting chessboard corner points, the device comprising:

[0043] A pixel-level candidate corner point extraction module is used to obtain a checkerboard grayscale image and extract pixel-level candidate corner points in the grayscale image using a Geiger corner point detection template;

[0044] A corner point number refinement module is used to refine the number of the candidate corner points, thereby filtering out pseudo corner points in the candidate corner points, and determining the candidate corner points retained after filtering as the checkerboard corner points;

[0045] A sub-pixel positioning module, used to obtain sub-pixel corner points after performing sub-pixel positioning on the chessboard corner points;

[0046] A chessboard quadrupole extraction module is used to extract chessboard quadrupoles from the sub-pixel corner points by using the minimum convex hull arrangement principle;

[0047] A chessboard starting point determination module is used to determine the starting point of the chessboard according to the interval distance between the four points of the chessboard;

[0048] The chessboard corner point sorting module is used to complete the chessboard corner point sorting based on the starting point of the chessboard and using the direction vectors between the four poles of the chessboard.

[0049] In combination with the second aspect, preferably, the module for extracting pixel-level candidate corner points includes:

[0050] A convolution filtering unit, configured to perform convolution filtering on the grayscale image using a Geiger corner detection template to obtain a corner likelihood map;

[0051] The pseudo corner point elimination unit is used to eliminate pseudo corner points in the corner point likelihood map by using a non-maximum suppression algorithm with an adaptive threshold, thereby extracting pixel-level candidate corner points from the corner point likelihood map after the pseudo corner points are eliminated.

[0052] In a third aspect, the present invention provides a device for automatically detecting and sorting chessboard corner points, including a processor and a storage medium;

[0053] The storage medium is used to store instructions;

[0054] The processor is used to operate according to the instructions to execute the steps of the method for automatic detection and sorting of checkerboard corner points as described in any one of the first aspects.

[0055] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for automatic detection and sorting of checkerboard corner points as described in any one of the first aspects are implemented.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The present invention combines a Geiger corner point detection template to extract pixel-level candidate corner points to complete sub-pixel extraction of chessboard corner points, and uses the minimum convex hull principle to extract chessboard quadrupoles and a starting point, and finally completes corner point sorting through direction vectors between the quadrupoles; the chessboard corner point automatic detection and sorting method provided by the present invention can still detect the precise positions of all valid corner points under conditions such as illumination changes, image rotation and image distortion, and the detected chessboard corner points do not have false corner points; compared with the chessboard corner point detection method based on OpenCV in the prior art, the corner point detection accuracy of the present invention is improved by 61.87%, and the extraction result and efficiency are not easily affected by image brightness and rotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a process flow of a method for automatically detecting and sorting chessboard corner points provided by an embodiment of the present invention;

[0059] Figure 2 A structural principle block diagram of a device for automatically detecting and sorting chessboard corner points provided by an embodiment of the present invention;

[0060] Figure 3 A schematic diagram of comparison between an original image of a checkerboard grayscale image and a corner likelihood map provided by an embodiment of the present invention;

[0061] Figure 4 A schematic diagram of the minimum convex hull arrangement principle provided by an embodiment of the present invention;

[0062] Figure 5 A chessboard quadrupole extraction effect diagram provided by an embodiment of the present invention;

[0063] Figure 6 A comparison chart of the effects of extracting four poles using the coordinate extremum method and the minimum convex hull provided in an embodiment of the present invention;

[0064] Figure 7 Four images to be extracted provided by the embodiment of the present invention;

[0065] Figure 8 The method for automatically detecting and sorting chessboard corner points provided by the embodiment of the present invention is Figure 7 The chessboard corner point extraction effect diagram of the image to be extracted (d);

[0066] Fig. 9 The chessboard corner point automatic detection and sorting method and the chessboard corner point detection method based on OpenCV provided by the embodiment of the present invention are for Figure 7 The effect of extracting checkerboard corner points of the image to be extracted (a). DETAILED DESCRIPTION

[0067] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0068] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0069] Embodiment 1:

[0070] like Figure 1 As shown, the embodiment of the present invention introduces a method for automatically detecting and sorting chessboard corner points, which specifically includes the following steps:

[0071] Step 1: Obtain a checkerboard grayscale image, and use a Geiger corner detection template to extract pixel-level candidate corner points in the grayscale image;

[0072] As an embodiment of the present invention, a black and white checkerboard target is photographed by a camera to obtain a checkerboard grayscale image; it should be noted that this step uses the Geiger corner point detection template as the reference [GeigerA, MoosmannF, Car O, et al. Automatic camera and range sensor calibration using a singleshot [C] / / IEEE International Conference on Robotics & Automation. IEEE, 2012: 3936-3943] proposed two corner point templates that are symmetrical about the origin. The template uses the local response of the image area to detect corner points at two angles, and uses the two edge lines of the corner points to filter out pseudo corner points. The detection accuracy and robustness are both high, and the corner points can be identified when the image is rotated and distorted. The embodiment of the present invention utilizes the two corner point templates proposed by Geiger to perform convolution filtering on the grayscale image respectively, and then obtain a corner point likelihood map. After the image is filtered, the more similar the partial area of ​​the image is to the filter kernel, the larger the filter value is, that is, the greater the corner point similarity of each pixel point is. Then, the non-maximum suppression algorithm of the adaptive threshold is used to eliminate the pseudo corner points in the corner point likelihood map, so as to extract the pixel-level candidate corner points from the corner point likelihood map after eliminating the pseudo corner points. The selected adaptive threshold is half of the maximum corner point similarity in the likelihood map.

[0073] Step 2: Refine the number of the candidate corner points, thereby filtering out pseudo corner points in the candidate corner points, and determining the candidate corner points retained after filtering as the checkerboard corner points;

[0074] Step 3: performing sub-pixel positioning on the chessboard corner points to obtain sub-pixel corner points;

[0075] In one embodiment, the specific positioning method of this step is to calculate the sub-pixel position of the corner point of the chessboard according to the gradient characteristics of the corner point in the chessboard by formula (1):

[0076]

[0077] Where c is the calculated sub-pixel corner point, is the neighborhood of the sub-pixel corner point c and p is the neighborhood A point inside, g p is the gradient vector of point p, and T is the vector transpose sign.

[0078] Step 4: extracting chessboard quadrupoles from the sub-pixel corner points by using the minimum convex hull arrangement principle;

[0079] Step 5: Determine the starting point of the chessboard according to the interval distance between the four points of the chessboard;

[0080] Step 6: Based on the starting point of the chessboard, the direction vectors between the four poles of the chessboard are used to complete the ordering of the corner points of the chessboard.

[0081] As an embodiment of the present invention, the corner point accuracy obtained by non-maximum suppression processing in step 1 can only reach the pixel level, and it cannot be guaranteed that there are no pseudo corner points in the candidate corner points. Therefore, it is necessary to refine the number of corner points. In step 2 provided in the embodiment of the present invention, the number of corner points of the candidate corner points is refined, including the following steps:

[0082] Step 2.1: Calculate a 32-bit weighted gradient direction histogram for each candidate corner point region, wherein the histogram uses the gradient angle as the abscissa and the gradient value as the ordinate;

[0083] Step 2.2: Use the mean shift algorithm to find the gradient angles corresponding to the two maximum peaks in the histogram;

[0084] Step 2.3: The two gradient angles are used as the angles of two lines passing through the candidate corner point. If the difference between the two gradient angles is greater than a preset threshold, the candidate corner point is retained; otherwise, the candidate point is discarded.

[0085] As an embodiment of the present invention, in step 4, the chessboard quadrupoles are extracted from the sub-pixel corner points by the minimum convex hull arrangement principle, which is to use Figure 4 The minimum convex hull sorting principle in extracts convex hull vertices from sub-pixel corner points, which specifically includes the following steps:

[0086] Step 4.1: Search for the corner point at the maximum value of the horizontal coordinate of the image according to the position relationship as the initial corner point;

[0087] Specifically, let the minimum number of convex hull points of the corner point be N (N ≥ 3), and start from the initial corner point w0 to find the remaining N-1 vertices according to the sorting rule; among them, Figure 4 As shown, the coordinate origin O of the image pixel coordinate system is the point in the upper left corner of the image, the horizontal rightward direction is the u-axis direction, and the vertical downward direction is the v-axis direction;

[0088] Step 4.2: extract the minimum convex hull point from the initial corner point in the sub-pixel corner point in a clockwise direction according to the convex hull arrangement principle;

[0089] The specific extraction method is:

[0090] Step a: Connect point w0 with the rest of the points. The point corresponding to the minimum clockwise angle between the ov axis and each connecting line is w1.

[0091] Step b: Connect point w1 with the remaining unused points. The point corresponding to the minimum clockwise angle between the straight line w0w1 and the other connecting lines is w2. Similarly, point w i (2≤i≤N) also satisfies the straight line w i-1 w i-2 With straight line w i-1 w i The sorting rule with the smallest clockwise angle;

[0092] Step c: Follow step b and repeat until you return to the starting point w0;

[0093] Step 4.3: traverse the minimum convex hull points, and extract the four points with the largest corners as the chessboard quadrupoles;

[0094] It should be noted in this step that the chessboard quadrupole is exactly the point with the largest corner in the minimum convex hull point. A characteristic is used to identify the chessboard quadrupole from the minimum convex hull. The specific method is as follows:

[0095] Step 4.3a: Traverse the above minimum convex hull points, let the current minimum convex hull point be w2, and the previous and next minimum convex hull points be w1 and w3 respectively;

[0096] Step 4.3b: Determine whether the current minimum convex hull point is a vector on w2 and vector Is the cosine value of the angle between [-t, t], where t is the control threshold? If so, the point w2 is considered to be a chessboard pole;

[0097] Step 4.3c: Repeat steps 4.3a and 4.3b until all convex hull points participate in the cycle, thereby extracting the quadrupole points of the chessboard; Figure 5 As shown, these are the four poles of the chessboard {o1, o2, o3, o4}, i.e., the four vertices of the chessboard.

[0098] The minimum convex hull arrangement method used in the embodiment of the present invention is highly accurate and robust in extracting the positions of the four poles; Figure 6 , which is an effect diagram of extracting quadrupoles by the coordinate extremum method in the prior art and the minimum convex hull arrangement method adopted in the embodiment of the present invention. It can be seen that the position of one point of the quadrupoles identified by the coordinate extremum method is not accurate enough. It can be seen that when the calibration plate is rotated too much, the coordinate extremum method is not effective in identifying the quadrupoles, and its robustness is lower than that of the minimum convex hull method adopted in the present invention.

[0099] As an embodiment of the present invention, determining the starting point of the chessboard according to the interval distance between the four points of the chessboard in step 5 specifically includes the following steps:

[0100] Step 5.1: According to the clockwise direction of the identified four poles {o1, o2, o3, o4}, use the distances o1o2 and o1o4 between the poles of the chessboard to determine the long side of the chessboard;

[0101] Step 5.2: Determine whether the straight line formed by o1o2 is a long side. If o1o2 is a long side, set o1 and o3 as candidate starting points, otherwise set o2 and o4 as candidate starting points;

[0102] Step 5.3: Select a point on the left side of the image from the candidate starting points as the starting point.

[0103] As an embodiment of the present invention, in step 6, the ordering of chessboard corner points is completed by using the direction vectors between the four poles of the chessboard, including the following steps:

[0104] Step 6.1: Name the four points of the chessboard o1, o2, o3, and o4 in clockwise order, with o1 being the starting point of the chessboard. Use formula (2) to identify the corner points on the straight lines o1o2 and o1o4 respectively.

[0105]

[0106] Wherein, au+bv+c=0 is the general equation of the straight line, a, b, c are the equation coefficients, and f is the control threshold; u and v represent the horizontal and vertical coordinate values ​​of the corner point to be confirmed relative to the starting point o1 of the chessboard, respectively. If the distance from the corner point to be confirmed to the straight line satisfies not greater than the control threshold f, then the corner point to be confirmed is a point on the straight line;

[0107] Step 6.2: Based on the confirmed corner points on the straight line o1o2 and the straight line o1o4, the initial values ​​of the horizontal and vertical direction vectors between the corner points adjacent to the corner points on the straight line o1o2 and the straight line o1o4 are calculated by formula (3) respectively.

[0108]

[0109] In the formula, m is the number of corner points on the straight line o1o2, and n is the number of corner points on the straight line o1o4; is the horizontal direction vector, is the vertical direction vector;

[0110] Step 6.3: Combine the initial values ​​of the horizontal and vertical vectors of adjacent corner points The adjacent virtual corner points C in the horizontal direction and vertical direction of the current adjacent corner point C0 are obtained by formula (4) x , C y :

[0111]

[0112] Step 6.4: Based on the adjacent virtual corner points C in the horizontal and vertical directions of the adjacent corner point C0 x , C y , detect the point closest to the adjacent virtual corner point from the unsorted corner point set U, and update the current corner point and the direction vector of the current corner point in turn by formula (5) and formula (6) respectively until all corner points are sorted:

[0113]

[0114]

[0115] In the formula, C i is the point in the unsorted corner point set U that is closest to the virtual corner point, C x ′ and C y ′ are the updated current corner points in the horizontal and vertical directions of the chessboard, and It is the updated direction vector of the current corner point.

[0116] Furthermore, the embodiment of the present invention uses Figure 7 The images to be detected are shown in Figure (a), Figure (b), Figure (c), and Figure (d) respectively; to verify the accuracy and robustness of corner point detection and sorting of the automatic checkerboard corner point detection method provided by the embodiment of the present invention. The method adopted by the present invention is comparable to the checkerboard corner point detection method based on OpenCV. Figure 7 The specific extraction results of the four images to be detected are shown in Table 1. It can be seen from the table that the OpenCV corner detection method fails to extract corners for images with large changes in image brightness and large rotation of the calibration plate. Figure 8 This is the corner point extraction result of the present invention for the rotation change of Figure (d). Fig. 9 The corner point extraction results of the two methods for Figure (a) are shown in Figure 2. It can be seen that the robustness and accuracy of the corner point detection of the method used in this paper are better than those of the OpenCV algorithm.

[0117] Table 1

[0118]

[0119] According to the above experimental results, it can be seen that the method adopted by the present invention aims at the problem that the checkerboard corner point detection method based on OpenCV fails to extract corner points due to excessive illumination changes and image rotation. The Geiger corner point detection template is used to complete the sub-pixel extraction of the checkerboard corner points, and the checkerboard quadrupoles and the starting point are extracted by using the minimum convex hull arrangement principle and the checkerboard starting point feature, and finally the corner point sorting is completed through the direction vector between the quadrupoles; compared with the OpenCV detection method, the corner point detection accuracy of the present invention is improved by 61.87%, and the extraction results and efficiency are not easily affected by image brightness and rotation; the method adopted by the present invention has high robustness and accuracy for corner point detection, and is suitable for popularization and use.

[0120] Embodiment 2:

[0121] like Figure 2 As shown, an embodiment of the present invention provides a device for automatically detecting and sorting chessboard corner points, which can be used to implement the method described in Embodiment 1. The device includes:

[0122] A pixel-level candidate corner point extraction module is used to obtain a checkerboard grayscale image and extract pixel-level candidate corner points in the grayscale image using a Geiger corner point detection template;

[0123] A corner point number refinement module is used to refine the number of the candidate corner points, thereby filtering out pseudo corner points in the candidate corner points, and determining the candidate corner points retained after filtering as the checkerboard corner points;

[0124] A sub-pixel positioning module, used to obtain sub-pixel corner points after performing sub-pixel positioning on the chessboard corner points;

[0125] A chessboard quadrupole extraction module is used to extract chessboard quadrupoles from the sub-pixel corner points by using the minimum convex hull arrangement principle;

[0126] A chessboard starting point determination module is used to determine the starting point of the chessboard according to the interval distance between the four points of the chessboard;

[0127] The chessboard corner point sorting module is used to complete the chessboard corner point sorting based on the starting point of the chessboard and using the direction vectors between the four poles of the chessboard.

[0128] As an embodiment of the present invention, a module for extracting pixel-level candidate corner points includes:

[0129] A convolution filtering unit, configured to perform convolution filtering on the grayscale image using a Geiger corner detection template to obtain a corner likelihood map;

[0130] The pseudo corner point elimination unit is used to eliminate pseudo corner points in the corner point likelihood map by using a non-maximum suppression algorithm with an adaptive threshold, thereby extracting pixel-level candidate corner points from the corner point likelihood map after the pseudo corner points are eliminated.

[0131] The automatic detection and sorting device for checkerboard corner points provided in the embodiment of the present invention and the automatic detection and sorting method for checkerboard corner points provided in Example 1 are based on the same technical concept, and can produce the beneficial effects described in Example 1. For the contents not described in detail in this embodiment, please refer to Example 1.

[0132] Embodiment three:

[0133] An embodiment of the present invention provides a device for automatically detecting and sorting chessboard corner points, including a processor and a storage medium;

[0134] The storage medium is used to store instructions;

[0135] The processor is used to operate according to the instructions to execute the steps of any method in the first embodiment.

[0136] Embodiment 4:

[0137] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any method in the first embodiment are implemented.

[0138] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0139] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for automatically detecting and sorting chessboard corner points, characterized in that: The method comprises: Obtain a checkerboard grayscale image, and extract pixel-level candidate corner points in the grayscale image using a Geiger corner detection template; Refining the number of the candidate corner points, thereby filtering out pseudo corner points in the candidate corner points, and determining the candidate corner points retained after filtering as the checkerboard corner points; Performing sub-pixel positioning on the chessboard corner points to obtain sub-pixel corner points; Extracting chessboard quadrupole points from the sub-pixel corner points by using the minimum convex hull arrangement principle; Determine the starting point of the chessboard according to the interval distance between the four points of the chessboard; Based on the starting point of the chessboard, the direction vectors between the four poles of the chessboard are used to complete the ordering of the corner points of the chessboard; The method of using the direction vectors between the four points of the chessboard to sort the corner points of the chessboard includes the following steps: The four points of the chessboard are named o1, o2, o3, and o4 in clockwise direction. o1 is the starting point of the chessboard. The corner points on the straight line o1o2 and the straight line o1o4 are respectively confirmed by formula (2); Wherein, au+bv+c=0 is the general equation of the straight line, a, b, c are the equation coefficients, and f is the control threshold; u and v represent the horizontal and vertical coordinate values ​​of the corner point to be confirmed relative to the starting point o1 of the chessboard, respectively. If the distance from the corner point to be confirmed to the straight line satisfies not greater than the control threshold f, then the corner point to be confirmed is a point on the straight line; Based on the confirmed corner points on the straight line o1o2 and the straight line o1o4, the initial values ​​of the horizontal and vertical direction vectors between the corner points adjacent to the corner points on the straight line o1o2 and the straight line o1o4 are calculated by formula (3): In the formula, m is the number of corner points on the straight line o1o2, and n is the number of corner points on the straight line o1o4; is the horizontal direction vector, is the vertical direction vector; Combine the initial values ​​of the horizontal and vertical vectors of adjacent corner points The adjacent virtual corner points C in the horizontal direction and vertical direction of the current adjacent corner point C0 are obtained by formula (4) x , C y : According to the adjacent virtual corner points C in the horizontal and vertical directions of the adjacent corner point C0 x , C y , detect the point closest to the adjacent virtual corner point from the unsorted corner point set U, and update the current corner point and the direction vector of the current corner point in turn by formula (5) and formula (6) respectively until all corner points are sorted: In the formula, C i is the point in the unsorted corner point set U that is closest to the virtual corner point, C x ' and C y ' are the updated current corner points in the horizontal and vertical directions of the chessboard, and It is the updated direction vector of the current corner point.

2. The method for automatically detecting and sorting chessboard corner points according to claim 1, characterized in that: The step of extracting pixel-level candidate corner points in the grayscale image using the Geiger corner point detection template comprises: Performing convolution filtering on the grayscale image using a Geiger corner detection template to obtain a corner likelihood map; The pseudo corner points in the corner likelihood map are removed by using a non-maximum suppression algorithm with an adaptive threshold, so as to extract pixel-level candidate corner points from the corner likelihood map after the pseudo corner points are removed.

3. The method for automatically detecting and sorting chessboard corner points according to claim 1, characterized in that: The step of refining the number of the candidate corner points comprises: Calculate a 32-bit weighted gradient direction histogram for each candidate corner point region, wherein the histogram uses the gradient angle as the abscissa and the gradient value as the ordinate; Using a mean shift algorithm to find the gradient angles corresponding to the two maximum peaks in the histogram; The two gradient angles are used as the angles of two lines passing through the candidate corner point. If the difference between the two gradient angles is greater than a preset threshold, the candidate corner point is retained, otherwise the candidate point is eliminated.

4. The method for automatically detecting and sorting chessboard corner points according to any one of claims 1 to 3, characterized in that: The step of performing sub-pixel positioning on the chessboard corner points comprises: According to the gradient characteristics of the corner points in the chessboard, the sub-pixel position of the corner points of the chessboard is calculated by formula (1): Where c is the calculated sub-pixel corner point, is the neighborhood of the sub-pixel corner point c and p is the neighborhood A point inside, g p is the gradient vector of point p, and T is the vector transpose sign.

5. The method for automatically detecting and sorting chessboard corner points according to claim 1, characterized in that: The step of extracting chessboard quadrupoles from the sub-pixel corner points by using the minimum convex hull arrangement principle comprises: According to the position relationship, search for the corner point located at the maximum value of the horizontal coordinate of the image as the initial corner point; According to the convex hull arrangement principle, the minimum convex hull point is extracted from the initial corner point in the sub-pixel corner point in a clockwise direction; The minimum convex hull points are traversed, and four points with the largest corners are extracted as the chessboard quadrupoles.

6. A device for automatically detecting and sorting chessboard corner points, characterized in that: The device comprises: A pixel-level candidate corner point extraction module is used to obtain a checkerboard grayscale image and extract pixel-level candidate corner points in the grayscale image using a Geiger corner point detection template; A corner point number refinement module is used to refine the number of the candidate corner points, thereby filtering out pseudo corner points in the candidate corner points, and determining the candidate corner points retained after filtering as the checkerboard corner points; A sub-pixel positioning module, used to obtain sub-pixel corner points after performing sub-pixel positioning on the chessboard corner points; A chessboard quadrupole extraction module is used to extract chessboard quadrupoles from the sub-pixel corner points by using the minimum convex hull arrangement principle; A chessboard starting point determination module is used to determine the starting point of the chessboard according to the interval distance between the four poles of the chessboard; A chessboard corner point sorting module is used to sort the chessboard corner points based on the starting point of the chessboard and using the direction vectors between the four poles of the chessboard; The method of using the direction vectors between the four points of the chessboard to sort the corner points of the chessboard includes the following steps: The four points of the chessboard are named o1, o2, o3, and o4 in clockwise direction. o1 is the starting point of the chessboard. The corner points on the straight line o1o2 and the straight line o1o4 are respectively confirmed by formula (2); Wherein, au+bv+c=0 is the general equation of the straight line, a, b, c are the equation coefficients, and f is the control threshold; u and v represent the horizontal and vertical coordinate values ​​of the corner point to be confirmed relative to the starting point o1 of the chessboard, respectively. If the distance from the corner point to be confirmed to the straight line satisfies not greater than the control threshold f, then the corner point to be confirmed is a point on the straight line; Based on the confirmed corner points on the straight line o1o2 and the straight line o1o4, the initial values ​​of the horizontal and vertical direction vectors between the corner points adjacent to the corner points on the straight line o1o2 and the straight line o1o4 are calculated by formula (3): In the formula, m is the number of corner points on the straight line o1o2, and n is the number of corner points on the straight line o1o4; is the horizontal direction vector, is the vertical direction vector; Combine the initial values ​​of the horizontal and vertical vectors of adjacent corner points The adjacent virtual corner points C in the horizontal direction and vertical direction of the current adjacent corner point C0 are obtained by formula (4) x , C y : According to the adjacent virtual corner points C in the horizontal and vertical directions of the adjacent corner point C0 x , C y , detect the point closest to the adjacent virtual corner point from the unsorted corner point set U, and update the current corner point and the direction vector of the current corner point in turn by formula (5) and formula (6) respectively until all corner points are sorted: In the formula, C i is the point in the unsorted corner point set U that is closest to the virtual corner point, C x ' and C y ' are the updated current corner points in the horizontal and vertical directions of the chessboard, and It is the updated direction vector of the current corner point.

7. The automatic detection and sorting device for chessboard corner points according to claim 6, characterized in that: The module for extracting pixel-level candidate corner points comprises: A convolution filtering unit, configured to perform convolution filtering on the grayscale image using a Geiger corner detection template to obtain a corner likelihood map; The pseudo corner point elimination unit is used to eliminate pseudo corner points in the corner point likelihood map by using a non-maximum suppression algorithm with an adaptive threshold, thereby extracting pixel-level candidate corner points from the corner point likelihood map after the pseudo corner points are eliminated.

8. A device for automatically detecting and sorting chessboard corner points, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of automatically detecting and sorting checkerboard corner points according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of automatically detecting and sorting checkerboard corner points as described in any one of claims 1 to 5 are implemented.

Citation Information

Cited By

  • Checkerboard corner detection method and system based on image texture

    CN119559035A