Checkerboard calibration board angular point screening method, device and equipment and storage medium

Abstract

The invention discloses a checkerboard calibration board angular point screening method, device and equipment and a storage medium, and relates to the technical field of visual processing. The method comprises the following steps: acquiring all angular points of a calibration board on a checkerboard to be detected by using an angular point detection algorithm, and constructing a first angular point set; performing feature analysis on each angular point in the first angular point set, comparing the feature value of each angular point with a first preset threshold value, and reserving the angular points within the range of the first preset threshold value to construct a second angular point set; screening out calibration target areas of black checks on the checkerboard calibration plate; constructing a circumscribed polygon surrounding the calibration target area; and judging whether the angular points in the second angular point set are located inside the circumscribed polygon, and reserving the angular points located inside the circumscribed polygon to construct an effective calibration point set. By screening out the effective area of the calibration plate, the calibration point is only located on the checkerboard, and angular points on a non-checkerboard calibration plate are avoided, so that the robustness of checkerboard calibration is improved.

Description

Technical Field

[0001] The present application relates to the field of visual processing technology, and in particular to a method, device, equipment and storage medium for screening corner points of a checkerboard calibration plate. Background Art

[0002] A checkerboard calibration plate is typically a flat plate printed with a black and white square grid pattern. The corners of each grid can be used as significant features in image processing. The coordinates of these corners are known, and their positions in the image can be detected using a corner detection algorithm through image processing. These serve as feature points for industrial camera calibration.

[0003] However, existing corner detection algorithms have poor anti-interference capabilities for corner points that are not black and white checkerboard patterns, and can also detect corner points in the background outside the checkerboard calibration plate. This requires the calibration plate image to have a simple background free of interfering points, ensuring that all detected corner points are valid to successfully complete industrial camera calibration. However, in industrial applications, the scenes are often complex and ever-changing, and the background environment is also relatively complex, which is very unfavorable for extracting valid corner points.

[0004] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method, device, equipment and storage medium for screening corner points of a checkerboard calibration plate, aiming to solve the technical problem in the prior art that corner points on non-checkerboard calibration plates are easily extracted when extracting checkerboard corner points in complex environments.

[0006] To achieve the above objectives, the present application proposes a checkerboard calibration plate corner point screening method, the checkerboard calibration plate corner point screening method comprising:

[0007] Use the corner detection algorithm to obtain all corner points of the calibration plate on the checkerboard to be detected and construct the first corner point set;

[0008] Performing feature analysis on each corner point in the first corner point set, comparing a feature value of each corner point with a first preset threshold, and retaining corner points within the first preset threshold to form a second corner point set;

[0009] Filter out the calibration target area of ​​the black squares on the checkerboard calibration plate;

[0010] Constructing a circumscribed polygon surrounding the calibration target area;

[0011] It is determined whether the corner points in the second corner point set are located inside the circumscribed polygon, and the corner points located inside the circumscribed polygon are retained to construct a valid calibration point set.

[0012] In one embodiment, the step of performing feature analysis on each corner point in the first corner point set, comparing a feature value of each corner point with a first preset threshold, and retaining corner points within the first preset threshold to construct the second corner point set includes:

[0013] Perform image segmentation on the chessboard to obtain a binary image;

[0014] Performing feature analysis on each corner point in the first corner point set using the binary image to determine a pairing rate of pixels and a difference rate of black pixels within a corner point region;

[0015] The second corner point set is constructed by retaining corner points whose pixel pairing ratio and black pixel difference ratio are both within the first preset threshold range.

[0016] In one embodiment, after the step of performing image segmentation on the checkerboard to obtain a binary image, the method further includes:

[0017] The checkerboard is segmented using Otsu automatic thresholding to obtain a binary image;

[0018] The point with a pixel value of 255 in the binary image is defined as a foreground target point;

[0019] Converting all of the foreground object points in the binary image into a foreground object area;

[0020] The foreground target area is stored in the form of row coordinates of adjacent target point pixels, start coordinates of adjacent target point pixels, and end coordinates of adjacent target point pixels.

[0021] In one embodiment, the step of screening out the calibration target area of ​​the black squares on the checkerboard calibration plate includes:

[0022] Performing morphological dilation processing on the image area of ​​the checkerboard calibration plate using a rectangular structure element of a first preset size to obtain a first dilated area;

[0023] Performing a connected domain analysis on the first expanded area to screen out the largest white connected area;

[0024] Performing hole filling on the white connection area to obtain a hole-filling area inside the white connection area;

[0025] Performing a difference operation on the hole-filled area and the white connected area to obtain a difference area;

[0026] Performing morphological dilation processing on the difference region using a rectangular structure element of a second preset size to obtain a second dilated region;

[0027] A connected domain analysis is performed on the second expanded area, and the black connected area with the largest area is screened out to obtain a calibration target area of ​​black squares.

[0028] In one embodiment, the step of constructing a circumscribed polygon surrounding the calibration target area includes:

[0029] Calculating the coordinates of the horizontal circumscribed rectangle of the calibration target area;

[0030] Filtering a boundary point set whose coordinate values ​​are respectively equal to the coordinates of the horizontal circumscribed rectangle from the point set represented by the second dilated area, and calculating the centroid points of the four sides of the boundary point set;

[0031] Constructing a coordinate system for the calibration target area;

[0032] Generate four straight lines surrounding the calibration target area according to the coordinate system and the four-side center of gravity points;

[0033] The intersection points of four straight lines surrounding the calibration target area are calculated as four side points, and the four side points are extended to the four sides by a preset distance according to the direction of the coordinate system to construct the circumscribed polygon.

[0034] In one embodiment, the step of constructing a coordinate system for calibrating the target area includes:

[0035] Calculating the coordinates of the center of gravity of the second corner point set;

[0036] Select the four coordinate system corner points closest to the coordinates of the center of gravity point from the second corner point set;

[0037] Three corner points are selected from the four corner points of the coordinate system as the origin, x-axis point and y-axis point of the coordinate system of the calibration target area. The difference between the coordinate axis angle of the constructed coordinate system and 90 degrees meets the preset angle threshold and the sum of the distances between the two axis points and the origin is the smallest.

[0038] In one embodiment, the step of generating four straight lines surrounding the calibration target area according to the coordinate system and the four-side center of gravity points includes:

[0039] Calculate the points on the left and right sides of the x-axis and y-axis of the coordinate axis among the four centroid points, obtain the x-axis left point located to the left of the x-axis and farthest from the x-axis, the x-axis right point located to the right of the x-axis and farthest from the x-axis, obtain the y-axis left point located to the left of the y-axis and farthest from the y-axis, and the y-axis right point located to the right of the y-axis and farthest from the y-axis;

[0040] Calculating coordinate axis vectors according to the corner points of the coordinate system;

[0041] According to the coordinate axis vector, straight lines passing through the left point of the x-axis, the right point of the x-axis, the left point of the y-axis and the right point of the y-axis and parallel to the coordinate axis are calculated respectively to construct four straight lines surrounding the calibration target area.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a checkerboard calibration plate corner point screening device, the checkerboard calibration plate corner point screening device comprising:

[0043] A corner point acquisition module is used to obtain all corner points of the calibration plate on the checkerboard to be detected using a corner point detection algorithm to construct a first corner point set;

[0044] a primary optimization module, configured to perform feature analysis on each corner point in the first corner point set, compare a feature value of each corner point with a first preset threshold, and retain corner points within the first preset threshold to form a second corner point set;

[0045] The area screening module is used to screen out the calibration target area of ​​the black squares on the checkerboard calibration board;

[0046] A region construction module, configured to construct a circumscribed polygon surrounding the calibration target region;

[0047] The secondary optimization module is used to determine whether the corner points in the second corner point set are located inside the circumscribed polygon, and retain the corner points located inside the circumscribed polygon to construct a valid calibration point set.

[0048] In addition, to achieve the above-mentioned purpose, the present application also proposes a checkerboard calibration plate corner point screening device, which includes: a memory, a processor, and a computer program stored on the memory and runnable on the processor, and the computer program is configured to implement the steps of the checkerboard calibration plate corner point screening method as described above.

[0049] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the checkerboard calibration plate corner point screening method described above are implemented.

[0050] The present application provides a method for screening corner points of a checkerboard calibration plate. The present application first uses a corner point detection algorithm to obtain all corner points of the calibration plate on the checkerboard to be detected, and constructs a first corner point set. The present application then performs feature analysis on each corner point in the first corner point set, compares the feature value of each corner point with a first preset threshold, and retains the corner points within the first preset threshold to construct a second corner point set. The present application then screens out a calibration target area of ​​black squares on the checkerboard calibration plate, and constructs a circumscribed polygon surrounding the calibration target area. The present application then determines whether the corner points in the second corner point set are located inside the circumscribed polygon, and retains the corner points located inside the circumscribed polygon to construct a valid calibration point set. By screening out the valid area of ​​the calibration plate, the calibration points are located only on the checkerboard, avoiding corner points on the non-checkerboard calibration plate, thereby improving the robustness of the checkerboard calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0053] Figure 1 This is a flow chart of the first embodiment of the checkerboard calibration plate corner point screening method of the present application;

[0054] Figure 2 A schematic diagram of the process flow provided for Example 2 of the checkerboard calibration plate corner point screening method of this application;

[0055] Figure 3 A schematic diagram of the process flow provided for the third embodiment of the checkerboard calibration plate corner point screening method of the present application;

[0056] Figure 4 A schematic diagram of a chessboard provided in Example 3 of the chessboard calibration plate corner point screening method of this application;

[0057] Figure 5 A schematic diagram of a chessboard for screening out white connected areas provided in Example 3 of the chessboard calibration plate corner point screening method of this application;

[0058] Figure 6 A schematic diagram of the calibration target area provided in Example 3 of the checkerboard calibration plate corner point screening method of this application;

[0059] Figure 7A schematic diagram of the process of constructing a circumscribed polygon provided in Example 3 of the checkerboard calibration plate corner point screening method of this application;

[0060] Figure 8 This is a schematic diagram of the module structure of the checkerboard calibration plate corner point screening device according to an embodiment of the present application;

[0061] Figure 9 Schematic diagram of the structure of a checkerboard calibration plate corner point screening device in the hardware operating environment involved in the checkerboard calibration plate corner point screening method in the embodiment of the present application.

[0062] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0063] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0064] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0065] The main solution of the embodiment of the present application is: use a corner point detection algorithm to obtain all corner points of the calibration plate on the chessboard to be detected, and construct a first corner point set; perform feature analysis on each corner point in the first corner point set, and retain the corner points within the first preset threshold range by comparing the feature value of each corner point with a first preset threshold to construct a second corner point set; filter out the calibration target area of ​​the black squares on the chessboard calibration plate; construct a circumscribed polygon surrounding the calibration target area; determine whether the corner points in the second corner point set are located inside the circumscribed polygon, and retain the corner points located inside the circumscribed polygon to construct a valid calibration point set.

[0066] Because existing corner detection algorithms have poor interference resistance against non-black and white checkerboard corners, corners in the background outside the checkerboard calibration plate can also be detected. This requires the calibration plate image to have a simple background free of interfering points, ensuring that all detected corners are valid to successfully complete industrial camera calibration. However, in industrial applications, on-site scenes are often complex and ever-changing, and the background environment is also relatively complex, which is very unfavorable for extracting valid corners.

[0067] This application provides a solution that restricts valid calibration points to only those on the checkerboard grid by locating the effective area of ​​the calibration plate, thereby avoiding corner points on non-checkerboard calibration plates in complex environments. This solution can effectively extract valid corner points for checkerboard calibration plates that are straight, perspectively tilted, distorted, or partially out of view, improving the usability and robustness of checkerboard calibration.

[0068] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or a checkerboard calibration board corner point screening device capable of performing the above functions. The following uses the checkerboard calibration board corner point screening device as an example to illustrate this embodiment and the following embodiments.

[0069] Based on this, the embodiment of the present application provides a checkerboard calibration plate corner point screening method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the checkerboard calibration plate corner point screening method of the present application.

[0070] In this embodiment, the checkerboard calibration plate corner point screening method includes steps S10 to S50:

[0071] Step S10: using a corner point detection algorithm to obtain all corner points of the calibration plate on the checkerboard to be detected, and constructing a first corner point set.

[0072] It should be noted that the corner detection algorithm can be an algorithm for detecting points with significant features in an image (for example, points located at the intersection of multiple edges, where the change in pixel grayscale is more obvious in multiple directions), for example, it can include the Harris corner detection algorithm or the Shi-Tomasi corner detection algorithm.

[0073] It should be understood that the above corner point detection algorithm has poor anti-interference ability for non-black and white checkerboard corner points, so corner points in the background outside the checkerboard calibration plate will also be detected. The first corner point set can be the set of all corner points inside and outside the checkerboard calibration plate detected by the corner point detection algorithm, pts = {x i ,y i}.

[0074] Step S20: performing feature analysis on each corner point in the first corner point set, comparing the feature value of each corner point with a first preset threshold, and retaining corner points within the first preset threshold to construct a second corner point set.

[0075] It should be noted that the process of performing feature analysis on the corner points may be to preliminarily screen the first set of corner points for those that meet the criteria of being surrounded by both white and black checkerboards based on characteristics of a black and white checkerboard (e.g., by using a white pixel pairing ratio and a black pixel difference ratio). The feature value may be a value that determines whether a corner point meets the criteria of being surrounded by both white and black checkerboards.

[0076] It should be understood that due to the perspective tilt or distortion of the checkerboard calibration plate, the characteristic values ​​of the corner points will not completely conform to the values ​​of the standard corner point characteristics. Therefore, the first preset threshold can be set in advance, and the corner points within the first preset threshold range can be preliminarily eliminated from the non-calibrated plate. The retained intersection points construct the second corner point set pts1 = {x i ,y i}.

[0077] Step S30: Filter out the calibration target area of ​​the black squares on the checkerboard calibration plate.

[0078] It should be noted that the black squares on the checkerboard calibration are filtered out, and the target located in the filtered area (i.e., the calibration target area) is located on the checkerboard calibration plate. The calibration target area can be determined for checkerboard calibration plates that are straight, perspectively tilted, distorted, or partially out of view.

[0079] It should be understood that the screening of the calibration target area can be achieved through morphological dilation processing and connected domain analysis, which is not specifically limited in this embodiment.

[0080] Step S40: constructing a circumscribed polygon surrounding the calibration target area.

[0081] Step S50: Determine whether the corner points in the second corner point set are located inside the circumscribed polygon, and retain the corner points located inside the circumscribed polygon to construct a valid calibration point set.

[0082] It should be noted that the circumscribed polygon surrounds the calibration target area, that is, all black squares on the checkerboard calibration board. Therefore, the circumscribed polygon must be on the calibration board, and all corner points within the circumscribed polygon are valid corner points. Therefore, by determining whether each corner point in the second corner point set is inside the circumscribed polygon, the valid calibration point set can be constructed using the corner points within the circumscribed polygon.

[0083] In this embodiment, a corner point detection algorithm is used to obtain all corner points of the calibration plate on the checkerboard to be detected, thereby constructing a first set of corner points. Feature analysis is performed on each corner point in the first set of corner points. The feature value of each corner point is compared with a first preset threshold, and corner points within the first preset threshold are retained to form a second set of corner points. A calibration target area of ​​black squares on the checkerboard calibration plate is screened out. A circumscribed polygon surrounding the calibration target area is constructed. A determination is made as to whether corner points in the second set of corner points are located within the circumscribed polygon, and corner points located within the circumscribed polygon are retained to form a set of valid calibration points. By screening the valid area of ​​the calibration plate, calibration points are located only on the checkerboard, avoiding corner points on non-checkerboard calibration plates, thereby improving the robustness of the checkerboard calibration.

[0084] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 , Figure 2 This is a flow chart of the second embodiment of the checkerboard calibration plate corner point screening method of the present application. In step S20, the checkerboard calibration plate corner point screening method includes:

[0085] Step S201: performing image segmentation on the chessboard to obtain a binary image.

[0086] It should be noted that the checkerboard image can be segmented using Otsu's automatic thresholding to obtain a binary image. The Otsu's automatic thresholding can be an image segmentation method that assumes that the image consists of two parts, a foreground and a background, and segments the foreground and background regions by calculating a threshold that maximizes the between-class variance.

[0087] It should be understood that the points with a pixel value of 255 in the binary image can be defined as foreground target points, and all foreground target points in the binary image can be converted into foreground target regions. The foreground target regions are stored in the form of a run set to reduce the amount of data and improve computing performance. A run is defined as a structure of consecutive adjacent pixels in an image row, that is, run r = {row, cb, ce}, where row is the row coordinate of the adjacent target point pixel, cb is the starting coordinate of the adjacent target point pixel, ce is the ending coordinate of the adjacent target point pixel, and foreground target region region = {r i}.

[0088] Step S202: performing feature analysis on each corner point in the first corner point set using the binary image to determine the pairing rate of pixels and the difference rate of black pixels within the corner point area.

[0089] Step S203: retaining corner points whose pixel pairing ratio and black pixel difference ratio are both within the first preset threshold range to form a second corner point set.

[0090] It should be noted that in a binary image, a pixel value in the corner area is equal to the pixel value at its diagonal, which is a paired pixel. Then the pixel pairing rate satisfies the formula:

[0091] PairRatio=pairNum / totalPixNum

[0092] This formula describes the pairing of pixels within the corner region. PairRatio is the pixel pairing ratio, pairNum is the number of paired pixels within the region, and totalPixNum is the total number of black and white pixels within the region.

[0093] Furthermore, the black pixel difference rate satisfies the formula:

[0094] diffRatio=abs(blackPixNum-0.5·totalPixNum) / (0.5·totalPixNum)

[0095] Among them, diffRatio is the melanin pixel difference rate, and blackPixNum is the number of black pixels in the area.

[0096] It should be understood that for a standard, upright, undistorted checkerboard calibration surface, if a given field size of 15 pixels is used, and within a 15-pixel square field around a corner point, the upper left and lower right portions of the corner point are entirely white and the lower left and upper right portions are entirely black, or the upper left and lower right portions of the corner point are entirely black and the lower left and upper right portions are entirely white, then the pixel pairing rate for the entire field is 100%, and the black pixel error rate is 0%. If the checkerboard calibration surface has perspective tilt or distortion, the pixel pairing rate will be less than 100%, and the black pixel error rate will be greater than 0%. The greater the perspective tilt or distortion, the lower the pairing rate and the greater the black pixel error rate. To ensure that corner points are on the calibration surface, the pixel pairing rate threshold is typically 0.75, and the black pixel error rate is 0.35. Therefore, the pixel pairing rate and black pixel error rate are used to minimize the number of corner points that are not on the calibration surface.

[0097] In this embodiment, a binary image is obtained by segmenting the checkerboard. This binary image is then used to perform feature analysis on each corner point in the first set of corner points, determining the pixel pairing ratio and black pixel difference ratio within the corner point region. Corner points whose pixel pairing ratio and black pixel difference ratio are both within the first preset threshold range are retained to form a second set of corner points. Corner points are screened based on the white pixel pairing ratio and black pixel difference ratio, minimizing the number of corner points on the non-calibrated surface. The majority of retained corner points will be located on the calibrated checkerboard surface.

[0098] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first and second embodiments above can be referred to the above introduction and will not be described in detail later. Figure 3 , Figure 3 This is a flow chart of the third embodiment of the checkerboard calibration plate corner point screening method provided in this application.

[0099] In step S30, the checkerboard calibration plate corner point screening method includes:

[0100] Step S301: performing morphological dilation processing on the image area of ​​the checkerboard calibration plate using a rectangular structure element of a first preset size to obtain a first dilated area.

[0101] It should be noted that image dilation can expand the highlighted areas or white portions of an image, making the image after dilation larger than the highlighted areas before dilation. This can be achieved by scanning each pixel of the image using a structuring element of a defined first preset size. The first preset size can be set according to the actual image conditions and is not specifically limited in this embodiment. Morphological dilation of a region using a rectangular structuring element of the first preset size s yields the region dilateRe.

[0102] Step S302: performing a connected domain analysis on the first expanded area to screen out the largest white connected area.

[0103] Reference Figure 4 and Figure 5 , Figure 4 A schematic diagram of a chessboard provided in Example 3 of the chessboard calibration plate corner point screening method of this application; Figure 5 Schematic diagram of a chessboard for screening out white connected areas provided in Example 3 of the chessboard calibration plate corner point screening method of this application. Figure 4 The first region 100, the second region 200, and the third region 300 in the image are all white regions at the boundaries of the calibration plate, i.e., regions on the non-checkerboard calibration plate. Connected domain analysis is performed on the dilated region dilateRe, connecting the white regions into a whole and selecting the region with the largest area, selRe.

[0104] It should be understood that the white connected region with the largest area of ​​the circumscribed matrix can also be screened out. Usually, the white connected region with the largest area can meet the screening requirements.

[0105] Step S303: performing hole filling on the white connected area to obtain a hole-filled area inside the white connected area.

[0106] Step S304: performing a difference operation on the hole-filling area and the white connected area to obtain a difference area.

[0107] Step S305: performing morphological dilation processing on the difference region using a rectangular structure element of a second preset size to obtain a second dilated region.

[0108] Step S306: performing a connected domain analysis on the second expanded area, screening out the black connected area with the largest area to obtain a calibration target area of ​​black squares.

[0109] Reference Figure 6 , Figure 6 This is a schematic diagram of the calibration target area provided in Example 3 of the checkerboard calibration plate corner point screening method of this application.

[0110] It should be noted that the hole filling of selRe obtains the area fullupRe, and the difference operation of fullupRe and selRe is performed to obtain the difference area diffRe, that is: diffRe = fullupRe-selRe. The difference area is the black square area on the calibration plate. The second dilated area dilateRe2 is obtained by using a rectangular structure element of a second preset size for the difference area diffRe. It is also possible to perform a connected domain analysis on the dilateRe2 area and filter out the calibration target area selRe2 that meets certain characteristics (such as the largest area). The black squares on the calibration plate are connected to form a whole area.

[0111] Further, refer to Figure 7 , Figure 7 This is a flow chart of the construction of a circumscribed polygon provided in Example 3 of the checkerboard calibration plate corner point screening method of this application. Taking the circumscribed quadrilateral as an example, in step S40, the checkerboard calibration plate corner point screening method includes:

[0112] Step S701: Calculate the coordinates of the horizontal circumscribed rectangle of the calibration target area.

[0113] It should be noted that the upper and lower y coordinates (y top 、y bottom ) and the left and right x coordinates (x left 、x rightcoordinate).

[0114] Step S702: Filtering out a set of boundary points whose coordinate values ​​are respectively equal to the coordinates of the horizontal circumscribed rectangle from the set of points represented by the second dilated area, and calculating the centroid points of the four sides of the set of boundary points.

[0115] It should be noted that, from the point set represented by the second dilated region dilateRe2 in the above step, the coordinate values ​​corresponding to y are selected. top 、y bottom 、x left 、x right Equal point sets, and then calculate the center of gravity of the point sets respectively, defined as topPt, bottomPt, leftPt, and rightPt points.

[0116] Step S703: constructing a coordinate system for the calibration target area.

[0117] It should be noted that the second corner point set pts1 = {x i ,y i}, denoted as gxy={x ave ,y ave}, take the four coordinate system corner points closest to the coordinates of the center of gravity in the second corner point set; take three corner points from the four coordinate system corner points as the origin originPt, x-axis point axisxPt and y-axis point axisyPt of the coordinate system of the calibration target area, which can form a total of 12 different coordinate systems. Select a coordinate system in which the difference between the coordinate axis angle and 90 degrees meets a certain threshold and the sum of the distances between the two axis points and the origin is the smallest. Obtain the x-axis line: lx = a1·x+b1·y+c1; the y-axis line: ly = a2·x+b2·y+c2.

[0118] Step S704: Generate four straight lines surrounding the calibration target area according to the coordinate system and the four-side center points.

[0119] It should be noted that the points on the left and right sides of the x-axis and y-axis of the coordinate axis among the four centroids can be calculated to obtain the left x-axis point xleftPt located on the left side of the x-axis and farthest from the x-axis, the right x-axis point xrightPt located on the right side of the x-axis and farthest from the x-axis, the left y-axis point yleftPt located on the left side of the y-axis and farthest from the y-axis, and the right y-axis point yrightPt located on the right side of the y-axis and farthest from the y-axis.

[0120] It should be understood that when the four centroid points (topPt, bottomPt, leftPt, rightPt) are substituted into the above-mentioned formulas for straight line lx and straight line ly, the point where a·xi+b·yi+c>0 is the left point of the straight line, and a·xi+b·yi+c<0 is the right point of the straight line.

[0121] Furthermore, the coordinate axis vector can be calculated based on the coordinate system corner points (originPt, axisxPt and axisyPt); wherein the X-axis vector axisxVec satisfies the formula:

[0122] axisxVec=[axisxPt.x-originPt.x, axisyPt.y-originPt.y]

[0123] The Y-axis vector axisxVec satisfies the formula:

[0124] axisyVec=[axisyPt.x-originPt.x, axisyPt.y-orighinPt.y]

[0125] Furthermore, the straight lines passing through the left point of the x-axis, the right point of the x-axis, the left point of the y-axis, and the right point of the y-axis and parallel to the coordinate axes can be calculated based on the coordinate axis vector to construct four straight lines surrounding the calibration target area. The straight lines passing through the points xleftPt and xrightPt are parallel to the vector axisxVec, the straight line passing through xleftPt is the top line topLine, the straight line passing through xrightPt is the top line bottomLine, the straight lines passing through the points yleftPt and yrightPt are parallel to axisyVec, the straight line passing through yleftPt is the left line leftLine, and the straight line passing through yrightPt is the right line rightLine.

[0126] Step S705: Calculate the intersection points of four straight lines surrounding the calibration target area as four side points, and extend the four side points to the surrounding areas by a preset distance according to the direction of the coordinate system to construct the circumscribed polygon.

[0127] It should be noted that we can calculate the intersection point p1 of topLine and leftLine; the intersection point p2 of topLine and rightLine; the intersection point p3 of bottomLine and leftLine; and the intersection point p4 of bottomLine and rightLine. We then expand p1, p2, p3, and p4 by a certain distance based on the coordinate system orientation so that they fully enclose the black grid area. The expansion distance is a certain ratio of the distance between the coordinate axis point and the coordinate system origin, for example, 10%.

[0128] It should be understood that pts1={x i ,y i} Whether the midpoint is within the polygon, if so, retain the valid calibration point set pts2 = {x i ,y i}.

[0129] In this embodiment, the coordinates of the horizontal circumscribed rectangle of the calibration target area are calculated; a set of boundary points whose coordinates correspond to the coordinates of the horizontal circumscribed rectangle are selected from the point set represented by the second expanded area, and the four centroids of the boundary points are calculated; a coordinate system for the calibration target area is constructed; four straight lines enclosing the calibration target area are generated based on the coordinate system and the four centroids; the intersection points of the four straight lines enclosing the calibration target area are calculated as the four boundary points, and the four boundary points are extended in all directions by a predetermined distance according to the coordinate system to construct the circumscribed polygon. By locating the valid area of ​​the calibration plate, valid calibration points are restricted to only those on the checkerboard grid, thereby avoiding corner points on non-checkerboard calibration plates in complex environments. Valid corner points can be effectively extracted for checkerboard calibration plates that are straight, perspective-tilted, distorted, or partially out of view, thereby improving the usability and robustness of checkerboard calibration.

[0130] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the checkerboard calibration plate corner point screening method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0131] This application also provides a checkerboard calibration plate corner point screening device, please refer to Figure 8 The checkerboard calibration plate corner point screening device includes:

[0132] A corner point acquisition module 10 is configured to acquire all corner points of the calibration plate on the checkerboard to be detected using a corner point detection algorithm to construct a first corner point set;

[0133] a primary optimization module 20 configured to perform feature analysis on each corner point in the first corner point set, compare a feature value of each corner point with a first preset threshold, and retain corner points within the first preset threshold to form a second corner point set;

[0134] The area screening module 30 is used to screen out the calibration target area of ​​the black squares on the checkerboard calibration plate;

[0135] An area construction module 40 is used to construct a circumscribed polygon surrounding the calibration target area;

[0136] The secondary optimization module 50 is configured to determine whether the corner points in the second corner point set are located inside the circumscribed polygon, and retain the corner points located inside the circumscribed polygon to construct a valid calibration point set.

[0137] Optionally, the primary optimization module 20 is also used to perform image segmentation on the checkerboard to obtain a binary image; use the binary image to perform feature analysis on each corner point in the first corner point set to determine the pairing rate of pixels and the difference rate of black pixels within the corner point area; retain the corner points whose pairing rate of pixels and the difference rate of black pixels are both within the first preset threshold range to construct a second corner point set.

[0138] Optionally, the primary optimization module 20 is further used to perform image segmentation on the checkerboard using Otsu automatic thresholding to obtain a binary image; define points with a pixel value of 255 in the binary image as foreground target points; convert all the foreground target points in the binary image into a foreground target area; and store the foreground target area in the form of row coordinates of adjacent target point pixels, starting coordinates of adjacent target point pixels, and ending coordinates of adjacent target point pixels.

[0139] Optionally, the region screening module 30 is further configured to perform morphological dilation processing on the image region of the checkerboard calibration plate using a rectangular structuring element of a first preset size to obtain a first dilated region; perform connected domain analysis on the first dilated region to screen out the white connected region with the largest area; perform hole filling on the white connected region to obtain a hole-filled region within the white connected region; perform a difference operation on the hole-filled region and the white connected region to obtain a difference region; perform morphological dilation processing on the difference region using a rectangular structuring element of a second preset size to obtain a second dilated region; perform connected domain analysis on the second dilated region to screen out the black connected region with the largest area to obtain a calibration target region of the black square.

[0140] Optionally, the region construction module 40 is further configured to calculate the coordinates of a horizontal circumscribed rectangle of the calibration target region; select a set of boundary points whose coordinate corresponding values ​​are equal to the coordinates of the horizontal circumscribed rectangle from the point set represented by the second expanded region, and calculate the four side centroids of the boundary point set; construct a coordinate system for the calibration target region; generate four straight lines enclosing the calibration target region based on the coordinate system and the four side centroids; calculate the intersection points of the four straight lines enclosing the calibration target region as four side points, and expand the four side points in all directions by a preset distance according to the direction of the coordinate system to construct the circumscribed polygon.

[0141] Optionally, the area construction module 40 is also used to calculate the coordinates of the center of gravity of the second corner point set; take the four coordinate system corner points closest to the coordinates of the center of gravity in the second corner point set; take three corner points from the four coordinate system corner points as the origin, x-axis point and y-axis point of the coordinate system of the calibration target area, and the difference between the coordinate axis angle of the constructed coordinate system and 90 degrees meets the preset angle threshold and the sum of the distances between the two axis points and the origin is the smallest.

[0142] Optionally, the area construction module 40 is also used to calculate the points on the left and right sides of the x-axis and y-axis of the coordinate axis among the four side centroids, obtain the x-axis left point located to the left of the x-axis and farthest from the x-axis, the x-axis right point located to the right of the x-axis and farthest from the x-axis, obtain the y-axis left point located to the left of the y-axis and farthest from the y-axis, and the y-axis right point located to the right of the y-axis and farthest from the y-axis; calculate the coordinate axis vector according to the corner point of the coordinate system; calculate the straight lines passing through the x-axis left point, the x-axis right point, the y-axis left point and the y-axis right point and parallel to the coordinate axis according to the coordinate axis vector, and construct four straight lines surrounding the calibration target area.

[0143] The checkerboard calibration plate corner point screening device provided in this application utilizes the checkerboard calibration plate corner point screening method described in the aforementioned embodiment, resolving the technical issue of extracting checkerboard corner points in complex environments, where corner points that are not located on the checkerboard calibration plate are easily extracted. Compared to the prior art, the checkerboard calibration plate corner point screening device provided in this application achieves the same beneficial effects as the checkerboard calibration plate corner point screening method described in the aforementioned embodiment. Other technical features of the checkerboard calibration plate corner point screening device are the same as those disclosed in the aforementioned embodiment and are not further detailed here.

[0144] The present application provides a checkerboard calibration plate corner point screening device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the checkerboard calibration plate corner point screening method in the above-mentioned embodiment one.

[0145] Reference below Figure 9, which shows a schematic structural diagram of a checkerboard calibration plate corner point screening device suitable for implementing embodiments of the present application. The checkerboard calibration plate corner point screening device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9 The checkerboard calibration plate corner point screening device shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

[0146] like Figure 9 As shown, the checkerboard calibration plate corner point screening device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the checkerboard calibration plate corner point screening device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the checkerboard calibration plate corner point screening device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a checkerboard calibration plate corner point screening device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0147] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0148] The checkerboard calibration plate corner point screening device provided in this application utilizes the checkerboard calibration plate corner point screening method described in the aforementioned embodiment, resolving the technical issue of extracting checkerboard corner points in complex environments, where corner points that are not on the checkerboard calibration plate are easily extracted. Compared to the prior art, the checkerboard calibration plate corner point screening device provided in this application achieves the same beneficial effects as the checkerboard calibration plate corner point screening method described in the aforementioned embodiment. Other technical features of this checkerboard calibration plate corner point screening device are the same as those disclosed in the aforementioned embodiment and are not further detailed here.

[0149] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0150] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0151] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the checkerboard calibration plate corner point screening method in the above embodiment.

[0152] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0153] The computer-readable storage medium may be included in the checkerboard calibration plate corner point screening device; or may exist independently without being assembled into the checkerboard calibration plate corner point screening device.

[0154] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0155] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0156] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0157] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned checkerboard calibration plate corner point screening method. This computer-readable storage medium can address the technical issue of easily extracting corner points that are not located on the checkerboard calibration plate when extracting checkerboard corner points in complex environments. Compared to the prior art, the computer-readable storage medium provided in this application offers the same beneficial effects as the checkerboard calibration plate corner point screening method provided in the aforementioned embodiment, and will not be further elaborated here.

[0158] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for selecting corner points of a checkerboard calibration plate, characterized in that: The method for selecting corner points of a chessboard calibration plate comprises: Use the corner point detection algorithm to obtain all corner points of the calibration plate on the chessboard to be detected, and construct the first corner point set; Performing feature analysis on each corner point in the first corner point set, comparing a feature value of each corner point with a first preset threshold, and retaining corner points within the first preset threshold to construct a second corner point set; Filter out the calibration target area of ​​the black squares on the checkerboard calibration board; Constructing a circumscribed polygon surrounding the calibrated target area; It is determined whether the corner points in the second corner point set are located inside the circumscribed polygon, and the corner points located inside the circumscribed polygon are retained to construct a valid calibration point set.

2. The method for selecting corner points of a checkerboard calibration plate according to claim 1, wherein: The step of performing feature analysis on each corner point in the first corner point set, comparing the feature value of each corner point with a first preset threshold, and retaining corner points within the first preset threshold range to construct a second corner point set comprises: Perform image segmentation on the chessboard to obtain a binary image; Performing feature analysis on each corner point in the first corner point set using the binary image to determine a pairing rate of pixels and a difference rate of black pixels within a corner point region; The second corner point set is constructed by retaining the corner points whose pixel pairing rate and black pixel difference rate are both within the first preset threshold range.

3. The method for selecting corner points of a checkerboard calibration plate according to claim 2, characterized in that: After the step of segmenting the chessboard into a binary image, the method further comprises: The checkerboard is segmented using Otsu automatic thresholding to obtain a binary image; A point with a pixel value of 255 in the binary image is defined as a foreground target point; Converting all of the foreground object points in the binary image into a foreground object area; The foreground target area is stored in the form of row coordinates of adjacent target point pixels, start coordinates of adjacent target point pixels, and end coordinates of adjacent target point pixels.

4. The method for selecting corner points of a checkerboard calibration plate according to claim 1, wherein: The step of screening out the calibration target area of ​​the black squares on the checkerboard calibration plate comprises: Performing morphological dilation processing on the image area of ​​the checkerboard calibration plate with a rectangular structure element of a first preset size to obtain a first dilated area; Performing a connected domain analysis on the first expansion area to screen out the largest white connected area; Filling holes in the white connection area to obtain a hole-filled area inside the white connection area; Performing a difference operation on the hole filling area and the white connection area to obtain a difference area; Performing morphological dilation processing on the difference region using a rectangular structure element of a second preset size to obtain a second dilated region; A connected domain analysis is performed on the second expansion area, and the black connected area with the largest area is screened out to obtain the calibration target area of ​​the black grid.

5. The method for selecting corner points of a checkerboard calibration plate according to claim 4, characterized in that: The step of constructing a circumscribed polygon surrounding the calibration target area comprises: Calculate the coordinates of the horizontal circumscribed rectangle of the calibration target area; Selecting a boundary point set whose coordinate corresponding values ​​are respectively equal to the coordinates of the horizontal circumscribed rectangle from the point set represented by the second dilated area, and calculating the centroid points of the four sides of the boundary point set; Constructing a coordinate system of the calibration target area; Generate four straight lines surrounding the calibration target area according to the coordinate system and the four-side centroid points; The intersection points of four straight lines surrounding the calibration target area are calculated as four side points, and the four side points are extended to the surrounding areas by a preset distance according to the direction of the coordinate system to construct the circumscribed polygon.

6. The method for selecting corner points of a checkerboard calibration plate according to claim 5, characterized in that: The step of constructing the coordinate system of the calibration target area includes: Calculating the coordinates of the center of gravity of the second corner point set; Select four coordinate system corner points closest to the coordinates of the center of gravity point from the second corner point set; Three corner points are selected from the four corner points of the coordinate system as the origin, x-axis point and y-axis point of the coordinate system of the calibration target area, and the difference between the coordinate axis angle of the coordinate system and 90 degrees meets the preset angle threshold and the sum of the distances between the two axis points and the origin is the smallest.

7. The method for selecting corner points of a checkerboard calibration plate according to claim 6, wherein: The step of generating four straight lines surrounding the calibration target area according to the coordinate system and the four-side centroid points includes: Calculate the points on the left and right sides of the x-axis and y-axis of the coordinate axis among the four-side centroid points, obtain the x-axis left point located on the left side of the x-axis and farthest from the x-axis, the x-axis right point located on the right side of the x-axis and farthest from the x-axis, obtain the y-axis left point located on the left side of the y-axis and farthest from the y-axis, and the y-axis right point located on the right side of the y-axis and farthest from the y-axis; Calculate the coordinate axis vector according to the corner point of the coordinate system; According to the coordinate axis vector, straight lines passing through the left point of the x-axis, the right point of the x-axis, the left point of the y-axis and the right point of the y-axis and parallel to the coordinate axis are calculated respectively to construct four straight lines surrounding the calibration target area.

8. A device for selecting corner points of a chessboard calibration plate, characterized in that: The device comprises: A corner point acquisition module is used to use a corner point detection algorithm to obtain all corner points of the calibration plate on the chessboard to be detected and construct a first corner point set; A primary optimization module, configured to perform feature analysis on each corner point in the first corner point set, compare a feature value of each corner point with a first preset threshold, and retain corner points within the first preset threshold to construct a second corner point set; The area screening module is used to screen out the calibration target area of ​​the black squares on the checkerboard calibration board; A region construction module, used to construct a circumscribed polygon surrounding the calibration target region; The secondary optimization module is used to determine whether the corner points in the second corner point set are located inside the circumscribed polygon, and retain the corner points located inside the circumscribed polygon to construct a valid calibration point set.

9. A checkerboard calibration plate corner point screening device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the checkerboard calibration plate corner point screening method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the checkerboard calibration plate corner point screening method according to any one of claims 1 to 7 are implemented.

Images