A method for extracting the center of a circular hole on a calibration plate

Abstract

This invention relates to a method for extracting the center of a circular hole on a calibration plate, comprising: projecting a point cloud after planar fitting processing onto an image plane to generate a point cloud image, wherein the point cloud is the point cloud of the calibration plate acquired by a lidar; and performing morphological processing on the point cloud image to extract the center of the circular hole on the calibration plate. This method is highly versatile, requires no excessive parameter settings, and is not limited by the size of the calibration plate. Compared with previous traditional methods, it yields a more accurate center.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for extracting the center of a circular hole on a calibration plate. Background Technology

[0002] Current methods for extracting the centers of circular holes on a calibration plate using point cloud data calculate the positions of the four centers based on the depth of the fitted planar point cloud and the actual physical dimensions of the calibration plate. However, this method has the following drawbacks:

[0003] 1. Poor stability; it cannot obtain the correct center position when the fitted plane is incomplete, thus causing calibration failure.

[0004] 2. It has poor versatility and requires manual setting of appropriate parameters and prior measurement of the calibration plate dimensions;

[0005] 3. The results are not accurate enough because there are multiple approximations in the calculation process, which can lead to some deviation in the results. Summary of the Invention

[0006] To address at least some of the aforementioned problems in the prior art, the present invention provides a method for extracting the center of a circular hole on a calibration plate, comprising:

[0007] The point cloud after plane fitting is projected onto the image plane to generate a point cloud image, where the point cloud is the point cloud of the calibration board collected by the lidar;

[0008] Morphological processing is performed on the point cloud image to extract the center of the circular hole on the calibration plate.

[0009] Furthermore, the process of projecting the plane-fitted point cloud onto the image plane to generate a point cloud image includes:

[0010] Determine the maximum and minimum values ​​of the x and y coordinates in the point cloud;

[0011] The number of rows in the image is determined based on the range of the y-coordinate;

[0012] Determine the number of columns in the image based on the range of the x-coordinates, and create a binary image with all 0s.

[0013] Convert point cloud coordinates to image coordinates; and

[0014] Change the pixel value of the point at the corresponding coordinate position in the image to 1.

[0015] Furthermore, the expressions for the number of rows and columns of the image are:

[0016] Img_rows=ceil((Ymax–Ymin) / cellSize),

[0017] Img_cols=ceil((Xmax–Xmin) / cellSize),

[0018] Where Img_rows is the number of image rows, Img_cols is the number of image columns, cellsize is the cell length of the raster, which is 0.001 here, and ceil indicates rounding up.

[0019] Furthermore, the expression for the image coordinates is:

[0020] Yimg=floor((Ymax–Ypt) / cellSize)

[0021] Ximg=floor((Xpt–Xmin) / cellSize)

[0022] Where Ypt is the Y-axis coordinate of the point cloud, Xpt is the X-axis coordinate of the point cloud, Yimg represents the Y-coordinate of the point in the image, Ximg represents the X-coordinate of the point in the image, and floor represents rounding down.

[0023] Furthermore, the morphological processing of the point cloud image to extract the center of the circular hole on the calibration plate includes:

[0024] Inflate the point cloud using a circular kernel;

[0025] Extract connected components, where a connected component is an image region consisting of foreground pixels with the same pixel value and adjacent positions; and

[0026] Filter connected regions, remove those with outer contours and areas smaller than a set value, and find the center of the smallest bounding rectangle for each connected region.

[0027] Furthermore, the process of dilating the point cloud using a circular kernel includes:

[0028] The circular kernel is compared with every point in the image through translation. If there are non-zero points within the area covered by the kernel, all pixel values ​​in the covered area are replaced with the maximum value within that area. The mathematical formula for dilation is:

[0029] dst(x,y)=max(src(x+x1,y+y1)),(x1,y1):element(x1,y1)! =0,

[0030] Where dst(x,y) represents the value of the pixel at position (x,y) after processing, max represents taking the maximum value in the set, src represents the input set of pixels, and element(x1,y1) represents the morphological kernel, which is the set of all pixels that need to be judged.

[0031] Furthermore, the extraction of connected components includes:

[0032] Use the contour extraction function findContours to extract connected regions.

[0033] Furthermore, finding the center of the minimum bounding rectangle for each connected region includes:

[0034] By traversing the coordinates of all pixels within the connected region and determining the maximum and minimum values ​​of x and y respectively, the coordinates of the top-left vertex of the minimum bounding rectangle are (xmin, ymin), and the coordinates of the bottom-right vertex are (xmax, ymax). Therefore, the center of the minimum bounding rectangle is ((xmax+xmin) / 2, (ymax+ymin) / 2).

[0035] Furthermore, it also includes determining the number of circle centers. If the number of circle centers meets the condition, the transformation matrix is ​​obtained; otherwise, the positions of the remaining circle centers are estimated based on the existing circle centers.

[0036] Furthermore, estimating the position of the remaining center based on the existing center includes:

[0037] If there are 3 centers, and the centers in the calibration plate are known to be located at the four corners of the square, then a parallelogram can be drawn using the known coordinates of the centers to fill in the missing centers.

[0038] The present invention has at least the following beneficial effects: The method for extracting the center of a circular hole on a calibration plate disclosed in the present invention has high stability and can still accurately extract the center even when the fitting plane is partially damaged. At the same time, it does not require measuring the size of the calibration plate. The method has strong versatility, does not require setting too many parameters, and is not limited by the size of the calibration plate. Compared with the previous traditional method, the obtained center is more accurate. Attached Figure Description

[0039] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the various embodiments of the present invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope.

[0040] Figure 1 A schematic diagram of a commonly used calibration plate is shown;

[0041] Figure 2 The flowchart of existing camera and lidar joint calibration is shown;

[0042] Figure 3 A flowchart illustrating a joint calibration of a camera and lidar based on the extraction of the center of a circular hole on an improved calibration plate, according to an embodiment of the present invention, is shown.

[0043] Figure 4 A schematic diagram of an image generated by projecting a point cloud onto an image plane according to an embodiment of the present invention is shown;

[0044] Figure 5 A schematic diagram of point cloud expansion according to an embodiment of the present invention is shown;

[0045] Figure 6 A schematic diagram of a circular core used for expansion according to an embodiment of the present invention is shown;

[0046] Figure 7 A schematic diagram illustrating the determination of the center of a circle based on four connected regions according to an embodiment of the present invention is shown;

[0047] Figure 8 A schematic diagram of an image generated by projecting a defective point cloud onto an image plane according to an embodiment of the present invention is shown;

[0048] Figure 9 A schematic diagram of a point cloud after dilation with defects is shown according to an embodiment of the present invention; and

[0049] Figure 10 A schematic diagram is shown illustrating how, according to an embodiment of the present invention, the exact position of a circle is estimated using three previously determined center positions. Detailed Implementation

[0050] It should be noted that the components in the accompanying drawings may be shown exaggerated for illustrative purposes and may not be to scale.

[0051] In this invention, the various embodiments are merely intended to illustrate the solutions of the invention and should not be construed as limiting.

[0052] In this invention, unless otherwise specified, the quantifiers “a” and “one” do not exclude scenarios involving multiple elements.

[0053] It should also be noted that, in the embodiments of the present invention, only a portion of the parts or components may be shown for clarity and simplicity. However, those skilled in the art will understand that, under the teachings of the present invention, the required parts or components can be added as needed for specific scenarios.

[0054] It should also be noted that within the scope of this invention, the terms "same", "equal", and "equal to" do not mean that the two values ​​are absolutely equal, but allow for a certain reasonable error. In other words, the terms also cover "substantially the same", "substantially equal", and "substantially equal to".

[0055] It should also be noted that in the description of this invention, the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not explicitly or implicitly suggest that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0056] Furthermore, the numbering of the steps in the methods of the present invention does not limit the execution order of the method steps. Unless otherwise specified, the method steps may be executed in different orders.

[0057] Figure 1 A schematic diagram of a commonly used calibration plate is shown. Figure 2 A flowchart of existing camera and lidar joint calibration is shown.

[0058] The principle of joint calibration of the camera and LiDAR is to find several sets of common points between the two sensors to determine the transformation matrix and complete the calibration. Common calibration boards include... Figure 1 As shown, the calibration board has a checkerboard pattern in the center, with four circular holes on either side of the checkerboard. After the camera and LiDAR captured data from the calibration board, analysis began.

[0059] like Figure 2 As shown, the camera and LiDAR acquire data; the camera acquires image data, and the LiDAR acquires point cloud data. The center of the calibration board is determined based on the image data. Specifically, the range of the checkerboard is determined based on the intersections of the checkerboard grid. Combined with the known relative positions of the holes and the checkerboard grid, the coordinates of the centers of the four holes in the image can be determined. Finally, the transformation matrix is ​​calculated using the centers determined from the image data and the centers determined from the point cloud data.

[0060] Point cloud data is used for computational analysis to determine the center of the circular holes on the calibration plate. The point cloud data processing involves filtering, preprocessing, and plane fitting to finally determine the center. Since the calibration plate is a plane with four circular holes, plane fitting yields a planar dataset containing all four holes (points outside this plane are removed). Traditionally, the actual dimensions of the calibration plate and the relative positions of the holes are known, allowing the determination of the center within the planar dataset. However, unevenness of the calibration plate or incomplete data acquisition can affect the center determination, rendering the calibration process invalid.

[0061] This invention improves the method for determining the center of the circular hole on the calibration plate using point cloud data collected by lidar in the camera-lidar joint calibration method.

[0062] Figure 3 A flowchart illustrating joint calibration of a camera and lidar based on the extraction of the center of a circular hole on an improved calibration plate, according to an embodiment of the present invention, is shown.

[0063] like Figure 3 As shown, the point cloud data is fitted to a plane, then mapped to the image plane, and the point cloud is converted into image data to generate an image. Using morphological correlation techniques in image processing to perform morphological processing on the image, the center position of the circular hole on the calibration plate can be accurately extracted, thus completing the calibration and greatly improving the stability and versatility of the calibration algorithm.

[0064] Plane fitting utilizes the Random Access Scale (RANSAC) algorithm to segment a plane, retaining only the point cloud within that plane. A rotation matrix is ​​then used to rotate the plane to the XOY plane for subsequent image mapping. Assuming the fitted point cloud plane is rectangular, the length and width of the rotated XOY plane are parallel to the X and Y coordinate axes. Mapping the point cloud data to an image plane after plane fitting facilitates image conversion. Image processing techniques are then used to extract circle centers. This image plane is independent of the camera-captured image. After extracting the circle centers, the coordinates in the point cloud are determined based on the mapping relationship; these coordinates are then used to calculate the transformation matrix.

[0065] 1. Project the point cloud after plane fitting onto the image plane to generate an image. The specific process is as follows:

[0066] (1) Determine the maximum (Xmax, Ymax) and minimum (Xmin, Ymin) values ​​of the x and y coordinates in the point cloud. Based on the range of the y coordinates, determine the number of rows in the image. Based on the range of the x coordinates, determine the number of columns in the image. Create a binary image with all values ​​equal to 0.

[0067] Number of rows Img_rows=ceil((Ymax–Ymin) / cellSize)

[0068] Number of columns Img_cols=ceil((Xmax–Xmin) / cellSize)

[0069] Note: cellsize is the cell length of the grid, which is 0.001 here. ceil means round up.

[0070] (2) Convert the point cloud coordinates to image coordinates, that is:

[0071] Yimg=floor((Ymax–Ypt) / cellSize)

[0072] Ximg=floor((Xpt–Xmin) / cellSize)

[0073] Where Ypt is the Y-axis coordinate of the point cloud, Xpt is the X-axis coordinate of the point cloud, Yimg represents the Y-coordinate of the point in the image, Ximg represents the X-coordinate of the point in the image, and floor represents rounding down.

[0074] (3) Change the pixel value of the point at the corresponding coordinate position in the image to 1. The generated image is as follows: Figure 4 As shown.

[0075] 2. Morphological processing of point cloud images is performed using morphological techniques in image processing. The specific process is as follows:

[0076] (1) Since the point cloud is relatively sparse in the image, we first dilate the point cloud using a circular kernel. The dilated image is as follows: Figure 5 As shown, the circular core used for expansion is as follows Figure 6 As shown.

[0077] The dilation method involves translating a circular kernel and comparing it with every point in the image. If there are non-zero points within the range covered by the kernel, all pixel values ​​in the kernel-covered portion are replaced with the maximum value of that range.

[0078] The mathematical formula for expansion is as follows:

[0079] dst(x,y) = max(src(x+x1,y+y1)), (x1,y1):element(x1,y1) ! = 0, where dst(x,y) represents the value of the pixel at position (x,y) after processing, max represents taking the maximum value in the set, src represents the set of input pixels, and element(x1,y1) represents the morphological kernel, which is the set of all pixels that need to be judged.

[0080] (2) Extract connected components.

[0081] Connected regions are image areas composed of foreground pixels with the same pixel value and adjacent positions. The extraction is specifically implemented using OpenCV's contour extraction function `findContours`, which works by using a boundary tracing algorithm to iterate through each pixel value in the image. Therefore, in this approach, all connected regions except the four near-circular regions also need to be removed.

[0082] (3) Filter connected regions, removing those with outer contours and areas smaller than a set value, and find the center of the minimum bounding rectangle for each connected region. The specific process for finding the center of the minimum bounding rectangle for a connected region is as follows:

[0083] By iterating through the coordinates of all pixels within the connected region and finding the maximum and minimum values ​​of x and y, the coordinates of the top-left vertex of the minimum bounding rectangle are (xmin, ymin), and the coordinates of the bottom-right vertex are (xmax, ymax). Therefore, the center of the minimum bounding rectangle is ((xmax+xmin) / 2, (ymax+ymin) / 2).

[0084] 3. Determine the number of circle centers. If the number of circle centers meets the condition, proceed to the step of obtaining the transformation matrix; otherwise, estimate the position of the remaining circle centers based on the existing circle centers.

[0085] If there are 4 centers, then all centers have been obtained, such as... Figure 7 As shown.

[0086] If the number of center points is less than 4, for example, 3, the location of the missing connected region needs to be determined based on the positions of the existing connected regions (i.e., top left, top right, etc.). Since the center points in the calibration board are known to be distributed at the four corners of the square, a parallelogram is drawn using the known coordinates of the center points to fill in the missing center points. Ideally, the four center points would be distributed at the four vertices of the square, but the algorithm extracts center points with some deviation; the line connecting the four center points does not form a square but is closer to a parallelogram. Therefore, a parallelogram is used to fill in the missing center points.

[0087] like Figure 8 As shown, some data in the collected point cloud will be incomplete after fitting to the plane. Figure 9 As shown, the defect persists even after morphological dilation, and the central connected region in the upper left corner cannot be extracted. Figure 10 As shown, a parallelogram is drawn based on the three already determined center positions to estimate the position of the missing center.

[0088] While some embodiments of the present invention have been described in this application, those skilled in the art will understand that these embodiments are merely illustrative. Numerous variations, alternatives, and improvements will arise in those skilled in the art under the teachings of this invention without departing from its scope. The appended claims are intended to define the scope of the invention and thereby cover methods and structures within the scope of the claims themselves and their equivalents.

Claims

1. A method for extracting the center of a circle on a calibration plate, characterized by, include: The point cloud after plane fitting is projected onto the image plane to generate a point cloud image, where the point cloud is the point cloud of the calibration board collected by the lidar; as well as Morphological processing is performed on the point cloud image to extract the center of the circular hole on the calibration plate; and Determine the number of circle centers. If the number of circle centers meets the condition, proceed to the step of obtaining the transformation matrix. Otherwise, estimate the position of the remaining circle centers based on the existing circle centers. The estimation of the position of the remaining circle centers based on the existing circle centers includes: if there are 3 circle centers, and it is known that the circle centers in the calibration plate are distributed at the four corners of the square, then use the known circle center coordinates to draw a parallelogram to fill in the missing circle centers. Projecting the point cloud after planar fitting onto an image plane to generate a point cloud image includes: determining the maximum and minimum values ​​of the x and y coordinates in the point cloud; determining the number of rows in the image based on the range of the y coordinates; determining the number of columns in the image based on the range of the x coordinates, and creating a binary image with all zeros; converting the point cloud coordinates to image coordinates; and changing the pixel values ​​of the corresponding coordinate positions in the image to 1; the expressions for the number of rows and columns in the image are: Img_rows = ceil((Ymax – Ymin) / cellSize), Img_cols = ceil((Xmax – Xmin) / cellSize), Where Img_rows is the number of image rows, Img_cols is the number of image columns, cellsize is the cell length of the raster, which is 0.001 here, and ceil indicates rounding up; The morphological processing of the point cloud image to extract the center of the circular hole on the calibration plate includes: Inflating the point cloud using a circular kernel involves: translating the circular kernel and comparing it with each point in the image; if there are non-zero points in the image within the range covered by the kernel, then all pixel values ​​in the portion covered by the kernel are replaced with the maximum value of that range. Extract connected components, where a connected component is an image region consisting of foreground pixels with the same pixel value and adjacent positions; and Filter connected regions, remove those with outer contours and areas smaller than a set value, and find the center of the smallest bounding rectangle of each remaining connected region as the center of the circle.

2. The method of claim 1, wherein, The expression for the image coordinates is: Yimg = floor((Ymax – Ypt) / cellSize) Ximg = floor((Xpt – Xmin) / cellSize) Where Ypt is the Y-axis coordinate of the point cloud, Xpt is the X-axis coordinate of the point cloud, Yimg represents the Y-coordinate of the point cloud in the image, Ximg represents the X-coordinate of the point cloud in the image, and floor represents rounding down.

3. The method of claim 1, wherein, The mathematical formula for expansion is: dst(x,y) = max(src(x+x1, y+y1)), (x1,y1):element(x1,y1)!=0, Where dst(x,y) represents the value of the pixel at position (x,y) after processing, max represents taking the maximum value in the set, src represents the input set of pixels, and element(x1,y1) represents the morphological kernel, which is the set of all pixels that need to be judged.

4. The method according to claim 1, characterized in that, The extraction of connected components includes: Use the contour extraction function findContours to extract connected regions.

5. The method according to claim 1, characterized in that, Finding the center of the minimum bounding rectangle for each connected region includes: By traversing the coordinates of all pixels within the connected region and determining the maximum and minimum values ​​of x and y respectively, the coordinates of the top-left vertex of the minimum bounding rectangle are (xmin, ymin), and the coordinates of the bottom-right vertex are (xmax, ymax). Therefore, the center of the minimum bounding rectangle is ((xmax+xmin) / 2, (ymax+ymin) / 2).

Images