Camera calibration method and device based on general calibration plate, equipment and storage medium

Abstract

The application relates to the technical field of image processing, in particular to a camera calibration method and device based on a general calibration board, equipment and a storage medium, wherein the method can avoid too large feature point extraction deviation caused by the inclination angle of the general calibration board or picture distortion, so as to improve the feature point extraction precision, through a circle fitting method, fitting of a circular mark point, and extraction of a fitted circle center; the ID and the main direction of the general calibration board are identified through the sorting of the coding area, so that the coding area and the control area are sorted uniquely, and the internal parameters of the optical camera are calibrated according to the circle center sorting result, so as to improve the calibration precision; meanwhile, a low-cost calibration field is established by using multiple general calibration boards with different IDs, the external parameters of the optical camera are calibrated, so that the internal and external parameters of the optical camera are obtained, and the technical problem of low calibration precision of the internal and external parameters of the camera is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a camera calibration method, apparatus, device, and storage medium based on a universal calibration board. Background Technology

[0002] Camera calibration is a fundamental task in computer vision, aiming to recover accurate camera intrinsic and extrinsic parameters. Although existing camera calibration methods have achieved good results, there are still significant limitations in calibration boards and extrinsic parameter estimation.

[0003] Currently, the estimation of camera intrinsic and extrinsic parameters is performed using calibration boards, but calibration boards and extrinsic parameter estimation have significant limitations. The traditional Zhang Zhengyou algorithm uses a planar board to obtain 2D-to-3D correction, but traditional calibration boards cannot adapt to complex and changing environments; therefore, the pattern shape of the traditional calibration board affects the accuracy of intrinsic parameter calibration. Furthermore, accurate extrinsic parameter calibration methods for multiple cameras are still lacking. For panoramic cameras, suitable pose estimation algorithms and data acquisition require careful planning in advance, which is usually very time-consuming. A common solution is to establish a 3D control field, but this is costly. Therefore, solving the low calibration accuracy of current camera extrinsic and extrinsic parameters has become an urgent technical problem to be addressed. Summary of the Invention

[0004] The main objective of this invention is to provide a camera calibration method, apparatus, device, and storage medium based on a universal calibration board, aiming to solve the technical problem of low calibration accuracy of current camera internal and external parameters.

[0005] To achieve the above objectives, the present invention provides a camera calibration method based on a universal calibration board, the camera calibration method based on the universal calibration board comprising:

[0006] A calibration image is acquired, and a universal calibration plate in the calibration image is detected based on a preset neural network to obtain circular marker points;

[0007] Based on a preset fitting method, the circular marker points are fitted with a circle center to obtain a fitted circle center. Based on the pattern of the circular marker points, a coding circle center is determined in the fitted circle center. The fitted circle center includes a coding circle center and a control circle center.

[0008] Based on the sorting result of the coded circle center, the ID and main direction in the general calibration board are determined, and based on the main direction and the coded circle center, the control circle center is expanded and sorted to obtain the target calibration board;

[0009] Based on the ID in the general calibration board, the sorting result of the fitted circle center in the target calibration board is determined, and based on the sorting result, the optical camera is calibrated to obtain the intrinsic parameters of the optical camera;

[0010] A target calibration field is constructed based on the target calibration plates with at least four different IDs, and the extrinsic parameters of the optical camera are estimated and optimized based on the target calibration field.

[0011] Further, the step of fitting the center of the circular marker point to obtain the fitted center based on the preset fitting method includes:

[0012] Based on the bounding box of the circular pattern detected by the preset network model, the circular pattern is cropped to obtain the circular sub-image;

[0013] Based on the binarization method, the outline of the circular sub-image is extracted to obtain an initial ellipse, and the initial ellipse is fitted based on a preset rotation matrix to obtain a fitted ellipse.

[0014] The fitted ellipse is optimized based on the rotation matrix and the minimization fitting method to obtain the center of the fitted circle.

[0015] Further, determining the ID and main direction in the universal calibration board based on the sorting result of the coded circle centers includes:

[0016] Based on the center of the coding region, a central subset is obtained, and based on the central subset, convex hull calculation is performed to obtain the minimum bounding box of the central subset.

[0017] Based on the homography matrix, the minimum bounding box is mapped to obtain a zero-rotation rectangle, and based on the zero-rotation rectangle, the coded circles in the central subset are sorted to obtain the coding matrix;

[0018] Based on a preset dictionary, the encoding matrix is ​​matched to determine the ID and main direction corresponding to the general calibration board.

[0019] Further, the step of expanding and sorting the control circles based on the main direction and the encoding circle center to obtain the target calibration board includes:

[0020] Based on the fitting circle center and the encoding circle center in the general calibration board, the control circle center is determined, and based on the control circle center, a subset of control regions is generated;

[0021] Based on the main direction and cyclic search algorithm corresponding to the encoding matrix, the four neighborhoods of the encoding circle center are searched to obtain the unsorted control circle centers in the control area subset;

[0022] When an unsorted control center is found, it is marked and sorted to obtain sorted control centers, and then the sorted control centers are cleared in the control area subset.

[0023] Furthermore, before detecting the universal calibration board in the calibration image based on a preset neural network to obtain circular marker points, the process further includes:

[0024] Based on the data collection and annotation of images captured by the camera, real data is obtained;

[0025] Virtual data is generated based on a preset data augmentation method.

[0026] Based on the dataset composed of real and virtual data, network training is performed to obtain the preset neural network.

[0027] Furthermore, the step of detecting the universal calibration board in the calibration image based on a preset neural network to obtain circular marker points includes:

[0028] Based on the preset neural network, the calibration image is detected to obtain a universal calibration board and at least one circular pattern in the universal calibration board;

[0029] Based on the preset training model, erroneous results in the general calibration board and the circular pattern are filtered out to obtain an unsorted circular pattern, which is used as the circular marker point.

[0030] Furthermore, before acquiring the calibration image, the process also includes:

[0031] The coding region is obtained based on a coding matrix of at least one circular marker point;

[0032] Based on preset scale information, at least two scale markers are set outside the coding region to obtain the control region;

[0033] A general calibration board is obtained based on the encoding region and the control region.

[0034] Furthermore, to achieve the above objectives, the present invention also provides a camera calibration device based on a universal calibration plate. The camera calibration device based on a universal calibration plate includes: a universal calibration plate detection module, used to acquire a calibration image, and based on a preset neural network, detect the universal calibration plate in the calibration image to obtain circular marker points; a circle center fitting module, used to perform circle center fitting on the circular marker points based on a preset fitting method to obtain a fitted circle center, and based on the pattern of the circular marker points, determine an encoded circle center in the fitted circle center, wherein the fitted circle center includes an encoded circle center and a control circle center; and a circle center sorting module, used to sort the circle center based on the pattern of the circular marker points. The system uses the sorting results of the coded circle centers to determine the ID and main direction in the general calibration board, and expands and sorts the control circle centers based on the main direction and the coded circle centers to obtain the target calibration board; the intrinsic parameter calibration module is used to determine the sorting results of the fitted circle centers in the target calibration board based on the ID in the general calibration board, and calibrates the optical camera based on the sorting results to obtain the intrinsic parameters of the optical camera; the extrinsic parameter calibration module is used to build a target calibration field based on the target calibration boards with at least four different IDs, and estimate and optimize the extrinsic parameters of the optical camera based on the target calibration field.

[0035] Furthermore, to achieve the above objectives, the present invention also provides a method for establishing a three-dimensional control field, the method comprising:

[0036] A virtual cube is constructed based on at least four universal calibration plates, wherein the virtual cube includes at least four faces, and the universal calibration plates are located in each face;

[0037] Based on the image acquisition device, the calibration images of each general calibration board are obtained, and the circular markers in the calibration images are extracted and identified to determine the ID of each general calibration board;

[0038] Based on the ID of each universal calibration board, the main direction of the circular marker point in each universal calibration board is obtained, and a three-dimensional control field is constructed based on the sorting result of the circular marker point along the main direction.

[0039] Furthermore, to achieve the above objectives, the present invention also provides a camera calibration device based on a universal calibration board. The camera calibration device based on the universal calibration board includes a processor, a memory, and a camera calibration program based on the universal calibration board stored in the memory and executable by the processor. When the camera calibration program based on the universal calibration board is executed by the processor, it implements the steps of the camera calibration method based on the universal calibration board as described above.

[0040] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a camera calibration program based on a universal calibration board, wherein when the camera calibration program based on the universal calibration board is executed by a processor, the steps of the camera calibration method based on the universal calibration board as described above are implemented.

[0041] This invention provides a camera calibration method based on a universal calibration board. The method acquires a calibration image; detects the universal calibration board in the calibration image using a preset neural network to obtain circular marker points; fits the center of the circular marker points using a preset fitting method to obtain a fitted center, and determines an coded center within the fitted center based on the pattern of the circular marker points, wherein the fitted center includes an coded center and a control center; determines the ID and principal direction in the universal calibration board based on the sorting result of the coded centers, and expands and sorts the control centers based on the principal direction and the coded centers to obtain a target calibration board; determines the sorting result of the fitted centers in the target calibration board based on the IDs in the universal calibration board, and calibrates the optical camera based on the sorting result to obtain the intrinsic parameters of the optical camera; constructs a target calibration field based on at least four target calibration boards with different IDs, and estimates and optimizes the extrinsic parameters of the optical camera based on the target calibration field. Through the above methods, this invention uses a preset neural network to detect circular markers in a universal calibration board. Circular markers are easier to identify and detect, resulting in higher detection accuracy. Simultaneously, by using a circle fitting method to fit the circular markers and extract the fitted circle centers, it avoids excessive deviations in feature point extraction caused by the tilt angle of the universal calibration board or image distortion, thereby improving feature point extraction accuracy. By sorting the encoding regions, the ID and main direction of the universal calibration board are identified, resulting in a unique sorting of the centers of each circular marker in the encoding and control areas. Based on the center sorting results, the intrinsic parameters of the optical camera are calibrated, improving calibration accuracy. Furthermore, by utilizing multiple universal calibration boards with different IDs, a low-cost calibration field is established to calibrate the extrinsic parameters of the optical camera, thereby obtaining the intrinsic and extrinsic parameters of the optical camera and solving the current technical problem of low calibration accuracy for camera intrinsic and extrinsic parameters. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the hardware structure of a camera calibration device based on a universal calibration board, as described in an embodiment of the present invention.

[0043] Figure 2 This is a flowchart illustrating the first embodiment of the camera calibration method based on a universal calibration board according to the present invention.

[0044] Figure 3 This is a flowchart illustrating the second embodiment of the camera calibration method based on a universal calibration board according to the present invention.

[0045] Figure 4 This is a flowchart illustrating the third embodiment of the camera calibration method based on a universal calibration board according to the present invention.

[0046] Figure 5 This is a schematic diagram illustrating a calibration board design example provided in an embodiment of the present invention.

[0047] Figure 6 This is a flowchart illustrating the first embodiment of the three-dimensional control field establishment method of the present invention.

[0048] Figure 7 This is a schematic diagram of the functional modules of the first embodiment of the camera calibration device based on a universal calibration board according to the present invention.

[0049] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0051] The camera calibration method based on a universal calibration board involved in the embodiments of the present invention is mainly applied to camera calibration equipment based on a universal calibration board. The camera calibration equipment based on the universal calibration board can be a PC, a portable computer, a mobile terminal, or other devices with display and processing functions.

[0052] Reference Figure 1 , Figure 1 This is a schematic diagram of the hardware structure of a camera calibration device based on a universal calibration board according to an embodiment of the present invention. In this embodiment, the camera calibration device based on the universal calibration board may include a processor 1001 (e.g., CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components; the user interface 1003 may include a display screen or an input unit such as a keyboard; the network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface); the memory 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk storage device, and optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0053] Those skilled in the art will understand that Figure 1The hardware structure shown does not constitute a limitation on camera calibration devices based on a general calibration board, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0054] Continue to refer to Figure 1 , Figure 1 The memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, and a camera calibration program based on a universal calibration board.

[0055] exist Figure 1 In this embodiment, the network communication module is mainly used to connect to the server and communicate with the server for data; while the processor 1001 can call the camera calibration program based on the universal calibration board stored in the memory 1005 and execute the camera calibration method based on the universal calibration board provided in this embodiment of the invention.

[0056] This invention provides a camera calibration method based on a universal calibration board.

[0057] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the camera calibration method based on a universal calibration board according to the present invention.

[0058] In this embodiment, the camera calibration method based on a universal calibration board includes the following steps:

[0059] Step S101: Obtain the calibration image; based on the preset neural network, detect the general calibration board in the calibration image to obtain circular marker points.

[0060] In this embodiment, at least one calibration image is acquired by an optical camera, and then the calibration image is input into a processing device. The calibration image is detected by a preset neural network to extract the universal calibration plate and the circular marker points in the universal calibration plate.

[0061] The preset neural network can be a YOLOv5 network. YOLOv5 delivers each batch of training data through a data loader, simultaneously augmenting the training data. The data loader performs three types of data augmentation: scaling, color space adjustment, and mosaic enhancement. This enables the recognition of small objects, thereby improving detection accuracy.

[0062] Step S102: Based on a preset fitting method, the circular marker points are fitted with a circle center to obtain a fitted circle center. Based on the pattern of the circular marker points, a coding circle center is determined in the fitted circle center. The fitted circle center includes a coding circle center and a control circle center.

[0063] In this embodiment, the circular markers are cropped based on the edges detected by the preset neural network, the outline of the circular markers is extracted, an initial ellipse is estimated, and a rotation matrix is ​​estimated to fit the ellipse. Then, the center and size of the fitted ellipse are iteratively optimized by minimizing the sum of squared distances between the fitted conic section and the outline. Finally, the optimized center is returned as the center of the fitted circle.

[0064] Specifically, the pattern in the universal calibration board can be circular, facilitating the extraction of the center point through circle fitting to obtain more accurate marker coordinates. The universal calibration board can be divided into two regions: an encoding region and a control region. The encoding region consists of 16 circular marker points, which can be encoded into a 4x4 matrix. The encoding region can include two types of circular marker points for easy identification and differentiation. The control region can surround the encoding region, providing scale information for the universal calibration board. The center of the circular marker points in the encoding region serves as the encoding center, and the center of the circular marker points in the control region serves as the control center.

[0065] Furthermore, the step of fitting the center of the circular marker point to obtain the fitted center based on the preset fitting method specifically includes:

[0066] Based on the bounding box of the circular pattern detected by the preset network model, the circular pattern is cropped to obtain the circular sub-image;

[0067] Based on the binarization method, the outline of the circular sub-image is extracted to obtain an initial ellipse, and the initial ellipse is fitted based on a preset rotation matrix to obtain a fitted ellipse.

[0068] The fitted ellipse is optimized based on the rotation matrix and the minimization fitting method to obtain the center of the fitted circle.

[0069] In this embodiment, all circular marker points are cropped along the detected bounding box, and the cropped sub-images are binarized. Then, the outermost contour of each sub-image is extracted to estimate an initial ellipse, and a rotation matrix is ​​estimated according to Fitzgibon to fit the symmetry constraint. Next, the center and size of the fitted ellipse are optimized by minimizing the sum of squared distances between the fitted conic section and the contour line. Finally, the optimized center is returned as the center of the fitted ellipse.

[0070] The rotation matrix is ​​a matrix that changes the direction of a vector without changing its magnitude when multiplied by it, while preserving its chirality. Combining minimum bounding circles and polygonal fitting curves yields a highly accurate fitted circle.

[0071] Step S103: Based on the sorting result of the coding circle center, determine the ID and main direction in the general calibration board, and based on the main direction and the coding circle center, expand and sort the control circle center to obtain the target calibration board;

[0072] In this embodiment, the binary matrix of the universal calibration board is obtained by sorting the 16 coding centers of the universal calibration board. Then, the corresponding marker is queried through the ArUco Marker dictionary to determine the unique ID of the universal calibration board. Since the ArUco Marker has rotation invariance, once the ID code of the coding area is determined, the main direction of the coding area can be known.

[0073] Specifically, the calibration board in the calibration image lacks a principal direction, meaning any direction can be used as the top-left direction of the current calibration board. This lack of rotation invariance means the identification of the current calibration board is not unique. In contrast, a sorted universal calibration board has a unique top-left direction and a uniquely determined sorting method. Regardless of rotation, its top-left direction and click order will not change, meaning it possesses a unique ID.

[0074] An ArUco marker is a binary squared marker consisting of a wide black border and an internal binary matrix that determines its ID. The black border facilitates rapid image detection, the binary encoding verifies the ID, and allows for the application of error detection and correction techniques. The size of the marker determines the size of the internal matrix. For example, a 4x4 marker consists of 16 bits. The Aruco module contains predefined dictionaries covering a range of dictionary sizes and marker sizes. An ArUco Marker dictionary is a collection of markers used in a specific application; it is simply a linked list of the binary encodings of each marker. The key properties of the dictionary are its size and the size of the marker: the size of the dictionary is the number of markers that make up the dictionary; the size of the marker is the size (number of bits) of these markers.

[0075] Step S104: Based on the ID in the general calibration board, determine the sorting result of the fitted circle center in the target calibration board, and calibrate the optical camera based on the sorting result to obtain the intrinsic parameters of the optical camera;

[0076] Step S105: Based on the target calibration boards with at least four different IDs, construct a target calibration field, and based on the target calibration field, estimate and optimize the extrinsic parameters of the optical camera.

[0077] In this embodiment, the actual dimensions between the control centers in the universal calibration board can be uniquely determined during the calibration board design. After determining the ID of the calibration board, the sorting direction and sorting result of the fitted center of each circular marker point on the marker board can be obtained by querying the dictionary. Based on the sorting result of the center, the intrinsic parameters of the optical camera can be calibrated to improve the calibration accuracy. At the same time, by using multiple universal calibration boards with different IDs, a low-cost calibration field can be established to calibrate the extrinsic parameters of the optical camera. Thus, the intrinsic and extrinsic parameters of the optical camera can be obtained.

[0078] This embodiment provides a camera calibration method based on a universal calibration board. The method acquires a calibration image; detects the universal calibration board in the calibration image using a preset neural network to obtain circular marker points; fits the center of the circular marker points using a preset fitting method to obtain a fitted center; and determines an coded center within the fitted center based on the pattern of the circular marker points, wherein the fitted center includes an coded center and a control center; determines the ID and principal direction in the universal calibration board based on the sorting result of the coded centers; expands and sorts the control centers based on the principal direction and the coded centers to obtain a target calibration board; determines the sorting result of the fitted centers in the target calibration board based on the IDs in the universal calibration board; calibrates the optical camera based on the sorting result to obtain the intrinsic parameters of the optical camera; constructs a target calibration field based on at least four target calibration boards with different IDs; and estimates and optimizes the extrinsic parameters of the optical camera based on the target calibration field. Through the above methods, this invention uses a preset neural network to detect circular markers in a universal calibration board. Circular markers are easier to identify and detect, resulting in higher detection accuracy. Simultaneously, by using a circle fitting method to fit the circular markers and extract the fitted circle center, it avoids excessive deviations in feature point extraction caused by the tilt angle of the universal calibration board or image distortion, thereby improving feature point extraction accuracy. By sorting the encoded regions, the ID and main direction of the universal calibration board are identified, thus determining the true dimensions between the centers of each circular marker in the control area. Based on these true dimensions, the intrinsic and extrinsic parameters of the optical camera are calibrated, improving calibration accuracy and solving the current technical problem of low calibration accuracy for camera intrinsic and extrinsic parameters.

[0079] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the camera calibration method based on a universal calibration board according to the present invention.

[0080] Based on the above Figure 2 In the illustrated embodiment, step S103 specifically includes:

[0081] Step S1031: Based on the center of the coding region, obtain a central subset, and based on the central subset, perform convex hull calculation to obtain the minimum bounding box of the central subset;

[0082] In this embodiment, the set of pattern centers is defined as Ω, and the subset of centers of the encoded region is defined as Ωo. To align the 2D centers with the 3D data, similar to the Zhang Zhengyou calibration algorithm, Ω is sorted according to the principal direction of B. If Ωc has fewer than 16 centers, the calibration board will not be verified or retained. If Ωc has 16 centers, its convex hull is calculated. The convex hull contains the four exterior angles of Ωc, so the angle between any two adjacent centers needs to be limited to [80°, 90°]. Then, the minimum bounding box of Ωo is obtained from the convex hull.

[0083] Step S1032: Based on the homography matrix, map the minimum bounding box to obtain a zero-rotation rectangle, and based on the zero-rotation rectangle, sort the coding centers in the central subset to obtain a coding matrix;

[0084] In this embodiment, the minimum bounding box is mapped to a zero-rotation rectangle using a homography matrix, and then Ωc can be sorted by simply comparing the uv coordinates. Since the number of Ωc is fixed, Ωc is divided into 4 groups according to ascending u coordinates, and each group is sorted according to ascending v coordinates. Ωc is sorted in a random principal direction, and the sorted center in the encoding area is defined as the point from which the binary matrix can be extracted once obtained.

[0085] Step S1033: Based on a preset dictionary, match the encoding matrix to determine the ID and main direction corresponding to the general calibration board.

[0086] In this embodiment, the ID of the calibration board is searched from the ArUco Marker dictionary to obtain the main direction of the encoding area, and finally the board is rotated and saved according to the main direction.

[0087] Reference Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the camera calibration method based on a universal calibration board according to the present invention.

[0088] Based on the above Figure 3 In the illustrated embodiment, step S103 further includes:

[0089] Step S1034: Based on the fitting circle center and the encoding circle center in the general calibration board, determine the control circle center, and generate a control region subset based on the control circle center;

[0090] Step S1035: Based on the main direction and cyclic search algorithm corresponding to the encoding matrix, search the four neighborhoods of the encoding circle center to obtain the unsorted control circle centers in the control area subset;

[0091] Step S1036: When the current unsorted control center is found, the unsorted control center is marked and sorted to obtain the sorted control center, and the sorted control center is cleared in the control area subset.

[0092] In this embodiment, the central subset of the control area can be defined as Ωc, which is obtained by expanding the sorted central set according to the main direction. First, assigning the central point to the empty set provides an initial expansion direction, and all sorted central points in Ωc are set to unvisited. Next is a loop structure where the algorithm checks the expansion capability of Ωc and returns a valid central point p(l); then, p(l) is expanded to four adjacent points to search for a central point in Ωc; if the search is successful, the index of the searched point is returned; then these central points are recorded and removed from Ωc. If all central points in Ωc have been visited and Ωc is not completely empty, then all centers in Ωc are set to unvisited. The loop ends when all points in Ωc are completely cleared.

[0093] Furthermore, based on the above Figure 2 In the illustrated embodiment, before detecting the universal calibration board in the calibration image based on a preset neural network to obtain circular marker points, the method further includes:

[0094] Based on the data collection and annotation of images captured by the camera, real data is obtained;

[0095] Virtual data is generated based on a preset data augmentation method.

[0096] Based on the dataset composed of real and virtual data, network training is performed to obtain the preset neural network.

[0097] In this embodiment, unlike traditional methods for detecting circular patterns, deep learning technology can be used to detect dots on a general calibration board. Both the training and evaluation datasets need to be labeled. To accomplish the training and evaluation tasks, a dataset consisting of real and virtual data is created for network training.

[0098] The real data was collected and labeled manually using various cameras, such as DSLR cameras, GoPro cameras, and fisheye cameras. To further enrich the dataset, Blender software was used to expand the data. Programs could be written in Blender to automatically generate virtual data.

[0099] In one exemplary implementation, the YOLOv5 network is trained to detect the complete universal calibration board and various different patterns on the universal calibration board. At the same time, the anchors of the YOLOv5 network are modified to make it easier to detect small targets. Then, VGG16 is trained to filter out false detections, further improving the accuracy.

[0100] Furthermore, based on the above Figure 2 In the illustrated embodiment, the step of detecting a universal calibration board in the calibration image based on a preset neural network to obtain circular marker points includes:

[0101] Based on the preset neural network, the calibration image is detected to obtain a universal calibration board and at least one circular pattern in the universal calibration board;

[0102] Based on the preset training model, erroneous results in the general calibration board and the circular pattern are filtered out to obtain an unsorted circular pattern, which is used as the circular marker point.

[0103] In this embodiment, the YOLOv5 network is trained to detect the complete general calibration board and various different patterns on it, including two circular patterns in the encoding region and one circular pattern in the control region. Since the three circular patterns are too small relative to the entire image, the anchors of the YOLOv5 network are modified to make them more effective at detecting small targets. Next, VGG16 is trained to filter out false detections, further improving accuracy.

[0104] Furthermore, based on the above Figure 2 In the illustrated embodiment, before acquiring the calibration image, the process further includes:

[0105] The coding region is obtained based on a coding matrix of at least one circular marker point;

[0106] Based on preset scale information, at least two scale markers are set outside the coding region to obtain the control region;

[0107] A general calibration board is obtained based on the encoding region and the control region.

[0108] In this embodiment, the calibration of the optical camera requires image acquisition based on a standard calibration board. The internal and external parameters of the optical camera are calculated based on the actual size of the standard calibration board and the pixel size of the image acquired by the optical camera. Before image acquisition, the calibration board needs to be designed to facilitate identification and size acquisition.

[0109] In one exemplary implementation, reference Figure 5 , Figure 5This is a schematic diagram illustrating a calibration board design example provided in an embodiment of the present invention. The general calibration board can be designed to be 500mm*500mm in size. The encoding area can consist of 16 circular patterns, each composed of two different types of circular markers. This area can be encoded as a 4*4 binary matrix. Similar to the design principles of ArUcoMarker, each binary matrix corresponds to a unique identifier in the dictionary. Therefore, each general calibration board has a unique ID code. Due to the rotation invariance of ArUcoMarker, once the ID code of the encoding area is determined, the main direction of the encoding area can be known. Eighty-four third-type circular markers are set outside the encoding area to form the control area. The control area only provides scale information, so it is designed to be large enough to cover more of the field of view. The distance between any two markers can be 48mm, and each complete calibration board can provide 100 centers with true dimensions and a unique arrangement order. The circular patterns on the calibration board are designed as rings, and the contents within the ring can be replaced with any other shape, such as a circle, triangle, or sector, to distinguish the encoding area markers from the control area markers.

[0110] Furthermore, this invention also provides a method for establishing a three-dimensional control field.

[0111] Reference Figure 6 , Figure 6 This is a flowchart illustrating the first embodiment of the three-dimensional control field establishment method of the present invention.

[0112] Step S201: Construct a virtual cube based on at least four universal calibration plates, wherein the virtual cube includes at least four faces, and the universal calibration plates are located in each face;

[0113] Step S202: Based on the image acquisition device, obtain the calibration images of each general calibration board, extract and identify the circular markers in the calibration images, and determine the ID of each general calibration board;

[0114] Step S203: Based on the ID of each universal calibration board, obtain the main direction of the circular marker point in each universal calibration board, and construct a three-dimensional control field based on the sorting result of the circular marker point along the main direction.

[0115] In this embodiment, a three-dimensional space can be designed and assembled using the aforementioned universal calibration plates. By identifying and calibrating each universal calibration plate in the three-dimensional space, a three-dimensional image can be fitted, and a three-dimensional control field can be constructed.

[0116] In one exemplary implementation, a virtual 1.8m*1.8m*1.8m cube is constructed, having four faces, with a universal calibration plate positioned in the center of each face. A laser scanner located in the center of the virtual cube is used to acquire laser point clouds. Circular patterns are then marked on the intensity image obtained by the laser scanner. A three-dimensional plane is then fitted from the point cloud of each circular pattern, and the intersection of this three-dimensional plane and the scanning center is used as the optimization center to establish a three-dimensional control field that eliminates scanning noise.

[0117] Among them, general algorithms and machines can accurately extract the center of the universal calibration board, and images of the image calibration board can be acquired using various devices and methods, such as point triangulation using a stereo camera, point scanning using a laser scanner, and point measurement using a total station.

[0118] Furthermore, embodiments of the present invention also provide a camera calibration device based on a universal calibration board.

[0119] Reference Figure 7 , Figure 7 This is a schematic diagram of the functional modules of the first embodiment of the camera calibration device based on a universal calibration board according to the present invention.

[0120] In this embodiment, the camera calibration device based on the universal calibration board includes:

[0121] The universal calibration board detection module 10 is used to acquire a calibration image and, based on a preset neural network, detect the universal calibration board in the calibration image to obtain circular marker points.

[0122] The circle center fitting module 20 is used to perform circle center fitting on the circular marker points based on a preset fitting method to obtain the fitted circle center, and to determine the coded circle center based on the pattern of the circular marker points in the fitted circle center, wherein the fitted circle center includes the coded circle center and the control circle center;

[0123] The center sorting module 30 is used to determine the ID and main direction in the general calibration board based on the sorting result of the coded center, and to expand and sort the control center based on the main direction and the coded center to obtain the target calibration board;

[0124] The intrinsic parameter calibration module 40 is used to determine the sorting result of the fitted circle centers in the target calibration board based on the ID in the general calibration board, and to calibrate the optical camera based on the sorting result to obtain the intrinsic parameters of the optical camera.

[0125] The extrinsic parameter calibration module 50 is used to construct a target calibration field based on at least four target calibration boards with different IDs, and to estimate and optimize the extrinsic parameters of the optical camera based on the target calibration field.

[0126] Furthermore, the universal calibration board detection module 10 specifically includes:

[0127] A circular pattern cropping unit is used to crop the circular pattern based on the bounding box of the circular pattern detected by the preset network model to obtain the circular sub-image;

[0128] An ellipse fitting unit is used to extract the contour of the circular sub-image based on a binarization method to obtain an initial ellipse, and to fit the initial ellipse based on a preset rotation matrix to obtain a fitted ellipse.

[0129] The fitting circle center acquisition unit is used to optimize the fitting ellipse based on the rotation matrix and the minimization fitting method to obtain the fitting circle center.

[0130] Furthermore, the center sorting module 30 specifically includes:

[0131] The central subset obtaining unit is used to obtain a central subset based on the coding circle center of the coding region, and to perform convex hull calculation based on the central subset to obtain the minimum bounding box of the central subset;

[0132] The encoding matrix obtaining unit is used to map the minimum bounding box based on the homography matrix to obtain a zero-rotation rectangle, and to sort the encoding circles in the central subset based on the zero-rotation rectangle to obtain the encoding matrix.

[0133] The encoding matrix matching unit is used to match the encoding matrix based on a preset dictionary to determine the ID and main direction corresponding to the general calibration board.

[0134] Furthermore, the center sorting module 30 specifically also includes:

[0135] The control region subset generation unit is used to determine the control circle center based on the fitting circle center and the encoding circle center in the general calibration board, and to generate a control region subset based on the control circle center;

[0136] The unsorted control circle center acquisition unit is used to search the four neighborhoods of the encoded circle center based on the main direction and cyclic search algorithm corresponding to the encoding matrix, and obtain the unsorted control circle centers in the control area subset.

[0137] The sorting control center acquisition unit is used to mark and sort the unsorted control centers when they are found, obtain sorted control centers, and clear the sorted control centers in the control area subset.

[0138] Furthermore, the camera calibration device based on the universal calibration board also includes a preset neural network acquisition module, which specifically includes:

[0139] The real data acquisition unit is used to obtain real data based on data collection and annotation of images captured by the camera;

[0140] The virtual data generation unit is used to generate virtual data based on a preset data augmentation method.

[0141] The preset neural network acquisition unit is used to train the network based on the dataset composed of real data and virtual data to obtain the preset neural network.

[0142] Furthermore, the general calibration board testing module 10 specifically also includes:

[0143] A calibration image detection unit is used to detect calibration images based on the preset neural network to obtain a universal calibration board and at least one circular pattern in the universal calibration board.

[0144] The circular marker acquisition unit is used to filter out erroneous results in the general calibration board and the circular pattern based on a preset training model, and obtain an unsorted circular pattern as the circular marker.

[0145] Furthermore, the camera calibration device based on the universal calibration board also includes a universal calibration board acquisition module, which specifically includes:

[0146] The coding region acquisition unit is used to acquire a coding region based on a coding matrix of at least one circular marker point;

[0147] A control region acquisition unit is used to set at least two scale markers outside the encoding region based on preset scale information to obtain a control region.

[0148] A universal calibration board acquisition unit is used to obtain a universal calibration board based on the encoding region and the control region.

[0149] In this context, each module in the camera calibration device based on the general calibration board corresponds to each step in the embodiment of the camera calibration method based on the general calibration board. Their functions and implementation processes will not be described in detail here.

[0150] In addition, embodiments of the present invention also provide a computer-readable storage medium.

[0151] The present invention provides a computer-readable storage medium storing a camera calibration program based on a universal calibration board, wherein when the camera calibration program based on the universal calibration board is executed by a processor, the steps of the camera calibration method based on the universal calibration board described above are implemented.

[0152] The method implemented when the camera calibration program based on the universal calibration board is executed can be referred to in various embodiments of the camera calibration method based on the universal calibration board of the present invention, and will not be repeated here.

[0153] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0154] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0155] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0157] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A camera calibration method based on a universal calibration board, characterized in that, The method includes the following steps: A calibration image is acquired, and a universal calibration plate in the calibration image is detected based on a preset neural network to obtain circular marker points; Based on a preset fitting method, the circular marker points are fitted with a circle center to obtain a fitted circle center. Based on the pattern of the circular marker points, a coding circle center is determined in the fitted circle center. The fitted circle center includes a coding circle center and a control circle center. Based on the sorting result of the coded circle center, the ID and main direction in the general calibration board are determined, and based on the main direction and the coded circle center, the control circle center is expanded and sorted to obtain the target calibration board; Based on the ID in the general calibration board, the sorting result of the fitted circle center in the target calibration board is determined, and based on the sorting result, the optical camera is calibrated to obtain the intrinsic parameters of the optical camera; Based on the target calibration plates with at least four different IDs, a target calibration field is constructed, and based on the target calibration field, the extrinsic parameters of the optical camera are estimated and optimized; The step of determining the ID and main direction in the universal calibration board based on the sorting result of the coded circle centers includes: Based on the center of the coding region, a central subset is obtained, and based on the central subset, convex hull calculation is performed to obtain the minimum bounding box of the central subset. Based on the homography matrix, the minimum bounding box is mapped to obtain a zero-rotation rectangle, and based on the zero-rotation rectangle, the coded circles in the central subset are sorted to obtain the coding matrix; Based on a preset dictionary, the encoding matrix is ​​matched to determine the ID and main direction corresponding to the general calibration board; The step of expanding and sorting the control circle centers based on the main direction and the encoding circle center to obtain the target calibration board includes: Based on the fitting circle center and the encoding circle center in the general calibration board, the control circle center is determined, and based on the control circle center, a subset of control regions is generated; Based on the main direction and cyclic search algorithm corresponding to the encoding matrix, the four neighborhoods of the encoding circle center are searched to obtain the unsorted control circle centers in the control area subset; When an unsorted control center is found, it is marked and sorted to obtain sorted control centers, and then the sorted control centers are cleared in the control area subset.

2. The camera calibration method based on a universal calibration board according to claim 1, characterized in that, Before detecting the universal calibration board in the calibration image based on a preset neural network to obtain circular marker points, the method further includes: Based on the data collection and annotation of images captured by the camera, real data is obtained; Virtual data is generated based on a preset data augmentation method. Based on the dataset composed of real and virtual data, network training is performed to obtain the preset neural network.

3. The camera calibration method based on a universal calibration board according to claim 1, characterized in that, The step of fitting the center of the circular marker point to obtain the fitted center based on the preset fitting method includes: Based on the bounding box of the circular pattern detected by the preset network model, the circular pattern is cropped to obtain the circular sub-image; Based on the binarization method, the outline of the circular sub-image is extracted to obtain an initial ellipse, and the initial ellipse is fitted based on a preset rotation matrix to obtain a fitted ellipse. The fitted ellipse is optimized based on the rotation matrix and the minimization fitting method to obtain the center of the fitted circle.

4. The camera calibration method based on a universal calibration board according to claim 1, characterized in that, The step of detecting a universal calibration board in the calibration image based on a preset neural network to obtain circular marker points includes: Based on the preset neural network, the calibration image is detected to obtain a universal calibration board and at least one circular pattern in the universal calibration board; Based on the preset training model, erroneous results in the general calibration board and the circular pattern are filtered out to obtain an unsorted circular pattern, which is used as the circular marker point.

5. The camera calibration method based on a universal calibration board according to claim 1, characterized in that, Before acquiring the calibration image, the process also includes: The coding region is obtained based on a coding matrix of at least one circular marker point; Based on preset scale information, at least two scale markers are set outside the coding region to obtain the control region; A general calibration board is obtained based on the encoding region and the control region.

6. A camera calibration device based on a universal calibration board, characterized in that, The camera calibration device based on the universal calibration board includes: A universal calibration board detection module is used to acquire a calibration image and, based on a preset neural network, detect the universal calibration board in the calibration image to obtain circular marker points. The circle center fitting module is used to perform circle center fitting on the circular marker points based on a preset fitting method to obtain the fitted circle center, and to determine the coded circle center based on the pattern of the circular marker points in the fitted circle center, wherein the fitted circle center includes the coded circle center and the control circle center; The center sorting module is used to determine the ID and main direction in the general calibration board based on the sorting result of the coded center, and to expand and sort the control center based on the main direction and the coded center to obtain the target calibration board; The intrinsic parameter calibration module is used to determine the sorting result of the fitted circle centers in the target calibration board based on the ID in the general calibration board, and to calibrate the optical camera based on the sorting result to obtain the intrinsic parameters of the optical camera. An extrinsic parameter calibration module is used to construct a target calibration field based on at least four target calibration boards with different IDs, and to estimate and optimize the extrinsic parameters of the optical camera based on the target calibration field. The circle center sorting module includes: The central subset obtaining unit is used to obtain a central subset based on the coding circle center of the coding region, and to perform convex hull calculation based on the central subset to obtain the minimum bounding box of the central subset; The encoding matrix obtaining unit is used to map the minimum bounding box based on the homography matrix to obtain a zero-rotation rectangle, and to sort the encoding circles in the central subset based on the zero-rotation rectangle to obtain the encoding matrix. The encoding matrix matching unit is used to match the encoding matrix based on a preset dictionary to determine the ID and main direction corresponding to the general calibration board; The center sorting module also includes: The control region subset generation unit is used to determine the control circle center based on the fitting circle center and the encoding circle center in the general calibration board, and to generate a control region subset based on the control circle center; The unsorted control circle center acquisition unit is used to search the four neighborhoods of the encoded circle center based on the main direction and cyclic search algorithm corresponding to the encoding matrix, and obtain the unsorted control circle centers in the control area subset. The sorting control center acquisition unit is used to mark and sort the unsorted control centers when they are found, obtain sorted control centers, and clear the sorted control centers in the control area subset.

7. A camera calibration device based on a universal calibration board, characterized in that, The camera calibration device based on the universal calibration board includes a processor, a memory, and a camera calibration program based on the universal calibration board stored in the memory and executable by the processor, wherein when the camera calibration program based on the universal calibration board is executed by the processor, it implements the steps of the camera calibration method based on the universal calibration board as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a camera calibration program based on a universal calibration board, wherein when the camera calibration program based on the universal calibration board is executed by a processor, it implements the steps of the camera calibration method based on a universal calibration board as described in any one of claims 1 to 5.

Images