A camera calibration method, apparatus, device, storage medium and program product

Abstract

The application relates to a camera calibration method, device, equipment, storage medium and program product. The method comprises the following steps: acquiring a calibration image obtained by image acquisition of a circle array target by a camera to be calibrated; performing initial calibration on the camera to be calibrated according to the center world coordinates of each solid circle in the circle array target and the center pixel coordinates of a target circle corresponding to each solid circle in the calibration image, so as to obtain calibration parameters; performing distortion correction on each target circle according to a distortion coefficient, so as to obtain the corresponding center normalized coordinates of each target circle in a normalized plane; determining the projection pixel coordinates of the center normalized coordinates of each target circle in the calibration image according to the calibration parameters; and recalibrating the camera to be calibrated according to the projection pixel coordinates of each target circle and the center world coordinates of the solid circle corresponding to each target circle, so as to update the calibration parameters. The method can improve the accuracy and stability of camera calibration.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a camera calibration method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of computer vision technology, camera calibration, as a fundamental and crucial technology in vision systems, has been widely applied in fields such as industrial inspection, 3D reconstruction, autonomous driving, and augmented reality. The purpose of camera calibration is to obtain the camera's calibration parameters by capturing calibration images with geometric structure information, thereby establishing an accurate mapping relationship between the three-dimensional world coordinate system and the two-dimensional image plane.

[0003] Currently, camera calibration is usually performed based on specific points of planar targets such as checkerboard or circular array targets. This involves acquiring images of the planar targets and extracting specific points of the planar targets from the acquired images (such as checkerboard corners, the center of circular array targets, etc.). Then, combining the geometric constraints between these feature points and their corresponding world coordinates, camera calibration is completed using linear and nonlinear optimization processes.

[0004] However, in the above methods, the detection of feature points of planar targets such as checkerboard or circular array targets is easily affected by factors such as image noise, distortion, resolution, as well as factors such as camera shooting angle tilt and radial and tangential distortion of the camera. Therefore, the positioning accuracy of the extracted feature points will decrease, which in turn affects the accuracy and stability of camera calibration. Summary of the Invention

[0005] Therefore, it is necessary to provide a camera calibration method, apparatus, device, storage medium, and program product to address the aforementioned technical problems, thereby improving the accuracy and stability of camera calibration.

[0006] In a first aspect, this application provides a camera calibration method, comprising: acquiring a calibration image obtained by the camera to be calibrated acquiring an image of a circular array target; wherein the circular array target includes multiple solid circles arranged in an orderly manner; performing initial calibration on the camera to be calibrated based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, to obtain calibration parameters; wherein the calibration parameters include intrinsic parameters, extrinsic parameters, and distortion coefficients; performing distortion correction on each target circle based on the distortion coefficients, to obtain the center normalized coordinates of each target circle in the normalized plane; determining the projected pixel coordinates of the center normalized coordinates of each target circle in the calibration image based on the calibration parameters; and recalibrating the camera to be calibrated based on the projected pixel coordinates of each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, to update the calibration parameters.

[0007] In one embodiment, distortion correction is performed on each target circle according to the distortion coefficient to obtain the center normalized coordinates of each target circle in the normalized plane. This includes: performing distortion correction on each target circle according to the distortion coefficient to obtain the calibration circle corresponding to the target circle in the normalized plane, and performing ellipse fitting on each calibration circle to obtain a calibration ellipse; determining a first hidden point in the horizontal direction and a second hidden point in the vertical direction in the normalized plane according to the extrinsic parameters; determining the center coordinates of each calibration ellipse according to the first hidden point and the second hidden point, and determining the center normalized coordinates of the target circle corresponding to each calibration ellipse according to the center coordinates of each calibration ellipse.

[0008] In one embodiment, determining the center coordinates of each calibration ellipse based on the first and second hidden points includes: for each calibration ellipse, determining a first set of tangent points corresponding to the first hidden point and a second set of tangent points corresponding to the second hidden point; and determining the center coordinates of the calibration ellipse based on the coordinates of the intersection of the line connecting the first and second sets of tangent points.

[0009] In one embodiment, after recalibrating the camera to be calibrated to update the calibration parameters, the method further includes: determining the calibration result corresponding to the camera to be calibrated, and if the calibration result does not meet the calibration termination condition, returning to perform the operation of distortion correction on each target circle according to the distortion coefficient; wherein the calibration result includes the number of calibration iterations or the change in reprojection error, and the reprojection error is determined according to the difference between the projection pixel coordinates and the center pixel coordinates corresponding to each target circle.

[0010] In one embodiment, the calibration result includes the change in reprojection error; determining the calibration result corresponding to the camera to be calibrated includes: determining the current reprojection error based on the average distance between the projected pixel coordinates and the center pixel coordinates of each target circle; and determining the change in reprojection error based on the difference between the current reprojection error and the historical reprojection error after the last calibration iteration.

[0011] In one embodiment, the initial calibration of the camera to be calibrated is performed based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image. This includes: performing edge detection on the calibration image to obtain multiple initial images in the calibration image; selecting target circles from the multiple initial images based on the graphic parameters of each initial image; wherein the graphic parameters include perimeter, area, and roundness index, and the roundness index is used to reflect the degree of similarity between the graphic and a standard circle; identifying the target circle corresponding to each solid circle in the circular array target from among the target circles, and determining the center pixel coordinates of the target circle corresponding to each solid circle; and performing the initial calibration of the camera to be calibrated based on the world coordinates of the center of each solid circle and the center pixel coordinates of the target circle corresponding to each solid circle.

[0012] In one embodiment, the plurality of solid circles includes a first solid circle and a second solid circle, wherein the diameter of the first solid circle is larger than the diameter of the second solid circle; identifying the target circle corresponding to each solid circle in the circular array target from among the target circles includes: selecting a specified number of target circles from each target circle in descending order of specified parameters as first target circles, and identifying the first target circle corresponding to each first solid circle from among the first target circles; wherein the specified parameters include circumference or area, and the specified number is the number of first solid circles; determining the coordinate transformation relationship between the world coordinate system corresponding to the circular array target and the pixel coordinate system corresponding to the calibration image based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the first target circle corresponding to each first solid circle; and identifying the second target circle corresponding to each second solid circle from among the second target circles other than the first target circles based on the coordinate transformation relationship.

[0013] In one embodiment, based on coordinate transformation relationships, identifying the second target circle corresponding to each second solid circle from among the second target circles other than the first target circle includes: mapping the world coordinates of the center of each second solid circle to the calibration image based on coordinate transformation relationships to obtain the center mapping coordinates corresponding to each second solid circle; for each second solid circle, selecting the second target circle whose center pixel coordinates are closest to the center mapping coordinates corresponding to the second solid circle from among the second target circles to obtain the second target circle corresponding to the second solid circle.

[0014] Secondly, this application provides a camera calibration device, comprising: an image acquisition module for acquiring a calibration image obtained by the camera to be calibrated acquiring an image of a circular array target; wherein the circular array target includes a plurality of solid circles arranged in an orderly manner; a first calibration module for performing initial calibration on the camera to be calibrated based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, to obtain calibration parameters; wherein the calibration parameters include intrinsic parameters, extrinsic parameters, and distortion coefficients; a distortion correction module for performing distortion correction on each target circle based on the distortion coefficients, to obtain the center normalized coordinates of each target circle in the normalized plane; a coordinate projection module for determining the projected pixel coordinates of the center normalized coordinates corresponding to each target circle in the calibration image based on the calibration parameters; and a second calibration module for recalibrating the camera to be calibrated based on the projected pixel coordinates corresponding to each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, to update the calibration parameters.

[0015] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the various method embodiments provided in the first aspect above.

[0016] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method embodiments provided in the first aspect above.

[0017] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the various method embodiments provided in the first aspect above.

[0018] The aforementioned camera calibration method, apparatus, device, storage medium, and program product acquire calibration images obtained by the camera to be calibrated from images of a circular array target comprising multiple ordered solid circles. Based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the corresponding target circle in the calibration image, the camera to be calibrated is initially calibrated to obtain calibration parameters. Then, based on the distortion coefficients in the obtained calibration parameters, distortion correction is performed on each target circle in the calibration image to obtain the center-normalized coordinates of each target circle in the normalized plane. Next, based on the obtained calibration parameters, the center-normalized coordinates of each target circle are projected onto the calibration image containing that target circle to obtain the projected pixel coordinates of that target circle in its own calibration image. Finally, based on the projected pixel coordinates of each target circle in the calibration image and the world coordinates of the center of the corresponding solid circle, the camera to be calibrated is recalibrated to update the aforementioned calibration parameters. In this way, on the one hand, after the initial calibration of the camera to be calibrated, distortion correction is performed on the target circle, and the center of the target circle is reprojected using the normalized coordinates of the target circle's center. This yields projected pixel coordinates that are closer to the true center pixel coordinates of the target circle, thus improving the center positioning accuracy of the target circle. On the other hand, the camera to be calibrated is then recalibrated using the projected pixel coordinates corresponding to the target circle. Since the projected pixel coordinates corresponding to the target circle are closer to the true center pixel coordinates of the target circle, the positioning accuracy of the projected pixel coordinates is higher, improving the accuracy of the updated calibration parameters obtained from the recalibration of the camera to be calibrated, thereby improving the accuracy and stability of camera calibration. Based on this, the actual geometric transformation law of the solid circular pattern of the circular array target under perspective projection and imaging distortion is fully considered, avoiding the deviation problem caused by directly using the center of the target circle corresponding to the solid circle. This significantly improves the accuracy and convergence stability of camera calibration, and is especially suitable for high-precision camera calibration applications under complex viewing angles and non-ideal imaging conditions. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 Application environment diagrams for camera calibration methods provided in some embodiments of this application;

[0021] Figure 2A A schematic flowchart illustrating a camera calibration method provided in some embodiments of this application;

[0022] Figure 2B This is a schematic diagram of a circular array target;

[0023] Figure 3 A flowchart illustrating the process of determining center-normalized coordinates provided in some embodiments of this application;

[0024] Figure 4 A flowchart illustrating the process of determining the coordinates of a circle's center, provided for some embodiments of this application;

[0025] Figure 5 A schematic flowchart illustrating a camera calibration method provided in other embodiments of this application;

[0026] Figure 6 A flowchart illustrating the process of determining calibration results provided for some embodiments of this application;

[0027] Figure 7 A schematic diagram illustrating the initial calibration process of the camera to be calibrated, provided for some embodiments of this application;

[0028] Figure 8 A flowchart illustrating the process of identifying a target circle corresponding to a solid circle, provided in some embodiments of this application;

[0029] Figure 9 A schematic flowchart illustrating a camera calibration method provided in some embodiments of this application;

[0030] Figure 10 A schematic diagram illustrating the distribution of reprojection points of a camera using some embodiments of this application;

[0031] Figure 11 Structural block diagrams of camera correction devices provided in some embodiments of this application;

[0032] Figure 12 Internal structural diagrams of a computer device provided in some embodiments of this application;

[0033] Figure 13 Internal structural diagrams of a computer device provided for other embodiments of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments or any combination of multiple embodiments.

[0036] Currently, camera calibration is typically performed based on specific points on planar targets such as checkerboard or circular array targets. This involves acquiring images of the planar target and extracting specific points from the acquired images (e.g., checkerboard corners, circular array center points). Then, combining these feature points with their corresponding world coordinates, linear and nonlinear optimization processes are used to complete the camera calibration. However, this method is susceptible to problems because the detection of feature points on planar targets like checkerboard or circular array targets is easily affected by image noise, distortion, resolution, camera tilt, and radial and tangential distortion. This leads to a decrease in the localization accuracy of the extracted feature points, thus affecting the accuracy and stability of the camera calibration.

[0037] To address the aforementioned technical problems, in an exemplary embodiment, a camera calibration method is provided. This method can be applied to computer devices with data processing and analysis functions. The computer device can be presented as a server or as a terminal. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc., while portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0038] In one exemplary embodiment, the camera calibration method provided in this application can be applied to, for example, Figure 1In the application environment shown, the camera 102 to be calibrated communicates with the server 104 via a network. The data storage system can store the data that the server 104 needs to process, such as the world coordinates of the center of each solid circle in the circular array target. The data storage system can be integrated onto the server 104 or placed in the cloud or on another network server. The camera 102 stores each calibration image obtained from acquiring images of the circular array target into the data storage system. Then, the server 104 retrieves the world coordinates of the center of each solid circle in the circular array target and each calibration image from the data storage system to execute the camera calibration method provided in this embodiment.

[0039] Before providing a detailed explanation of the camera calibration method, let's briefly explain the relevant concepts involved.

[0040] Camera calibration refers to the process of solving for unknown parameters (intrinsic parameters, extrinsic parameters, distortion parameters) in the camera imaging model by using the known correspondence between image features and real-world coordinates. Its core is to transform the camera from an "imaging black box" into a quantifiable model and establish a precise mapping relationship between two-dimensional image pixel coordinates and three-dimensional world coordinates, providing a foundation for subsequent tasks such as geometric measurement, image correction, and three-dimensional reconstruction.

[0041] The calibration parameters are a set of key data used to quantify the imaging patterns, errors, and spatial relationships of a camera. Their core function is to establish a precise mapping relationship between three-dimensional world coordinates and two-dimensional image pixel coordinates, including intrinsic parameters, extrinsic parameters, and distortion coefficients.

[0042] Intrinsic parameters are a set of parameters that describe the inherent optical and sensor characteristics of a camera. They are determined by the camera hardware (lens, image sensor) and do not change with the camera's position, orientation, or shooting scene. They typically include focal length (…). ), principal point coordinates ( ), pixel aspect ratio ( ) and tilt factor ( ).

[0043] Among them, focal length ( ) is a scaling factor (in pixels) that converts the physical focal length of a lens (in millimeters) into "pixel units". Corresponding to the x-axis (horizontal direction) of the image. Corresponding to the y-axis (vertical direction) of the image; principal point coordinates ( ) is the coordinate (in pixels) of the intersection point of the camera's optical axis (the line connecting the lens centers) and the image sensor plane in the pixel coordinate system; the pixel aspect ratio ( ) is the width of a single pixel in an image sensor. ) and height ( )ratio( The ideal value is 1 (square pixel); tilt factor Used to describe the degree of non-orthogonality between the x-axis and y-axis of an image sensor (ideally 0), which is caused by the non-perpendicularity of the coordinate axes due to the sensor being mounted at an angle.

[0044] Typically, intrinsic parameters can be represented by a 3×3 intrinsic parameter matrix ( This is centrally reflected in the following formula:

[0045] .

[0046] Extrinsic parameters are parameters that describe the relative position and attitude relationship between the camera coordinate system and the world coordinate system. Their core characteristic is that they change in real time as the camera moves and rotates, and typically include a rotation matrix. Translation vector ).

[0047] Wherein, rotation matrix ( ) is a 3×3 orthogonal matrix (satisfying) (with a determinant of 1), used to describe the rotational attitude of the camera coordinate system relative to the world coordinate system, that is, the rotation angle of the camera around the x-axis, y-axis, and z-axis of the world coordinate system (usually corresponding to pitch angle, yaw angle, and roll angle); translation vector ( ) is a 3×1 vector (in physical units such as millimeters and meters) that describes the position coordinates of the camera coordinate system origin (optical center) in the world coordinate system, that is, the distances from the world origin in the x, y, and z directions.

[0048] The core function of extrinsic parameters is to achieve coordinate transformation. If a point in the world coordinate system is... The point in the camera coordinate system is The relationship between the two can be expressed as: That is, a point in the world coordinate system, first through a rotation matrix ( Align the camera's orientation with the translation vector ( The point is "moved" to the camera's position, and finally the coordinates of that point in the camera's coordinate system are obtained.

[0049] Distortion coefficients are parameters that describe the inherent distortion errors of a lens's optical system, used to quantify the degree to which "actual straight lines are distorted into curves" in an image. Distortion is generally divided into radial distortion and tangential distortion. Radial distortion is caused by the spherical nature of the lens and manifests as a mismatch between the scaling ratio of the image center and edges; it can be further divided into barrel distortion and pincushion distortion. The radial distortion coefficients include... , and The larger the value, the more severe the distortion; tangential distortion is caused by the tilt (non-strict parallelism) of the lens and image sensor assembly, manifested as straight lines in the image "tilting and bending" in a certain direction. The tangential distortion coefficient includes... and It is used to correct pixel offset caused by plane tilt.

[0050] In one exemplary embodiment, such as Figure 2A As shown, a camera calibration method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation may include the following steps:

[0051] S201, acquire the calibration image obtained by the camera to be calibrated from the image acquisition of the circular array target.

[0052] The circular array target consists of multiple solid circles arranged in an orderly manner.

[0053] The term "camera to be calibrated" refers to a camera that needs calibration to solve for the aforementioned unknown parameters, which can be called calibration parameters. The term "image acquisition device" is a general term for all systems or devices capable of converting optical signals into digital image signals, and its components may include lenses, sensors, data transmission modules, and control units. A "camera" (whether an industrial camera, SLR camera, or mobile phone camera) is a device that integrates these core components and can independently perform image acquisition functions; it is the most common and standardized form of image acquisition device. Furthermore, in addition to cameras, image acquisition devices may also include more specialized or combined devices, such as industrial vision inspection modules with special light sources and lenses, or stereo vision acquisition systems composed of multiple cameras. Therefore, the camera to be calibrated can be an independent image acquisition device or a camera component within other image acquisition devices used to perform image acquisition functions. In other words, the camera to be calibrated can be any device capable of performing image acquisition functions and requiring the solution of calibration parameters to establish pixel coordinates and three-dimensional world coordinates; it can be an independent device or a functional component included in a device.

[0054] A circular array target is a calibration tool composed of regularly arranged circular patterns. Its core function is to provide high-precision feature points for camera calibration. By solving for calibration parameters (intrinsic, extrinsic, and distortion parameters), it achieves a precise mapping between pixel coordinates and 3D world coordinates. The physical form of a circular array target typically consists of multiple circular patterns arranged according to a specific pattern on the same plane; that is, a circular array target comprises multiple solid circles arranged in an orderly fashion. For example, Figure 2B This is a schematic diagram of a circular array target, such as... Figure 2BAs shown, the circular array target includes 99 solid circles arranged in a 11×9 pattern. Among them, there are five large circles (I, II, III, IV, and V) with diameters larger than the other circles, which serve as feature circles in the circular array target for reference positioning during camera calibration.

[0055] As mentioned earlier, during the calibration process of the camera to be calibrated, the camera is first controlled to acquire images of the circular array target to obtain calibration images for camera calibration. The calibration images include images of all the solid circles in the circular array target.

[0056] In one optional embodiment, the camera to be calibrated is controlled to acquire images of the circular array target from multiple different angles to obtain multiple calibration images for camera calibration. The circular array target in each calibration image has a different pose, and each calibration image includes images of all the solid circles in the circular array target. For example, at least three calibration images are acquired for camera calibration.

[0057] In one alternative embodiment, the position of the circular array target in space can be static or dynamic during calibration image acquisition.

[0058] S202, based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, the camera to be calibrated is initially calibrated to obtain calibration parameters.

[0059] The calibration parameters include intrinsic parameters, extrinsic parameters, and distortion coefficients.

[0060] After acquiring each calibration image, for each calibration image, the target circle corresponding to each solid circle in the circular array target can be determined in that calibration image, that is, a one-to-one correspondence can be established between the target circle in the calibration image and the solid circle of the circular array target. Optionally, due to geometric distortion of the camera lens to be calibrated, the true shape of the target circle is closer to an ellipse.

[0061] For example, each solid circle in a circular array target can have different image features. Thus, for each calibration image, by identifying the image features of each target circle in the calibration image, solid circles and target circles with the same image features can be identified as corresponding solid circles and target circles, thereby establishing a one-to-one correspondence between the target circles in the calibration image and the solid circles of the circular array target.

[0062] Thus, after establishing the above correspondence, the camera to be calibrated can be initially calibrated based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the corresponding target circle in the calibration image, in order to obtain calibration parameters. Here, "initial calibration" simply means that this calibration is the first calibration of the camera to be calibrated, and does not constitute a limitation on the calibration method.

[0063] Optionally, ellipse fitting can be performed on each target circle to obtain the center pixel coordinates of the fitted ellipse, which can then be used as the center pixel coordinates of the target circle.

[0064] Optionally, for a circular array target, the two-dimensional homogeneous coordinates of each solid circle in the circular array target can be defined as the world coordinates of the center of that solid circle. For example, as Figure 2B The circular array target shown has its two-dimensional homogeneous coordinates set as (0,0,1) for the first solid circle in the upper left corner. Therefore, the two-dimensional homogeneous coordinates of the solid circle in the first row and second column are (0,1,1), and so on, assigning two-dimensional homogeneous coordinates to each solid circle. Specifically, the two-dimensional homogeneous coordinates of large circle I are (2,5,1), large circle II are (4,8,1), large circle III are (6,6,1), large circle IV are (6,5,1), and large circle V are (4,2,1).

[0065] In one optional embodiment, when multiple calibration images are acquired, a one-to-one correspondence can be established between the target circle in the calibration image and the solid circle of the circular array target for each calibration image. Then, based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to that solid circle in each calibration image, the camera to be calibrated is initially calibrated to obtain calibration parameters.

[0066] For example, when multiple calibration images are obtained, the Zhang Zhengyou calibration method can be used. Based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the corresponding target circle in each calibration image, the camera to be calibrated is initially calibrated to obtain calibration parameters. Specifically, for each calibration image, the homography matrix can be solved using the least squares method based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the corresponding target circle in the calibration image, resulting in multiple homography matrices. Then, constraint equations are constructed for the obtained homography matrices, and intrinsic parameter matrices are derived using Singular Value Decomposition (SVD) or Cholesky Decomposition to obtain intrinsic parameters. Next, based on the obtained homography matrices and the homography matrix of each calibration image, extrinsic parameters are solved through orthogonalization. Finally, a distortion model is introduced, and the distortion coefficients are estimated iteratively using the intrinsic and extrinsic parameters.

[0067] It should be emphasized that the embodiments of this application do not limit the specific implementation of S202 above. Any method that can calibrate the camera to be calibrated and obtain calibration parameters by using the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image is within the protection scope of the embodiments of this application.

[0068] S203, based on the distortion coefficient, perform distortion correction on each target circle to obtain the center normalized coordinates of each target circle in the normalized plane.

[0069] The so-called normalized plane is a virtual plane in the camera coordinate system. Its vertical distance from the optical center (the origin of the camera coordinate system) is 1 unit (usually a meter). When a point in the three-dimensional world coordinate system is projected onto this plane, its coordinates are normalized to dimensionless normalized coordinates (x, y, 1). The normalized plane can unify cameras with different focal lengths and principal point positions to the same reference.

[0070] Correspondingly, normalized coordinates are dimensionless coordinates (x, y, 1) in a normalized coordinate system that maps coordinates in other coordinate systems (such as pixel coordinates in two-dimensional images and three-dimensional world coordinates) to the normalized plane.

[0071] Distortion correction refers to the process of eliminating or correcting geometric distortions in an image caused by factors such as the optical characteristics of a camera lens, so as to restore the actual shape and positional relationship of objects in the real scene.

[0072] To eliminate the impact of geometric distortion introduced by the camera lens on the imaging accuracy of the calibration image and improve the accuracy of camera calibration, distortion correction can be performed on each target circle in the calibration image using the distortion coefficients in the obtained calibration parameters. Specifically, during distortion correction, the pixels of each target circle can be mapped to a distortion-free normalized plane based on the distortion coefficients in the calibration parameters. Each target circle mapped to the normalized plane after distortion correction is no longer affected by distortion and more closely approximates the geometric position under the ideal imaging model of the camera to be calibrated.

[0073] For example, distortion correction can be performed on each target circle using the following formula:

[0074]

[0075] in, The center pixel coordinates of the target circle. Let the radius of the target circle be . , , The radial distortion coefficient is... , The tangential distortion coefficient is... These are the coordinates after distortion correction.

[0076] Optionally, the memory size and distortion coefficients in the calibration parameters, as well as the pixel points of each target circle in the two-dimensional image pixel coordinate system of the calibration image, can be input into the cv2.undistortPoints (OpenCV Undistort Points) function in OpenCV (Open Source Computer Vision Library). This function can then automatically perform distortion correction operations on the input pixels based on the camera's intrinsic parameter matrix and distortion coefficients.

[0077] Then, for each target circle after distortion correction mapped to the normalization plane, the center normalized coordinates can be further determined to obtain the center normalized coordinates of each target circle in the normalization plane.

[0078] In one optional embodiment, when multiple calibration images are acquired, for each calibration image, distortion correction is performed on each target circle in the calibration image according to the distortion coefficient, so as to obtain the center normalized coordinates of each target circle in the calibration image in the normalized plane.

[0079] S204. Based on the calibration parameters, determine the projected pixel coordinates of the center normalized coordinates of each target circle in the calibration image.

[0080] As mentioned earlier, for each target circle, the distortion-corrected target circle mapped to the normalized plane is closer to the geometric position under the ideal imaging model of the camera to be calibrated. Then, after projecting each distortion-corrected target circle mapped to the normalized plane back to the calibration image, the distortion-corrected target circle projected onto the calibration image is also closer to the geometric position under the ideal imaging model of the camera to be calibrated. Therefore, using the center pixel coordinates of each distortion-corrected target circle projected onto the calibration image to recalibrate the calibration image can improve the accuracy of camera calibration.

[0081] Therefore, based on the calibration parameters, the center normalized coordinates corresponding to each target circle can be mapped back to the calibration image where the target circle is located, so as to obtain the pixel coordinates corresponding to the center normalized coordinates of the target circle in the calibration image where the target circle is located, which can be used as the projected pixel coordinates of the target circle.

[0082] Optionally, the calibration parameters and the center-normalized coordinates of each target circle can be input into the OpenCV function `cv2.projectPoints(OpenCV Project Points)`. This function can then automatically project the center-normalized coordinates forward according to the calibration parameters. Specifically, the center-normalized coordinates are first transformed to the camera coordinate system, then mapped to the world coordinate system through rotation and translation, and finally the projection is completed based on the intrinsic parameters. Taking into account the effects of lens distortion, the function outputs the projected pixel coordinates of the center-normalized coordinates of each target circle in the calibration image containing that target circle.

[0083] In one optional embodiment, when multiple calibration images are obtained, for each calibration image, the projection pixel coordinates of the corresponding center normalized coordinates in the calibration image are determined according to the calibration parameters.

[0084] S205, based on the projected pixel coordinates corresponding to each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, recalibrate the camera to be calibrated in order to update the calibration parameters.

[0085] As mentioned earlier, the projected pixel coordinates corresponding to each target circle are equivalent to the pixel coordinates after distortion correction of the center pixel coordinates of each target circle. Therefore, the camera to be calibrated can be recalibrated based on the projected pixel coordinates of each target circle and the world coordinates of the center of each solid circle. Furthermore, since the two-dimensional image pixel coordinates corresponding to the world coordinates of the center of each solid circle in the circular array target change during this calibration process, the calibration parameters obtained will also change compared to the previous calibration. In other words, the camera to be calibrated is recalibrated, and the calibration parameters are updated.

[0086] The specific implementation method for calibrating the camera to be calibrated based on the projected pixel coordinates corresponding to each target circle and the world coordinates of the center of the solid circle corresponding to each target circle is the same as the specific implementation method for the initial calibration of the camera to be calibrated in S202 above, and will not be repeated here.

[0087] In one optional embodiment, when multiple calibration images are acquired, the camera to be calibrated is recalibrated based on the world coordinates of the center of each solid circle in the circular array target and the projected pixel coordinates of the target circle corresponding to that solid circle in each calibration image, so as to update the calibration parameters.

[0088] In the aforementioned camera calibration method, a calibration image is obtained by acquiring images of a circular array target consisting of multiple ordered solid circles from the camera to be calibrated. Based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the corresponding target circle in the calibration image, the camera to be calibrated initially to obtain calibration parameters. Then, based on the distortion coefficients in the obtained calibration parameters, distortion correction is performed on each target circle in the calibration image to obtain the center-normalized coordinates of each target circle in the normalized plane. Next, based on the obtained calibration parameters, the center-normalized coordinates of each target circle are projected onto the calibration image containing that target circle to obtain the projected pixel coordinates of that target circle in its own calibration image. Finally, based on the projected pixel coordinates of each target circle in the calibration image and the world coordinates of the center of the corresponding solid circle, the camera to be calibrated again to update the aforementioned calibration parameters. In this way, on the one hand, after the initial calibration of the camera to be calibrated, distortion correction is performed on the target circle, and the center of the target circle is reprojected using the normalized coordinates of the target circle's center. This yields projected pixel coordinates that are closer to the true center pixel coordinates of the target circle, thus improving the center positioning accuracy of the target circle. On the other hand, the camera to be calibrated is then recalibrated using the projected pixel coordinates corresponding to the target circle. Since the projected pixel coordinates corresponding to the target circle are closer to the true center pixel coordinates of the target circle, the positioning accuracy of the projected pixel coordinates is higher, improving the accuracy of the updated calibration parameters obtained from the recalibration of the camera to be calibrated, thereby improving the accuracy and stability of camera calibration. Based on this, the actual geometric transformation law of the solid circular pattern of the circular array target under perspective projection and imaging distortion is fully considered, avoiding the deviation problem caused by directly using the center of the target circle corresponding to the solid circle. This significantly improves the accuracy and convergence stability of camera calibration, and is especially suitable for high-precision camera calibration applications under complex viewing angles and non-ideal imaging conditions.

[0089] Based on the above embodiments, in one exemplary embodiment, the determination of the center normalized coordinates is further refined. Optionally, such as... Figure 3As shown, the following steps may be included:

[0090] S301, Based on the distortion coefficient, perform distortion correction on each target circle to obtain the calibration circle corresponding to the target circle in the normalized plane, and perform ellipse fitting on each calibration circle to obtain the calibration ellipse.

[0091] Based on the distortion coefficients in the calibration parameters, the pixels of each target circle are mapped onto a distortion-free normalized plane to obtain the corresponding calibration circle in the normalized plane. This calibration circle is the distortion-corrected target circle mapped to the normalized plane. Then, ellipse fitting is performed on this calibration circle to obtain the calibration ellipse. For example, various methods such as least squares fitting and geometric distance fitting can be used for ellipse fitting.

[0092] S302, based on the extrinsic parameters, determine the first hidden point in the horizontal direction and the second hidden point in the vertical direction in the normalized plane.

[0093] A hidden point is the intersection of the projections of a set of parallel lines in space onto the image plane. In other words, it's the point where a set of parallel lines in space intersects in the image under perspective projection, reflecting the perspective convergence in that direction. Correspondingly, for a circular array target, based on the ordered arrangement of solid circles in the target, connecting the centers of the solid circles in each row yields a set of parallel lines in the horizontal direction. Similarly, connecting the centers of the solid circles in each column yields a set of parallel lines in the vertical direction. Then, based on the extrinsic parameters in the calibration parameters, the point where these horizontally parallel lines intersect under perspective projection in the normalized plane can be determined—this is the first hidden point in the horizontal direction of the normalized plane. Likewise, the point where these vertically parallel lines intersect under perspective projection in the normalized plane can also be determined—this is the first hidden point in the vertical direction of the normalized plane.

[0094] In one alternative embodiment, the horizontal unit vector in the world coordinate system of the circular array target is obtained. and the unit vector in the vertical direction , respectively with the rotation matrix in the extrinsic parameters ( Multiplying these two vectors yields the direction vector in the camera coordinate system. and The obtained direction vector and Projecting onto the normalized plane yields the first hidden point in the horizontal direction of the normalized plane. and the coordinates of the second hidden point in the vertical direction .

[0095] S303, based on the first and second hidden point, determine the center coordinates of each calibration ellipse, and based on the center coordinates of each calibration ellipse, determine the center normalized coordinates of the target circle corresponding to each calibration ellipse.

[0096] After obtaining the first and second hidden point, the center coordinates of each calibration ellipse can be determined based on these points. For each calibration ellipse, the center coordinates can then be determined, which are the normalized center coordinates of the target circle corresponding to that calibration ellipse. This yields the normalized center coordinates of the target circle in the normalized plane. Thus, the normalized center coordinates of each target circle in the normalized plane can be obtained.

[0097] In this embodiment, a first hidden point in the horizontal direction and a second hidden point in the vertical direction are introduced in the normalized plane to determine the center normalized coordinates of each target circle in the normalized plane. This can further improve the accuracy of the obtained center normalized coordinates, thereby improving the accuracy of subsequent camera calibration.

[0098] Based on the above embodiments, in one exemplary embodiment, the determination of the center coordinates is further refined. Optionally, such as... Figure 4 As shown, the following steps may be included:

[0099] S401, for each calibration ellipse, determine the first set of tangent points on the calibration ellipse corresponding to the first hidden point and the second set of tangent points corresponding to the second hidden point.

[0100] As mentioned earlier, a hidden point is equivalent to a point in an image. Therefore, for an ellipse in the image, the ellipse has two tangents that pass through the hidden point, and the line connecting each of these two tangents to the point of tangency of the ellipse passes through the center of the ellipse.

[0101] Therefore, for each calibration ellipse in the normalized plane, two tangents to the calibration ellipse passing through the first hidden point are determined, and the point of tangency between each of these two tangents and the calibration ellipse is determined, resulting in a first set of tangent points on the calibration ellipse corresponding to the first hidden point. Furthermore, the line connecting two tangent points in the first set passes through the center of the calibration ellipse. Similarly, two tangents to the calibration ellipse passing through the second hidden point can be determined, and the point of tangency between each of these two tangents and the calibration ellipse is determined, resulting in a second set of tangent points on the calibration ellipse corresponding to the second hidden point. Likewise, the line connecting two tangent points in the second set also passes through the center of the calibration ellipse.

[0102] S402, determine the coordinates of the center of the calibration ellipse based on the coordinates of the intersection of the first set of tangent points and the second set of tangent points.

[0103] For each calibration ellipse in the normalized plane, after obtaining the first set of tangent points and the second set of tangent points, the line connecting two tangent points in the first set of tangent points can be determined as the first set of tangent point connection line, and the line connecting two tangent points in the second set of tangent points can be determined as the second set of tangent point connection line. Then, the coordinates of the intersection point of the first set of tangent point connection line and the second set of tangent point connection line can be determined. The coordinates of the intersection point are the coordinates of the center of the calibration ellipse.

[0104] In one optional embodiment, the first hidden point in the horizontal direction of the normalization plane and the coordinates of the second hidden point in the vertical direction Furthermore, using the first disappearance point For example, let's set a point. It is the first hidden point on the calibration ellipse P. The point of tangency of the tangent line originating from the starting point, then point... With the first hidden point The slope of the line connecting the points is equal to the slope of the line connecting the points on the ellipse P. The slope of the tangent at point A can be used to derive the following equation:

[0105]

[0106] in, This is the general quadratic equation for calibrating the ellipse P. , , , , and All are constants, and , and Not all zeros .

[0107] By analytically solving the above equations, we can obtain the points on the calibration ellipse P and the first hidden point. The corresponding first set of tangent points and Its coordinate value expression is as follows:

[0108]

[0109] in, , 1. The formula for calculating the value of 1 is as follows:

[0110]

[0111] With the first set of tangent points mentioned above and The analytical solution method is similar, and we can obtain the intersection of the calibration ellipse P and the second hidden surface removal. The corresponding second set of tangent points and .

[0112]

[0113] in, 2. 2. The formula for calculating the binary value is as follows:

[0114]

[0115] Furthermore, the coordinates of the center of the calibration ellipse P can be determined. As shown in the following formula:

[0116]

[0117] In this embodiment, by determining the first set of tangent points corresponding to the first hidden point and the second set of tangent points corresponding to the second hidden point on the calibration ellipse, the center coordinates of the calibration ellipse can be determined based on the coordinates of the intersection of the line connecting the first set of tangent points and the line connecting the second set of tangent points. This can further improve the accuracy of the obtained center normalized coordinates, thereby improving the accuracy of subsequent camera calibration.

[0118] Based on the above embodiments, in an exemplary embodiment, such as Figure 5 As shown, the camera calibration method may include the following steps:

[0119] S501, acquire the calibration image obtained by the camera to be calibrated from the image acquisition of the circular array target.

[0120] S502, based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, the camera to be calibrated is initially calibrated to obtain calibration parameters.

[0121] S503, based on the distortion coefficient, perform distortion correction on each target circle to obtain the center normalized coordinates of each target circle in the normalized plane.

[0122] S504, based on the calibration parameters, determine the projected pixel coordinates of the center normalized coordinates of each target circle in the calibration image.

[0123] S505 recalibrates the camera to be calibrated based on the projected pixel coordinates of each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, so as to update the calibration parameters.

[0124] The specific implementation methods of S501-S505 are the same as those of S201-S205, and will not be repeated here.

[0125] S506: Determine the calibration result corresponding to the camera to be calibrated, and return to S503 if the calibration result does not meet the calibration termination condition.

[0126] The calibration results include the number of calibration iterations or the change in reprojection error. The reprojection error is determined based on the difference between the projected pixel coordinates and the center pixel coordinates of each target circle.

[0127] Reprojection error refers to the distance error between the projected point and the actual observed point after a point in three-dimensional space is projected onto the image plane through a camera model.

[0128] In this context, the projected pixel coordinates corresponding to each target circle can be understood as the pixel coordinates obtained by reprojecting the center pixel coordinates of the target circle in the calibration image onto the calibration image. Therefore, the reprojection error can be determined based on the difference between the projected pixel coordinates of each target circle and the center pixel coordinates. Furthermore, the change in reprojection error can be determined based on the reprojection error after multiple calibration iterations. For example, the difference between two adjacent reprojection errors can be used as the change in reprojection error, or the rate of change of reprojection error over multiple consecutive iterations can be used as the change in reprojection error, etc., all of which are reasonable.

[0129] After recalibrating the camera to be calibrated to update the calibration parameters, it is possible to further determine whether it is necessary to continue iterative calibration of the camera to be calibrated based on the calibration results corresponding to the camera to be calibrated, so as to improve the accuracy of the final calibration parameters and the final calibration accuracy of the calibration results.

[0130] In this way, after recalibrating the camera to be calibrated to update the calibration parameters, the calibration result of the camera to be calibrated is first determined. This calibration result can be used to characterize the current calibration effect of the camera to be calibrated, and may include the number of calibration iterations or the amount of change in reprojection error.

[0131] If the calibration result does not meet the calibration termination condition, it can be said that the current calibration effect of the camera to be calibrated has not achieved the expected effect, and recalibration is required. In order to further eliminate the influence of geometric distortion introduced by the lens of the camera to be calibrated on the imaging accuracy of the calibration image, S503 can be executed again to perform distortion correction on each target circle according to the distortion coefficient in the latest calibration parameters, so as to recalibrate the camera to be calibrated again.

[0132] Correspondingly, if the calibration result meets the calibration termination condition, it can be said that the current calibration effect of the camera to be calibrated has reached the expected effect. Therefore, the calibration of the camera to be calibrated can be stopped, and the camera to be calibrated after this calibration is taken as the calibrated camera. The calibration parameters updated after this calibration can be used as calibration parameters in subsequent geometric measurement, image correction, 3D reconstruction and other tasks.

[0133] In one optional embodiment, the calibration result includes the number of calibration iterations. After recalibrating the camera to be calibrated to update the calibration parameters, it can be determined whether the number of calibration iterations after this calibration has reached a preset number (i.e., whether the number of calibration iterations is equal to the preset number). That is, the calibration termination condition is that the number of calibration iterations reaches the preset number. The preset number can be limited based on empirical values, experimental values ​​from multiple trials, and the needs of actual applications, and is usually a positive integer greater than 1. For example, 10; however, no specific limitation is made. If the preset number is not reached, it can be determined that the calibration result does not meet the calibration termination condition, and thus, execution returns to step S503. Conversely, if the preset number is reached, it can be determined that the calibration result meets the calibration termination condition, and thus, the continued calibration of the camera to be calibrated is stopped, resulting in the calibrated camera and calibration parameters.

[0134] In this embodiment, after recalibrating the camera to be calibrated to update the calibration parameters, it can be determined whether to continue recalibrating the camera to be calibrated again by checking whether the calibration result of the camera to be calibrated meets the calibration termination condition. Thus, through multiple iterations of calibration, the calibration effect of the camera to be calibrated can be gradually optimized, the accuracy of the calibration parameters of the camera to be calibrated can be gradually improved, and the accuracy and convergence stability of the camera calibration can be significantly improved in the end, thereby improving the adaptability of the camera calibration method in high-precision camera calibration application scenarios.

[0135] Based on the above embodiments, in an exemplary embodiment, the determination of the calibration result is further refined. Optionally, the calibration result includes the change in reprojection error, such as... Figure 6 As shown, the following steps may be included:

[0136] S601, determine the current reprojection error based on the average distance between the projected pixel coordinates and the center pixel coordinates of each target circle.

[0137] After recalibrating the camera to be calibrated to update the calibration parameters, the distance between the projected pixel coordinates and the center pixel coordinates corresponding to each target circle can be determined first to obtain multiple distances. Then, the average value of the multiple distances can be determined as the current reprojection error.

[0138] Optionally, the current reprojection error can be determined using the following formula:

[0139]

[0140] in, This represents the current reprojection error. The total number of target circles, For the first The center pixel coordinates of the target circle For the first The projected pixel coordinates corresponding to each target circle; This represents the Euclidean norm, which is used to calculate the distance between two coordinates.

[0141] Since the projected pixel coordinates corresponding to each target circle are equivalent to the pixel coordinates after distortion correction of the center pixel coordinates corresponding to each target circle, the smaller the difference between the projected pixel coordinates corresponding to each target circle and the center pixel coordinates, the smaller the distortion of the camera to be calibrated, the better the current calibration effect of the camera to be calibrated, and the higher the accuracy of the obtained calibration parameters.

[0142] S602, determine the change in reprojection error based on the difference between the current reprojection error and the historical reprojection error after the last calibration iteration.

[0143] Since the reprojection error is determined each time the camera to be calibrated is recalibrated to update the calibration parameters, after obtaining the current reprojection error, the difference between the current reprojection error and the historical reprojection error after the last calibration iteration can be determined, and this difference can be used as the change in the reprojection error.

[0144] Among them, the change in reprojection error can reflect the degree of optimization of the calibration effect of the camera to be calibrated after two adjacent calibration iterations, as well as the degree of improvement in the accuracy of the calibration parameters of the camera to be calibrated.

[0145] Therefore, if the change in reprojection error is less than a certain amount, it indicates that the optimization of the calibration effect of the camera to be calibrated by the recalibration iteration and the improvement in the accuracy of the calibration parameters of the camera to be calibrated are both small. This indicates that the camera to be calibrated already has a relatively high calibration effect and the calibration parameters of the camera to be calibrated already have a high accuracy. Thus, it can be considered that the current calibration effect of the camera to be calibrated has reached the expected effect, and the continued calibration of the camera to be calibrated can be stopped to obtain the calibrated camera and calibration parameters.

[0146] Based on this, optionally, when the calibration result includes the change in reprojection error, the calibration termination condition can be that the change in reprojection error is less than a preset change. The preset change can be limited based on empirical values, experimental values ​​from multiple tests, and the needs of actual applications; for example, 0.1, etc., without specific limitation. Furthermore, if the change in reprojection error is not less than the preset change, it can be determined that the calibration result does not meet the calibration termination condition, and thus, execution returns to step S503; correspondingly, if the change in reprojection error is less than the preset change, it can be determined that the calibration result meets the calibration termination condition, and thus, the continued calibration of the camera to be calibrated is stopped, obtaining the calibrated camera and calibration parameters.

[0147] In this embodiment, the change in reprojection error is used as the calibration result corresponding to the camera to be calibrated. After recalibrating the camera to be calibrated to update the calibration parameters, it can be determined whether to continue recalibrating the camera to be calibrated again by checking whether the calibration result of the camera to be calibrated meets the calibration termination condition. Thus, through multiple iterative calibrations, the calibration effect of the camera to be calibrated can be gradually optimized, the accuracy of the calibration parameters of the camera to be calibrated can be gradually improved, and the accuracy and convergence stability of the camera calibration can be significantly improved in the end, thereby improving the adaptability of the camera calibration method in high-precision camera calibration application scenarios.

[0148] Based on the above embodiments, in one exemplary embodiment, the initial calibration of the camera to be calibrated is further defined. Optionally, such as... Figure 7 As shown, the following steps may be included:

[0149] S701 performs edge detection on the calibration image to obtain multiple initial graphics in the calibration image.

[0150] When calibrating a camera, it is necessary to first determine the target circle corresponding to each solid circle in the circular array target in the calibration image. Therefore, each target circle in each calibration image can be determined first.

[0151] After obtaining the calibration image, edge detection is first performed on it. For example, the Canny (Canny Edge Detection) algorithm is used to perform edge detection on the calibration image to obtain multiple initial shapes in the calibration image. The edge contour pixels of each initial shape extracted by edge retrieval constitute a set of contour pixels.

[0152] Optionally, to ensure the accuracy of the target circle in the determined calibration image, and thus the accuracy of the final calibration of the camera to be calibrated, after acquiring the image of the circular array target from the camera to be calibrated, the image can be preprocessed, for example, by grayscale conversion or noise suppression, to improve the image quality. Using the preprocessed image as the calibration image can then improve the boundary features of each shape in the calibration image, thereby improving the accuracy of the extracted initial shapes.

[0153] S702, based on the graphic parameters of each initial graphic, select each target circle from multiple initial graphics.

[0154] The graphic parameters include perimeter, area, and roundness index, with the roundness index reflecting how closely the graphic approximates a standard circle.

[0155] The Circularity Index, also known as the Circularity Factor, is a numerical index that measures how similar a planar graphic (or the cross-section of a three-dimensional object) is to a standard circle. The Circularity Index of a standard circle is 1, and the Circularity Index of other shapes is less than 1. The smaller the value, the greater the difference between the shape and the circle.

[0156] Due to the influence of light and interference in the environment of the circular array target, the number of initial patterns detected in the calibration image can be much greater than the number of solid circles in the circular array target. In addition, some initial patterns have shapes that differ significantly from circles, such as rectangles and irregular shapes. Therefore, based on the graphic parameters of each initial pattern, target circles that correspond to solid circles in the circular array target can be selected from multiple initial patterns.

[0157] Considering that the solid circles in the circular array target have a fixed size and shape, we can select from multiple initial graphics that match the size and shape distribution of the solid circles in the circular array target, based on graphic parameters that characterize the size and shape of the initial graphics, to serve as the target circles. Specifically, graphic parameters can include area, perimeter, and roundness index. Thus, we can use area and perimeter to filter graphic size, and roundness index to filter graphic shape.

[0158] In the calibration graphic, each initial graphic is a contour composed of multiple edge contour pixels. The perimeter mentioned above is actually the edge length of the contour composed of edge contour pixels, and the area mentioned above is actually the area of ​​the closed region of the contour composed of edge contour pixels. The roundness index mentioned above is used to reflect the degree of closeness between the contour composed of edge contour pixels and the standard circle. When the roundness index is close to 1, it means that the contour composed of edge contour pixels is closer to the standard circle.

[0159] Optionally, the target circle in the calibration image can be selected using the following formula, thereby satisfying the following formula as the initial shape of the target circle in the calibration image.

[0160]

[0161] in, For the first The perimeter of the initial shape, in pixels; Indicates the first The area of ​​an initial shape, in square pixels; For the first The roundness index of an initial shape, where a roundness index close to 1 indicates that the initial shape is closer to a standard circle. , , , , and The parameter threshold can be set according to the solid circles in the circular array target, the resolution of the calibration image, etc., and no specific limitation is made.

[0162] S703: Identify the target circle corresponding to each solid circle in the circular array target from each target circle, and determine the center pixel coordinates of the target circle corresponding to each solid circle.

[0163] After filtering out the target circles in the calibration image, the target circle corresponding to each solid circle in the circular array target can be identified from each target circle. For example, each solid circle may have different image features. Therefore, for each calibration image, the target circle corresponding to each solid circle in the circular array target can be identified from each target circle by recognizing the image features of each target circle in the calibration image.

[0164] Furthermore, the center pixel coordinates of the target circle corresponding to each solid circle can be determined. For example, an ellipse fitting can be performed on each target circle to obtain the center pixel coordinates of the fitted ellipse, which can then be used as the center pixel coordinates of the target circle. Thus, after identifying the target circle corresponding to each solid circle in the circular array target, the center pixel coordinates of the target circle corresponding to each solid circle can be determined.

[0165] In one optional embodiment, when multiple calibration images are obtained, the above S701-S703 can be performed on each calibration image to obtain the target circle corresponding to each solid circle in the circular array target in each calibration image.

[0166] S704 performs initial calibration of the camera to be calibrated based on the world coordinates of the center of each solid circle and the center pixel coordinates of the target circle corresponding to each solid circle.

[0167] The specific implementation of S704 is the same as that of S202, and will not be repeated here.

[0168] In this embodiment, each target circle in the calibration image can be filtered according to the graphic parameters of the initial graphic in the calibration image to avoid the influence of interfering graphics in the calibration image on subsequent camera calibration, thereby ultimately improving the accuracy and convergence stability of camera calibration.

[0169] Based on the above embodiments, in one exemplary embodiment, the identification of the target circle corresponding to the solid circle is further refined. Optionally, the plurality of solid circles includes a first solid circle and a second solid circle, wherein the diameter of the first solid circle is larger than the diameter of the second solid circle; as shown... Figure 8 As shown, the following steps may be included:

[0170] S801, select a specified number of target circles from each target circle according to the specified parameters in descending order, and use them as the first target circles, and identify the first target circle corresponding to each first solid circle from each first target circle.

[0171] The specified parameters include perimeter or area, and the specified quantity is the number of the first solid circles.

[0172] In some cases, to improve the recognition efficiency of the target circle corresponding to each solid circle in a circular array target, and thus improve the camera calibration efficiency, multiple solid circles with prominent features can be set among the multiple solid circles included in the circular array target. First, the target circle corresponding to the solid circle with prominent features is located in the calibration image. Then, based on the positional relationship between the solid circles in the circular array target and the solid circles with prominent features, and the positional relationship between other target circles in the calibration image and the target circles corresponding to the solid circles with prominent features, the target circles corresponding to the other solid circles in the calibration image are determined. For example, as... Figure 2B As shown, it includes five large circles, I, II, III, IV and V, whose diameters are larger than the diameters of the other circles.

[0173] Based on this, the circular array target may include a first solid circle and a second solid circle, wherein the diameter of the first solid circle is larger than the diameter of the second solid circle, and the number of first solid circles is a specified number. Optionally, the number of first solid circles is not less than 5. Furthermore, since each solid circle in the circular array target corresponds to a target circle in the calibration image, the diameter of the target circle corresponding to the first solid circle is larger than the diameter of the target circle corresponding to the second solid circle, and the larger the diameter of the circle, the larger its area and circumference.

[0174] In this way, after selecting the target circles in the calibration image, the circumference or area of ​​each target circle can be determined as a specified parameter. The target circles are then sorted in descending order of this parameter. Starting with the first target circle in the sorted array, the target circles within the first specified number of positions are selected as the first target circles. These selected first target circles then correspond one-to-one with the first solid circles in the circular array target, allowing for the identification of the first target circle corresponding to each first solid circle.

[0175] Optionally, for each first solid circle, the positional relationship between the first solid circle and another first solid circle can be determined as a first positional relationship. Correspondingly, for the first target circle corresponding to the first solid circle and the other first target circle corresponding to the other first solid circle, the positional relationship between the first target circle and the other first target circle can be determined as a second positional relationship. The first positional relationship and the second positional relationship are the same. Therefore, the first target circle corresponding to each first solid circle can be identified from each first target circle based on the positional relationship between each solid circle and the positional relationship between each target circle.

[0176] For example, with Figure 2B Taking the circular array target shown as an example, the method for identifying the first target circle corresponding to each first solid circle from each first target circle will be explained. Wherein, the first solid circle is... Figure 2B The large circles I, II, III, IV, and V are used as examples, and it is further assumed that the first target circles are target circles A, B, C, D, and E selected from the calibration image. Specifically:

[0177] 1) Select the two farthest target circles from target circles A, B, C, D, and E, let's call them target circles B and E. Then select the two closest target circles from the same set of target circles, let's call them target circles C and D. Correspondingly, such as... Figure 2B As shown, among large circles I, II, III, IV and V, the two solid circles that are farthest apart are large circles II and V, and the two solid circles that are closest to each other are large circles III and IV.

[0178] Each of the large circles II and V corresponds to one of the target circles B and E, respectively, while each of the large circles III and IV corresponds to one of the target circles C and D, respectively. Therefore, it can be determined that large circle I corresponds to target circle A.

[0179] 2) For target circle E, construct line AE connecting target circle E and target circle A, and construct line AC connecting target circle A and target circle C, as well as line AD connecting target circle A and target circle D. Then, determine the angle EAD between line AE and line AD, and the angle EAC between line AE and line AC. Select the smaller angle between angles EAD and EAC, and assume it to be angle EAD.

[0180] For target circle B, construct the line AB connecting target circle B and target circle A. Then, we can determine the angle BAD between line AB and line AD, and the angle BAC between line AB and line AC, and select the smaller angle between BAD and BAC, assuming it to be angle BAC.

[0181] Next, select the smallest included angle between included angle EAD and included angle BAC, let's say it is included angle BAC. Then included angle BAC is the smallest included angle among any two lines that form the multiple included angles between the target circle A and the other great circles B, C, D and E respectively.

[0182] Correspondingly, such as Figure 2B As shown, among the great circles I, II, III, IV and V, among the various angles formed by any two lines connecting great circle I to other great circles II, III, IV and V, the angle with great circle I as the vertex and the line connecting great circle I and great circle II, and the line connecting great circle I and great circle III as the sides is the smallest.

[0183] Based on this, large circles I, II, and III correspond to one of the target circles A, B, and C, respectively. Thus, when large circle I corresponds to target circle A, large circle II corresponds to one of the target circles B and E, and large circle III corresponds to one of the target circles C and D, it can be determined that large circle II corresponds to target circle B, and large circle III corresponds to target circle C.

[0184] 3) If each of the large circles in large circles II and V corresponds to one of the target circles B and E respectively, and large circle II corresponds to target circle B, then large circle V corresponds to target circle E; correspondingly, if each of the large circles in large circles III and IV corresponds to one of the target circles C and D respectively, and large circle III corresponds to target circle C, then large circle IV corresponds to target circle D.

[0185] At this point, the first target circle corresponding to each large circle has been identified from the first target circles A, B, C, D, and E. Among them, large circle I corresponds to target circle A, large circle II corresponds to target circle B, large circle III corresponds to target circle C, large circle IV corresponds to target circle D, and large circle V corresponds to target circle E.

[0186] S802, based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the first target circle corresponding to each first solid circle, determine the coordinate transformation relationship between the world coordinate system corresponding to the circular array target and the pixel coordinate system corresponding to the calibration image.

[0187] After identifying the first target circle corresponding to each first solid circle from each first target circle, the coordinate transformation relationship between the world coordinate system corresponding to the circular array target and the pixel coordinate system corresponding to the calibration image can be determined based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the first target circle corresponding to each first solid circle.

[0188] For example, if the world coordinates of the center of each solid circle in the circular array target are its two-dimensional homogeneous coordinates on the circular array target, a homography matrix can be determined based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the corresponding first target circle, serving as the aforementioned coordinate transformation relationship. Thus, the two-dimensional homogeneous coordinates of each second solid circle in the circular array target can be mapped onto the calibration image based on the aforementioned homography matrix.

[0189] In one optional embodiment, when multiple calibration images are acquired, for each calibration image, a first target circle corresponding to each first solid circle in the calibration image is determined. Then, based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the corresponding first target circle in the calibration image, a coordinate transformation relationship between the world coordinate system of the circular array target and the pixel coordinate system of the calibration image is determined. The obtained coordinate transformation relationship may differ for different calibration images.

[0190] S803, based on coordinate transformation relationship, identifies the second target circle corresponding to each second solid circle from all second target circles except the first target circle.

[0191] After obtaining the above coordinate transformation relationship, the world coordinates of the center of each second solid circle in the circular array target can be mapped to the calibration image according to the above coordinate transformation relationship, so as to obtain the pixel coordinates corresponding to the world coordinates of the center of the second solid circle in the calibration image, which are used as the center mapping coordinates corresponding to the second solid circle. Thus, according to the center mapping coordinates corresponding to each second solid circle and the center pixel coordinates of each second target circle other than the first target circle in the calibration image, the second target circle corresponding to each second solid circle can be identified from the second target circles other than the first target circle.

[0192] In one optional embodiment, S803 may include mapping the world coordinates of the center of each second solid image to the calibration image based on coordinate transformation relationships to obtain the center-mapped coordinates corresponding to each second solid circle. For each second solid circle, from all second target circles, the second target circle whose center pixel coordinates are closest to the center-mapped coordinates corresponding to the second solid circle is selected to obtain the second target circle corresponding to the second solid circle.

[0193] In this embodiment, after mapping the world coordinates of the center of each second solid circle to the calibration image based on coordinate transformation relationships to obtain the center-mapped coordinates corresponding to each second solid circle, for each second solid circle, the distance between the center-mapped coordinates of the second solid circle and the center pixel coordinates of each second target circle can be determined, such as Euclidean distance; then, the second target circle with the closest distance can be selected from all the second target circles as the second target circle corresponding to the second solid circle. Optionally, for each second solid circle, the distance between the center-mapped coordinates of the second solid circle and the center pixel coordinates of each target circle can also be determined, thereby selecting the target circle with the closest distance as the target circle corresponding to the second solid circle.

[0194] In this embodiment, by setting first and second solid circles with different diameters in the circular array target, the first target circle corresponding to the first solid circle can be selected from the calibration image according to specified parameters. After identifying the first target circle corresponding to each first solid circle from each first target circle, the second target circle corresponding to each second solid circle can be identified from each second target circle other than the first target circle by means of the coordinate transformation relationship determined according to the world coordinates of the center of each first solid circle and the center pixel coordinates of the first target circle corresponding to each first solid circle. This simplifies the identification process of the second target circle corresponding to each second solid circle, thereby improving the identification efficiency of the target circle corresponding to each solid circle in the circular array target, and thus improving the camera calibration efficiency.

[0195] Based on the above embodiments, in an exemplary embodiment, using Figure 2B As shown, the plurality of solid circles includes a first solid circle and a second solid circle, wherein the diameter of the first solid circle is larger than the diameter of the second solid circle, as shown. Figure 9 As shown, the camera calibration method may include the following steps:

[0196] S901, acquire the calibration image obtained by the camera to be calibrated from the image acquisition of the circular array target.

[0197] S902, perform edge detection on the calibration image to obtain multiple initial shapes in the calibration image.

[0198] S903, based on the graphic parameters of each initial graphic, selects each target from multiple initial graphics.

[0199] S904, select a specified number of target circles from each target circle according to the specified parameters in descending order, and use them as the first target circles, and identify the first target circle corresponding to each first solid circle from each first target circle.

[0200] S905, based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the first target circle corresponding to each first solid circle, determine the coordinate transformation relationship between the world coordinate system corresponding to the circular array target and the pixel coordinate system corresponding to the calibration image.

[0201] S906, based on the coordinate transformation relationship, the world coordinates of the center of each second solid image are mapped to the calibration image to obtain the center mapping coordinates corresponding to each second solid circle. For each second solid circle, the second target circle whose center pixel coordinates are closest to the center mapping coordinates corresponding to the second solid circle is selected from each second target circle to obtain the second target circle corresponding to the second solid circle.

[0202] S907 performs initial calibration of the camera to be calibrated based on the world coordinates of the center of each solid circle and the center pixel coordinates of the target circle corresponding to each solid circle, so as to obtain calibration parameters.

[0203] S908, based on the distortion coefficient, perform distortion correction on each target circle to obtain the calibration ellipse corresponding to the target circle in the normalized plane, and perform ellipse fitting on each calibration ellipse to obtain the calibration ellipse.

[0204] S909, based on the extrinsic parameters, determine the first hidden point in the horizontal direction and the second hidden point in the vertical direction in the normalized plane.

[0205] S910, for each calibration ellipse, determine the first set of tangent points corresponding to the first hidden point and the second set of tangent points corresponding to the second hidden point on the calibration ellipse, and determine the center coordinates of the calibration ellipse based on the coordinates of the intersection of the line connecting the first set of tangent points and the line connecting the second set of tangent points.

[0206] S911, based on the calibration parameters, determine the projected pixel coordinates of the center normalized coordinates of each target circle in the calibration image.

[0207] S912 recalibrates the camera to be calibrated based on the projected pixel coordinates of each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, so as to update the calibration parameters.

[0208] S913: Determine the current reprojection error based on the average distance between the projected pixel coordinates and the center pixel coordinates of each target circle, and determine the change in reprojection error based on the difference between the current reprojection error and the historical reprojection error after the last calibration iteration. If the change in reprojection error is greater than the preset change threshold, return to S908.

[0209] The specific implementation methods of S901-S913 are the same as those in the above method embodiments, and will not be repeated here.

[0210] Furthermore, based on the above embodiments, Table 1 below shows the comparison results of reprojection errors obtained by using conventional techniques and the camera calibration method provided in the embodiments of the present invention.

[0211] Table 1. Reprojection errors of different calibration methods

[0212]

[0213] According to the data in Table 1, it can be seen that, compared with the traditional method, the reprojection error of the camera calibration method provided in this embodiment of the invention is further reduced to 0.06061 pixels, which has higher calibration accuracy.

[0214] Furthermore, such as Figure 10 The figure shows the distribution of reprojection points of the camera after calibration using the camera calibration method provided in this embodiment of the invention. The horizontal and vertical coordinates are in pixels, and each point represents the offset of a pixel in the horizontal and vertical directions. Therefore, the errors of most reprojection points are concentrated within a small range, and the overall distribution of the reprojection points is compact, with no significant outliers, indicating that the camera calibration parameters are highly accurate.

[0215] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0216] Based on the same inventive concept, this application also provides a camera correction apparatus for implementing the camera correction method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more camera correction apparatus embodiments provided below can be found in the limitations of the camera correction method described above, and will not be repeated here.

[0217] In one exemplary embodiment, such as Figure 11 As shown, a camera calibration device is provided, including: an image acquisition module 1110, a first calibration module 1120, a distortion correction module 1130, a coordinate projection module 1140, and a second calibration module 1150, wherein:

[0218] The image acquisition module 1110 is used to acquire the calibration image obtained by the camera to be calibrated from the image acquisition of the circular array target; wherein, the circular array target includes multiple solid circles arranged in an orderly manner;

[0219] The first calibration module 1120 is used to perform initial calibration on the camera to be calibrated based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, so as to obtain calibration parameters; wherein, the calibration parameters include intrinsic parameters, extrinsic parameters and distortion coefficients;

[0220] The distortion correction module 1130 is used to perform distortion correction on each target circle according to the distortion coefficient, so as to obtain the center normalized coordinates of each target circle in the normalized plane.

[0221] The coordinate projection module 1140 is used to determine the projected pixel coordinates of the center normalized coordinates of each target circle in the calibration image according to the calibration parameters.

[0222] The second calibration module 1150 is used to recalibrate the camera to be calibrated based on the projected pixel coordinates corresponding to each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, so as to update the calibration parameters.

[0223] In an exemplary embodiment, the distortion correction module 1130 includes: a distortion correction unit, configured to perform distortion correction on each target circle according to the distortion coefficient to obtain the calibration ellipse corresponding to the target circle in the normalized plane, and to perform ellipse fitting on each calibration ellipse to obtain the calibration ellipse; a hidden point determination unit, configured to determine a first hidden point in the horizontal direction and a second hidden point in the vertical direction in the normalized plane according to the extrinsic parameters; and a coordinate determination unit, configured to determine the center coordinates of each calibration ellipse according to the first hidden point and the second hidden point, and to determine the center normalized coordinates of the target circle corresponding to each calibration ellipse according to the center coordinates of each calibration ellipse.

[0224] In an exemplary embodiment, the coordinate determination unit is specifically used to determine, for each calibration ellipse, a first set of tangent points corresponding to the first hidden point and a second set of tangent points corresponding to the second hidden point; and to determine the center coordinates of the calibration ellipse based on the coordinates of the intersection of the line connecting the first set of tangent points and the line connecting the second set of tangent points.

[0225] In one exemplary embodiment, the camera calibration device further includes: a result determination module, configured to determine the calibration result corresponding to the camera to be calibrated after recalibrating the camera to be calibrated to update the calibration parameters, and to trigger the distortion correction module 1130 if the calibration result does not meet the calibration termination condition; wherein the calibration result includes the number of calibration iterations or the change in reprojection error, and the reprojection error is determined based on the difference between the projection pixel coordinates and the center pixel coordinates corresponding to each target circle.

[0226] In one exemplary embodiment, the calibration result includes the change in reprojection error; the result determination module is specifically used to: determine the current reprojection error based on the average distance between the projected pixel coordinates and the center pixel coordinates corresponding to each target circle; and determine the change in reprojection error based on the difference between the current reprojection error and the historical reprojection error after the last calibration iteration.

[0227] In an exemplary embodiment, the first calibration module 1120 includes: an edge detection unit, used to perform edge detection on the calibration image to obtain multiple initial graphics in the calibration image; a graphics filtering unit, used to filter target circles from the multiple initial graphics according to the graphics parameters of each initial graphic; wherein the graphics parameters include perimeter, area and roundness index, and the roundness index is used to reflect the degree of closeness of the graphic to a standard circle; a graphics recognition unit, used to identify the target circle corresponding to each solid circle in the circular array target from the target circles, and determine the center pixel coordinates of the target circle corresponding to each solid circle; and a camera calibration unit, used to perform initial calibration of the camera to be calibrated according to the world coordinates of the center of each solid circle and the center pixel coordinates of the target circle corresponding to each solid circle.

[0228] In one exemplary embodiment, the plurality of solid circles includes a first solid circle and a second solid circle, wherein the diameter of the first solid circle is larger than the diameter of the second solid circle; the image recognition unit includes: a first recognition subunit, configured to select a specified number of target circles from each target circle in descending order of specified parameters as first target circles, and to identify the first target circle corresponding to each first solid circle from each first target circle; wherein the specified parameters include circumference or area, and the specified number is the number of first solid circles; a relationship determination subunit, configured to determine the coordinate transformation relationship between the world coordinate system corresponding to the circular array target and the pixel coordinate system corresponding to the calibration image based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the first target circle corresponding to each first solid circle; and a second recognition subunit, configured to identify the second target circle corresponding to each second solid circle from each second target circle other than the first target circle based on the coordinate transformation relationship.

[0229] In an exemplary embodiment, the second identification subunit is specifically used to: map the world coordinates of the center of each second solid image to the calibration image based on the coordinate transformation relationship, so as to obtain the center mapping coordinates corresponding to each second solid circle; for each second solid circle, select the second target circle from each second target circle whose center pixel coordinates are closest to the center mapping coordinates corresponding to the second solid circle, so as to obtain the second target circle corresponding to the second solid circle.

[0230] Each module in the aforementioned camera calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0231] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as calibration images and the world coordinates of the centers of each solid circle in a circular array target. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a camera correction method.

[0232] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 13 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a camera correction method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0233] Those skilled in the art will understand that Figure 12 and 13The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0234] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0235] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0236] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0237] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0238] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0239] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A camera calibration method, characterized in that, The method includes: A calibration image is obtained by the camera to be calibrated capturing images of a circular array target; wherein the circular array target comprises multiple solid circles arranged in an orderly manner; The camera to be calibrated is initially calibrated based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, so as to obtain calibration parameters; wherein, the calibration parameters include intrinsic parameters, extrinsic parameters and distortion coefficients; Based on the distortion coefficient, distortion correction is performed on each target circle to obtain the calibration circle corresponding to the target circle in the normalized plane, and ellipse fitting is performed on each calibration circle to obtain the calibration ellipse; wherein, the normalized plane refers to a virtual plane in the camera coordinate system, which is used to unify cameras with different focal lengths and principal point positions to the same reference. After the coordinates of points in the three-dimensional world coordinate system are projected onto the normalized plane, the coordinates are normalized to dimensionless normalized coordinates; Based on the extrinsic parameters, determine the first hidden point in the horizontal direction and the second hidden point in the vertical direction in the normalized plane; Based on the first and second hidden point, determine the center coordinates of each calibration ellipse, and based on the center coordinates of each calibration ellipse, determine the center normalized coordinates of the target circle corresponding to each calibration ellipse. Based on the calibration parameters, determine the projected pixel coordinates of the center normalized coordinates of each target circle in the calibration image; The camera to be calibrated is recalibrated based on the projected pixel coordinates of each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, so as to update the calibration parameters.

2. The method according to claim 1, characterized in that, The initial calibration of the camera to be calibrated is performed based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, to obtain calibration parameters, including: Based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, the homography matrix is ​​solved by the least squares method to obtain multiple homography matrices. For the obtained homography matrices, constraint equations are constructed, and intrinsic parameter matrices are derived using matrix decomposition to obtain the intrinsic parameters. Based on the obtained intrinsic parameter matrix and the homography matrix of the calibration image, the extrinsic parameters are solved by orthogonalization. A distortion model is introduced, and the distortion coefficients are estimated iteratively using the intrinsic parameters and the extrinsic parameters. The intrinsic parameters, extrinsic parameters, and distortion coefficients are used as calibration parameters.

3. The method according to claim 2, characterized in that, The step of determining the center coordinates of each calibration ellipse based on the first and second hidden point includes: For each calibration ellipse, determine the first set of tangent points on the calibration ellipse corresponding to the first hidden point and the second set of tangent points corresponding to the second hidden point; The coordinates of the center of the calibration ellipse are determined based on the coordinates of the intersection point of the first set of tangent points and the second set of tangent points.

4. The method according to any one of claims 1-3, characterized in that, After recalibrating the camera to be calibrated to update the calibration parameters, the method further includes: Determine the calibration result corresponding to the camera to be calibrated, and if the calibration result does not meet the calibration termination condition, return to execute the operation of distortion correction for each target circle based on the distortion coefficient; The calibration result includes the number of calibration iterations or the change in reprojection error, wherein the reprojection error is determined based on the difference between the projected pixel coordinates and the center pixel coordinates of each target circle.

5. The method according to claim 4, characterized in that, The calibration result includes the change in the reprojection error; determining the calibration result corresponding to the camera to be calibrated includes: The current reprojection error is determined based on the average distance between the projected pixel coordinates and the center pixel coordinates of each target circle. The change in reprojection error is determined based on the difference between the current reprojection error and the historical reprojection error after the last calibration iteration.

6. The method according to any one of claims 1-3, characterized in that, The initial calibration of the camera to be calibrated, based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, includes: Edge detection is performed on the calibration image to obtain multiple initial shapes in the calibration image; Based on the graphic parameters of each initial graphic, target circles are selected from multiple initial graphics; wherein, the graphic parameters include perimeter, area and roundness index, and the roundness index is used to reflect the degree of similarity between the graphic and a standard circle; From each target circle, identify the target circle corresponding to each solid circle in the circular array target, and determine the center pixel coordinates of the target circle corresponding to each solid circle; The camera to be calibrated is initially calibrated based on the world coordinates of the center of each solid circle and the center pixel coordinates of the target circle corresponding to each solid circle.

7. The method according to claim 6, characterized in that, The plurality of solid circles includes a first solid circle and a second solid circle, wherein the diameter of the first solid circle is larger than the diameter of the second solid circle; the step of identifying the target circle corresponding to each solid circle in the circular array target from among the target circles includes: According to the specified parameters in descending order, a specified number of target circles are selected from each target circle as the first target circles, and the first target circle corresponding to each first solid circle is identified from each first target circle; wherein, the specified parameters include circumference or area, and the specified number is the number of first solid circles; Based on the world coordinates of the center of each first solid circle and the center pixel coordinates of the first target circle corresponding to each first solid circle, determine the coordinate transformation relationship between the world coordinate system corresponding to the circular array target and the pixel coordinate system corresponding to the calibration image; Based on the coordinate transformation relationship, the second target circle corresponding to each second solid circle is identified from all the second target circles other than the first target circle.

8. The method according to claim 7, characterized in that, The step of identifying the second target circle corresponding to each second solid circle from among the second target circles other than the first target circle, based on the coordinate transformation relationship, includes: Based on the coordinate transformation relationship, the world coordinates of the center of each second solid image are mapped to the calibration image to obtain the center mapping coordinates corresponding to each second solid circle. For each second solid circle, select the second target circle whose center pixel coordinates are closest to the center mapping coordinates of the corresponding second solid circle from among the second target circles, so as to obtain the second target circle corresponding to the second solid circle.

9. A camera calibration device, characterized in that, The device includes: The image acquisition module is used to acquire the calibration image obtained by the camera to be calibrated from the image acquisition of the circular array target; wherein, the circular array target includes multiple solid circles arranged in an orderly manner; The first calibration module is used to perform initial calibration on the camera to be calibrated based on the world coordinates of the center of each solid circle in the circular array target and the center pixel coordinates of the target circle corresponding to each solid circle in the calibration image, so as to obtain calibration parameters; wherein, the calibration parameters include intrinsic parameters, extrinsic parameters and distortion coefficients; The distortion correction module is used to perform distortion correction on each target circle according to the distortion coefficient, so as to obtain the center normalized coordinates of each target circle in the normalized plane; wherein, the normalized plane refers to a virtual plane in the camera coordinate system, which is used to unify cameras with different focal lengths and principal point positions to the same reference. After the coordinates of points in the three-dimensional world coordinate system are projected onto the normalized plane, the coordinates are normalized into dimensionless normalized coordinates. The coordinate projection module is used to determine the projected pixel coordinates of the center normalized coordinates of each target circle in the calibration image according to the calibration parameters. The second calibration module is used to recalibrate the camera to be calibrated based on the projected pixel coordinates corresponding to each target circle and the world coordinates of the center of the solid circle corresponding to each target circle, so as to update the calibration parameters. The distortion correction module includes: The distortion correction unit is used to perform distortion correction on each target circle according to the distortion coefficient to obtain the calibration ellipse corresponding to the target circle in the normalized plane, and to perform ellipse fitting on each calibration ellipse to obtain the calibration ellipse. The hidden point determination unit is used to determine the first hidden point in the horizontal direction and the second hidden point in the vertical direction in the normalized plane based on the external parameters. The coordinate determination unit is used to determine the center coordinates of each calibration ellipse based on the first and second hidden point, and to determine the center normalized coordinates of the target circle corresponding to each calibration ellipse based on the center coordinates of each calibration ellipse.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images