Monocular vision camera calibration and downward inclination angle measurement method

Abstract

The invention discloses a monocular vision camera calibration and downward inclination angle measurement method. The method comprises the following steps: acquiring a calibration image; extracting a pixel coordinate value of each ground feature point in the calibration image, and converting each pixel coordinate value into an image physical coordinate value; constructing a unified optimization model based on the physical coordinate values of the image and the corresponding known real distance; solving the unified optimization model by adopting a numerical optimization method of joint iteration of a Newton method and a Bruent method, and synchronously obtaining a calibration angle value of a downward inclination angle and parameters of a variable focal length model; and completing the calibration of the monocular vision camera by using the calibration angle value and the parameters of the variable focal length model. By establishing a unified optimization model related to the downtilt angle and the variable focal length model, integrated calibration of the variable focal length and the downtilt angle can be realized; moreover, the variable focal length changes along with the physical coordinate value of the image, the pixel-level difference and the lens distortion are considered, and the calibration precision is improved. According to the method provided by the invention, low-cost, light-weight and high-precision one-time calibration is realized.

Description

Technical Field

[0001] This application relates to the technical field of computer vision, specifically to a method for calibrating and measuring the tilt angle of a monocular vision camera. Background Technology

[0002] The calibration of monocular vision cameras and the determination of their tilt angle have wide applications in scenarios such as vehicle speed measurement, traffic monitoring, and roadside perception. Currently, calibration methods for monocular vision cameras include the checkerboard calibration method proposed by Zhang Zhengyou, the lidar-camera joint calibration method, and the calibration board-less method based on neural networks.

[0003] The checkerboard calibration method requires a calibration board with known geometric dimensions. After changes in camera height or tilt angle, multiple images need to be re-acquired. Assuming a constant focal length, it does not consider pixel-level differences and lens distortion, resulting in a significant increase in reprojection error as the field of view increases, leading to a large calibration error. The LiDAR-camera joint calibration method can achieve millimeter-level accuracy, but it relies on expensive external hardware, is complex, and is not suitable for large-scale roadside rapid deployment. The calibration board-less method based on neural networks requires a large amount of training data, and retraining is required after changes in camera attitude, resulting in poor repeatability.

[0004] Furthermore, current technologies for determining the tilt angle of monocular vision cameras rely on manual measurement using tilt meters. This manual measurement is subject to ±0.5° installation errors and human parallax, resulting in significant errors in tilt angle determination. In the field of monocular vision geometric ranging, Xue Lixia et al. ('Monocular Vision Ranging and Camera Focal Length Calibration', Scientific Programming, 2021) proposed a variable focal length calibration model based on polynomial regression, achieving high-precision ranging. However, the effective application of this method requires a known camera tilt angle. Gao Tian et al. ('Height-Variable Monocular Vision Ranging Technology for SmartAgriculture', IEEE Access, 2023) further explored this method, but their calibration results still depend on a fixed camera orientation. Therefore, current technologies lack an integrated 'focal length-tilt angle' calibration method that can automatically, accurately, and synchronously complete the 'focal length-tilt angle' calibration process when the camera's mounting orientation (tilt angle) is unknown. Summary of the Invention

[0005] In view of this, this application provides a method for calibrating and measuring the tilt angle of a monocular vision camera, which aims to solve the problem of how to automatically and synchronously complete the integrated calibration of the camera's variable focal length and tilt angle with high precision using only a single calibration image in the initial installation scenario where the camera's tilt angle is unknown. This achieves low-cost, lightweight, and high-precision one-time deployment calibration.

[0006] This application provides a method for calibrating and measuring the tilt angle of a monocular vision camera, including: A calibration image is acquired by a monocular vision camera at a known installation height. This image contains multiple ground feature points with known real-world coordinates. Pixel coordinates of each ground feature point are extracted and converted to corresponding image physical coordinates. Based on these image physical coordinates and the corresponding known real distances, a unified optimization model is constructed. The optimization variables of this model include: a unique tilt angle of the monocular vision camera, and parameters of a variable focal length model describing the change of the equivalent focal length with the image physical coordinates. The known real distance is the distance from the ground feature point corresponding to the image physical coordinates to a reference point, which is the intersection of the optical center of the monocular vision camera perpendicularly downwards with the plane containing the ground. A numerical optimization method combining Newton's method and Brent's method is used to solve the unified optimization model, simultaneously obtaining the calibration angle value of the tilt angle and the parameters of the variable focal length model. The monocular vision camera is then calibrated using the calibration angle value of the tilt angle and the parameters of the variable focal length model.

[0007] Optionally, the variable focal length model is solved in the following way: using the least squares algorithm, based on the image physical coordinates of each ground feature point and the equivalent focal length calculated based on the image physical coordinates, the parameters of the variable focal length model are fitted; wherein, the variable focal length model is a polynomial function of the image physical coordinates.

[0008] Alternatively, the variable focal length model can be represented as: in, The physical coordinates of the image are ( The equivalent focal length corresponding to the ground feature points. These are the parameters in the variable focal length model. All are integers greater than or equal to 0 and less than 4.

[0009] Optionally, the geometric relationships upon which the unified optimization model is based are described by the following nonlinear function: in, Indicates the known true distance. Indicates the height of the monocular vision camera above the ground. This represents the angle value of the tilt angle of a monocular vision camera. This indicates that the physical coordinates of the image are ( The predicted focal length is calculated from the ground feature points.

[0010] Optionally, after solving the unified optimization model, the following robust optimization steps are also included: obtaining the calibration angle values ​​of the downslope angle initially estimated based on multiple ground feature points; sorting the calibration angle values ​​and removing abnormal angle values ​​based on the box plot outlier detection method to obtain a set of effective angle values; using the average value of the set of effective angle values ​​as the initial estimated angle value of the downslope angle; and iteratively optimizing the unified optimization model based on the initial estimated angle value to obtain the target calibration angle value of the downslope angle.

[0011] Optionally, based on the initial estimated angle value, the unified optimization model is iteratively optimized to obtain the target calibration angle value of the downtilt angle, including: substituting the initial estimated angle value into the unified optimization model for calculation to obtain a reference specific distance; based on the difference between the reference specific distance and the average value of the known true distances corresponding to multiple ground feature points, the unified optimization model is iteratively optimized to obtain the target calibration angle value.

[0012] Optionally, the monocular vision camera calibration and tilt angle determination method further includes: acquiring multiple test feature points with known real-world coordinate values ​​and their image physical coordinate values; calculating the predicted world coordinate values ​​of each test feature point based on the calibration angle values, and calculating the positioning error of each test feature point based on the real-world coordinate values ​​and the corresponding predicted world coordinate values; fitting the positioning error with the corresponding image physical coordinate values ​​to obtain an error compensation function; and compensating for the positioning error of the feature point to be observed based on the error compensation function; the feature point to be observed is any point in the image acquired by the monocular vision camera after calibration.

[0013] Optionally, the error compensation function is a fifth-order polynomial function with respect to the physical coordinates of the image.

[0014] Optionally, based on the error compensation function, the positioning error of the feature point to be observed is compensated, including: obtaining the pixel coordinate values ​​of the feature point to be observed and converting them into the image physical coordinate values ​​of the feature point to be observed; substituting the image physical coordinate values ​​of the feature point to be observed into the error compensation function to obtain the error compensation value; calculating the initial world coordinate values ​​of the feature point to be observed based on the variable focal length model, the calibration angle value and the image physical coordinate values ​​of the feature point to be observed; and calculating the target world coordinate values ​​of the feature point to be observed based on the error compensation value and the initial world coordinate values ​​of the feature point to be observed.

[0015] According to the scheme provided in this application, a calibration image acquired by a monocular vision camera at a known installation height is obtained. The calibration image contains multiple ground feature points with known real-world coordinates. Pixel coordinate values ​​of each ground feature point are extracted and converted into corresponding image physical coordinate values. Based on the image physical coordinate values ​​and the known real distances corresponding to them, a unified optimization model is constructed. A numerical optimization method combining Newton's method and Brent's method is used to solve the unified optimization model, simultaneously obtaining the calibration angle value of the tilt angle and the parameters of the variable focal length model. The calibration of the monocular vision camera is completed using the calibration angle value of the tilt angle and the parameters of the variable focal length model. In this process, by establishing a unified optimization model related to the monocular camera's tilt angle and the variable focal length model describing the change of equivalent focal length with image physical coordinate values, integrated calibration of the camera's variable focal length and tilt angle can be achieved. Furthermore, the variable focal length model in the unified optimization model changes with the image physical coordinate values, making the focal length no longer a constant, taking into account pixel-level differences and lens distortion, thereby improving the accuracy of the calibration. Furthermore, the camera calibration method provided in this application does not require an inclinometer or lidar, and the calibration process is fully automated, achieving low-cost, lightweight, and high-precision one-time calibration. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 A schematic flowchart illustrating a method for calibrating and measuring the tilt angle of a monocular vision camera, provided in an embodiment of this application. Figure 2 This is a schematic diagram of the fitted surface of a variable focal length model provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0019] It should be noted that the terms "first" and "second" used in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0020] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0021] Figure 1 This is a flowchart illustrating a method for calibrating and measuring the tilt angle of a monocular vision camera according to an embodiment of this application. This method can be executed by an electronic device, such as a computer or server. Figure 1 As shown, the monocular vision camera calibration and tilt angle measurement method provided in this application includes: S101. Acquire calibration images captured by a monocular vision camera at a known installation height.

[0022] It should be noted that the calibration image acquired by the monocular vision camera can be an image obtained by taking a picture of a calibration board (or calibration paper) located below the monocular vision camera. The calibration image contains multiple ground feature points with known real-world coordinates. These ground feature points can be feature points in the calibration image, such as detection points (corner points) on the calibration board below the monocular vision camera. The world coordinate values ​​of these ground feature points are known, for example, they can be obtained through surveying data or pre-calibration.

[0023] S102. Extract the pixel coordinates of each surface feature point and convert each pixel coordinate into the corresponding physical coordinates of the image.

[0024] In some embodiments, the pixel coordinate values ​​are the pixel coordinates of ground feature points. These pixel coordinates can be represented as (u, v), where u and v represent the horizontal and vertical positions of the ground feature point's pixel in the image, respectively. Image processing algorithms can be used to extract the corresponding pixel coordinates of the ground feature points in the image. In practice, the pixel coordinates of multiple ground feature points can be obtained.

[0025] In some embodiments, the physical coordinates of the image can be physical coordinates on the image plane. The physical coordinates of the ground feature points can be calculated based on the number of pixels in the horizontal direction and the number of pixels in the vertical direction in the acquired image, as well as the pixel coordinates of the ground feature points.

[0026] S103. Based on the physical coordinates of the image and the known real distances corresponding to the physical coordinates of the image, construct a unified optimization model.

[0027] The optimization variables in the unified optimization model include: the tilt angle of a unique monocular vision camera, and parameters of a variable focal length model describing the change of the equivalent focal length with the physical coordinates of the image; the known true distance is the distance from the ground feature point corresponding to the physical coordinates of the image to the reference point, which is the intersection of the optical center of the monocular vision camera perpendicularly downwards and the plane containing the ground. Here, the tilt angle of the monocular vision camera refers to the angle between the optical axis of the monocular vision camera and the horizontal plane, which can be a plane perpendicular to the direction of gravity, i.e., the horizon.

[0028] In some embodiments, the equivalent focal length varies with the position of ground feature points, and the variable focal length model is a cubic polynomial function associated with the physical coordinates of the image. The equivalent focal length can be a quantity that changes with the physical coordinates of ground feature points in the image, and can be used to compensate for distortion, perspective effects, etc., in monocular vision cameras. The variable focal length model can be used to handle variable focal length and distortion in monocular vision camera calibration. For example, in wide-angle or fisheye lenses, the focal length may change with the image position, and this change can be approximated by the variable focal length model.

[0029] S104. A numerical optimization method combining Newton's method and Brent's method is used to solve the unified optimization model, and the calibration angle value of the downtilt angle and the parameters of the variable focal length model are obtained simultaneously.

[0030] In some embodiments, a unified optimization model can be directly solved using a numerical optimization method that combines Newton's method and Brent's method to simultaneously obtain the calibration angle value of the downtilt angle and the parameters of the variable focal length model.

[0031] In some embodiments, Newton's method and Brent's method can be combined to solve the nonlinear function established by the geometric relationship on which the unified optimization model is based. For each ground feature point, Newton's method can be used first. If Newton's method converges to the tolerance within the maximum number of iterations... If the solution is found to be within the range, the calibrated angle value of the downslope is obtained. If the Newton method fails (e.g., too many iterations or exceeding the range), the Brent method is used. If the Brent method also fails (e.g., no interval can be found or the number of iterations exceeds the range), the ground feature point can be discarded.

[0032] S105. Using the calibration angle value of the tilt angle and the parameters of the variable focal length model, complete the calibration of the monocular vision camera.

[0033] In some embodiments, the parameters of the variable focal length model and the image physical coordinates of the ground feature points can be substituted into the solved unified optimization model to obtain the equivalent focal length of the monocular vision camera (corresponding to the image physical coordinates of the ground feature points). The calibration angle values ​​of the equivalent focal length and the downtilt angle are used as the calibration results of the monocular vision camera, thereby completing the calibration of the monocular vision camera.

[0034] In this embodiment, a calibration image is acquired by a monocular vision camera at a known installation height. This calibration image contains multiple ground feature points with known real-world coordinates. Pixel coordinate values ​​of each ground feature point are extracted and converted into corresponding image physical coordinate values. Based on the image physical coordinate values ​​and the known real distances corresponding to them, a unified optimization model is constructed. A numerical optimization method combining Newton's method and Brent's method is used to solve the unified optimization model, simultaneously obtaining the calibration angle value of the tilt angle and the parameters of the variable focal length model. The monocular vision camera is calibrated using the calibration angle value of the tilt angle and the parameters of the variable focal length model. In this process, by establishing a unified optimization model related to the monocular camera's tilt angle and the variable focal length model describing the change of equivalent focal length with image physical coordinate values, integrated calibration of the camera's variable focal length and tilt angle can be achieved. Furthermore, the variable focal length model in the unified optimization model changes with the image physical coordinate values, making the focal length no longer constant, taking into account pixel-level differences and lens distortion, thereby improving the accuracy of the calibration. Furthermore, the camera calibration method provided in this application does not require an inclinometer or lidar, and the calibration process is fully automated, achieving low-cost, lightweight, and high-precision one-time calibration.

[0035] In some embodiments of this application, the variable focal length model can be solved in the following ways: Using the least squares algorithm, the parameters of the variable focal length model are fitted based on the physical coordinates of multiple feature points in the image and the equivalent focal length calculated based on the physical coordinates of the image; where the variable focal length model is a polynomial function of the physical coordinates of the image.

[0036] In some embodiments, based on the image physical coordinates and corresponding focal lengths of various ground feature points, the least squares algorithm is used to minimize the sum of squared errors between the equivalent focal length calculated by the variable focal length model and the actual focal length. Through continuous iterative optimization, the parameters in the variable focal length model associated with the image physical coordinates are obtained. The actual focal length can be calculated based on the pixel coordinates and physical coordinates of the ground feature points, as well as the imaging model of the monocular vision camera.

[0037] In some embodiments, the variable focal length model can be represented by the following formula (1): (1); in, The physical coordinates of the image are ( The equivalent focal length corresponding to the ground feature points. These are the parameters in the variable focal length model. All are integers greater than or equal to 0 and less than 4, i.e. The values ​​of are all 0, 1, 2, 3. The right side of the equation (1) is a double cubic polynomial, which, after expansion, includes the parameters of 16 variable focal length models. This variable focal length model can fit extremely complex nonlinear surfaces.

[0038] For example, such as Figure 2 The above is a schematic diagram of the fitting surface of a variable focal length model provided in an embodiment of this application, where F1 represents the focal length, x0 and y0 represent the image physical coordinates of ground feature points, and the goodness of fit of the model is... Therefore, the variable focal length model created in this scheme can fit the relationship between the physical coordinates of the image and the focal length well, and has high reliability.

[0039] Understandably, by using the least squares algorithm, the physical coordinates of each ground feature point in the image, and the equivalent focal length calculated based on the physical coordinates, the parameters of the variable focal length model are fitted. Based on these parameters, the physical coordinates of the image, and the variable focal length model, the predicted focal length of the monocular vision camera can be calculated. This takes into account pixel-level differences, ensuring that the physical coordinates of the ground feature points correspond to the camera's focal length, thereby improving the rationality and accuracy of the calibrated focal length.

[0040] In some embodiments of this application, the geometric relationship on which the unified optimization model is constructed in step S103 is described by the nonlinear function shown in the following formula (2): (2); in, Represents the known true distance (image physical coordinates ( (This corresponds to the distance from the ground feature point to the reference point, which is the intersection of the optical center of the monocular vision camera perpendicularly downwards and the plane containing the ground.) Indicates the height of the monocular vision camera above the ground. This represents the angle value of the tilt angle of a monocular vision camera. This indicates that the physical coordinates of the image are ( The predicted focal length is calculated from the ground feature points.

[0041] In the above formula (2), the left side of the equal sign is the known ground distance, and the right side is the equivalent focal length calculated based on the physical coordinates of the ground feature points in the image, the variable focal length model, the height of the monocular vision camera, and the angle of the downtilt angle to be solved. The distance was calculated theoretically based on the pinhole model and coordinate system rotation.

[0042] It should be noted that in the unified optimization model described above, for any single ground feature point, when the camera height... The known true distance corresponding to this ground feature point and its image physical coordinates ( When the information is known, formulas (1) and (2) still contain two unknowns: the equivalent focal length and the angle of the downtilt corresponding to the ground feature point. Therefore, a single equation cannot be directly solved. This application utilizes the fact that feature points at multiple (usually no less than 20) known locations in the same calibration image share the same downtilt angle. However, each of these features has different image coordinates and corresponding equivalent focal lengths. By combining the constraint equations (1) and (2) of these ground feature points, an overdetermined set of equations (or a unified optimization problem) is formed, which consists of the angle value of the unique unknown downtilt angle and multiple focal length parameters. Only by solving this system can the unique downtilt angle calibration value and the variable focal length model parameters describing the focal length distribution be stably and accurately derived simultaneously.

[0043] In some embodiments of this application, after solving the unified optimization model (step S104), the following robust optimization steps are also included: S106. Obtain the calibration angle value of the downtilt angle based on the preliminary estimate of multiple ground feature points.

[0044] In some embodiments, it can be seen from the nonlinear function shown in formula (2) that the angle value of the downtilt angle is related to the image physical coordinate value of the ground feature point. Different ground feature points have different image physical coordinates, and the corresponding predicted focal lengths are different. Therefore, the angle value of the downtilt angle calculated by the image position coordinates and the predicted focal length may also be different. Thus, multiple calibration angle values ​​of the downtilt angle can be calculated based on the image coordinate values ​​of multiple ground feature points. The multiple ground feature points can be some or all of the ground feature points in the calibration image.

[0045] S107. Sort the various calibrated angle values ​​and remove abnormal angle values ​​based on the box plot outlier detection method to obtain a set of valid angle values.

[0046] In some embodiments, multiple calibrated angle values ​​can be arranged in ascending order to obtain a sorting result; the first quartile and the third quartile are determined from the sorting result, and the abnormal angle values ​​and valid angle values ​​in the sorting result are determined according to the box plot outlier detection method; the abnormal angle values ​​in the sorting result are removed to obtain a set of valid angle values.

[0047] It should be noted that the first quartile, also known as the lower quartile, is the data value that is in the top 25% of the sorted results, meaning that 25% of the data in the sorted results are less than or equal to the first quartile; the third quartile, also known as the upper quartile, is the data value that is in the top 75% of the sorted results, meaning that 75% of the data are less than or equal to the third quartile.

[0048] In some embodiments, since the predicted angle values ​​of the downtilt angle calculated based on the image coordinate values ​​of different ground feature points may be different, after obtaining multiple predicted angle values, they can be arranged in ascending order of angle value to obtain a sorting result.

[0049] In some embodiments, the box plot outlier detection method can use the difference between the third quartile and the first quartile as the interquartile range (a key indicator for measuring the dispersion of data). Then, based on the interquartile range, the first quartile, and the third quartile, rules for judging outlier angle values ​​are set. Based on these rules, outlier angle values ​​in the sorting results are determined and removed. The remaining angle values ​​are the valid angle values.

[0050] For example, the rule for judging abnormal angle values ​​could be: less than Angle value; or, greater than The angle values, where Q1 and Q3 represent the first quartile and the third quartile, respectively. This represents the interquartile range, reflecting the dispersion of the middle 50% of the data. Choose 1.5 (mild rejection) or 3.0 (strict rejection). Based on this rule for judging abnormal angle values, in... If we take 1.5, then less than The angle value, and greater than The angle values ​​are determined from the sorting results. Then, all predicted angle values ​​that fall within the range of abnormal angle values ​​are filtered and removed to obtain the effective angle values.

[0051] S108. Use the average value of the effective angle value set as the initial estimated angle value of the downtilt angle.

[0052] In some embodiments, the arithmetic mean of multiple effective angle values ​​in the effective angle value set can be used as the initial estimated angle value of the downtilt angle.

[0053] It should be noted that, theoretically, the downtilt angle values ​​calculated from all ground feature points should be the same. However, in practice, due to factors such as camera noise, feature point extraction errors, and fitting errors in the variable focal length model, the downtilt angle values ​​obtained by substituting the image coordinates of each ground feature point into the nonlinear function may differ. Because factors such as ground feature point extraction noise and coordinate transformation errors may cause some ground feature points to be "outliers" in the calculated calibration angle values, box plots can automatically identify and exclude outliers far from the main data set, retaining statistically reasonable "inside points" and taking the average of the "inside points," thereby greatly improving the robustness and reliability of the final estimated initial angle value.

[0054] S109. Based on the initial estimated angle value, the unified optimization model is iteratively optimized and solved to obtain the target calibration angle value of the downtilt angle.

[0055] In some embodiments, the initial estimated angle value can be substituted into the unified optimization model for verification to determine the calibration accuracy of the initial estimated angle value. If the calibration accuracy does not meet the accuracy requirements, the unified optimization model can be iteratively optimized until the target calibration angle value that meets the accuracy requirements is obtained.

[0056] Understandably, by using the calibration angle value of the downslope angle initially estimated based on multiple ground feature points to detect outliers in the box plot and obtain the initial estimated angle value of the downslope angle, and then iteratively optimizing the unified optimization model based on the initial estimated angle value, the robustness of the unified optimization model can be improved, and a more accurate target calibration angle value of the downslope angle can be obtained.

[0057] In some embodiments of this application, the step S109, which involves iteratively optimizing the unified optimization model based on the initial estimated angle value to obtain the target calibration angle value of the downtilt angle, can be achieved through the following steps S1091 to S1092. Each step is described below.

[0058] S1091. Substitute the initial estimated angle value into the unified optimization model for calculation to obtain the reference specific distance.

[0059] In some embodiments, the initial estimated angle value, the average value of the image physical coordinates corresponding to each of the multiple ground feature points in step S106, and the average value of the equivalent focal length calculated based on the image physical coordinates corresponding to each of the multiple ground feature points in step S106 can be substituted into the nonlinear function of the unified optimization model shown in formula (2) to obtain a reference specific distance.

[0060] S1092. Based on the difference between the reference specific distance and the average value of the known true distances corresponding to multiple ground feature points, the unified optimization model is iteratively optimized and solved until the convergence condition is met, and the target calibration angle value is obtained.

[0061] In some embodiments, the absolute value of the difference between the average value of the known true distances corresponding to multiple ground feature points and the reference specific distance can be calculated. If the absolute value of the difference is greater than or equal to the distance threshold, the unified optimization model can be iteratively optimized and solved. The initial estimated angle value is continuously adjusted (e.g., increased by 5% or decreased by 5%) until the calculated absolute value of the difference is less than the distance threshold. Then, the convergence condition is considered to be met, and the target calibration angle value can be obtained.

[0062] It is understandable that by substituting the initial estimated angle value into the unified optimization model for calculation, and by iteratively optimizing the unified optimization model based on the calculated reference distance and the difference between the average values ​​of the known true distances corresponding to multiple ground feature points, the robustness of the unified optimization model can be continuously improved, thereby obtaining a more accurate calibrated angle value for the downtilt angle.

[0063] In some embodiments of this application, after step S105, steps S110 to S113 can also be performed. Each step will be described below.

[0064] S110. Obtain test feature points with known real-world coordinates and their image physical coordinates.

[0065] It should be noted that the test feature points can be feature points in the calibration image. The test feature points and ground feature points may be completely identical, partially identical, or different. The real-world coordinates of the test feature points can be coordinates in a world coordinate system. A world coordinate system can be pre-established with the ground directly below the monocular vision camera as the XOY plane, and the Z-axis pointing vertically upwards. The real-world coordinates of multiple test feature points can then be obtained.

[0066] It should be noted that the X and Y axes of the world coordinate system are located on the plane of the ground below the monocular vision camera, and the Z axis of the world coordinate system is perpendicular to the plane of the ground below the monocular vision camera. The directions of the X and Y axes of the world coordinate system can be arbitrarily determined on the plane of the ground below the monocular vision camera. For example, the direction of the X axis of the world coordinate system can be the same as the direction directly in front of the monocular vision camera, and the direction of the Y axis of the world coordinate system can be the same as the direction directly in front of the monocular vision camera. The method of setting the direction of the X and Y axes of the world coordinate system is merely an example and is not limited in this application.

[0067] In some embodiments, the real-world coordinates of multiple test feature points can be obtained in advance when the height of the monocular vision camera above the ground is a specific height. This specific height can be set arbitrarily, for example, 0.7 meters, 0.8 meters, 0.9 meters, 1.0 meters, 1.1 meters, 1.2 meters, etc., and then the real-world coordinate values ​​corresponding to each test feature point can be obtained respectively.

[0068] S111. Calculate the predicted world coordinates of each test feature point based on the calibrated angle values, and calculate the positioning error of each test feature point based on the real world coordinates and the corresponding predicted world coordinates.

[0069] In some embodiments, the equivalent focal lengths (calculated by a variable focal length model) obtained from multiple test feature points, the calibration angle value of the downtilt angle, and the height of the monocular vision camera can be substituted into the unified optimization model after calculation to obtain the predicted image physical coordinate values ​​corresponding to multiple test feature points. Then, each predicted physical coordinate value is converted into a predicted world coordinate value.

[0070] In some embodiments, the positioning error represents the difference between the real-world coordinates and the predicted world coordinates. For example, if the real-world coordinates of the test feature point are... , Predicted world coordinates are , The corresponding positioning error can be S112. Fit the positioning error to the corresponding predicted image physical coordinates to obtain the error compensation function.

[0071] In some embodiments, the error compensation function is a fifth-order polynomial function with respect to the physical coordinates of the image. This fifth-order polynomial function has an extremely strong nonlinear fitting capability. It can capture the very complex and non-uniform error field caused by changes in height. The magnitude and direction of this error field are different at different locations in the image. Therefore, the error compensation function can fit the nonlinear relationship between the positioning error and the predicted physical coordinates of the image very well.

[0072] In some embodiments, the error compensation function can be expressed by the following formulas (3) and (4): (3); (4); in, These represent the positioning errors corresponding to the x and y coordinates, respectively. The predicted physical coordinates of the test feature points in the image. Each represents a fitting coefficient associated with the physical coordinates of the predicted image.

[0073] In some embodiments, the error compensation function can be solved by the least squares method to minimize the difference between the prediction error calculated from the physical coordinates of the predicted image and the actual error (positioning error). Through iterative optimization, the fitting coefficients associated with the physical coordinates of the predicted image are continuously adjusted until the convergence condition is met, and the final fitting coefficients can be obtained.

[0074] Understandably, by fitting the positioning error to the corresponding predicted image physical coordinates, the intrinsic relationship between the positioning error and the predicted image physical coordinates can be continuously learned. This allows the camera to be recalibrated after its installation height changes, instead of recalibrating. Instead, the positioning error can be compensated by the created error compensation function, thereby improving the accuracy of feature point positioning.

[0075] S113. Based on the error compensation function, perform positioning error compensation on the feature points to be observed.

[0076] In this context, the feature point to be observed is any point in the image acquired by the monocular vision camera after calibration. After the monocular vision camera is calibrated, the world coordinate values ​​of the feature points in the image acquired by the monocular vision camera can be compensated based on an error compensation function. For example, in vehicle speed measurement applications, the error compensation function can be used to compensate for the world coordinate values ​​of the target vehicle estimated by the unified optimization model, thereby improving the positioning accuracy of the target vehicle and the accuracy of vehicle speed measurement.

[0077] In some embodiments of this application, the positioning error compensation of the observed feature point based on the error compensation function described in step S113 can be achieved by the following steps S1131 to S1134, and each step is described below.

[0078] S1131. Obtain the pixel coordinates of the feature points to be observed and convert them into the physical coordinates of the image of the feature points to be observed.

[0079] In some embodiments, image processing techniques, such as corner detection and feature matching, can be used to extract the pixel coordinates of the feature points to be observed from the images acquired by a monocular vision camera, and then convert the pixel coordinates of the feature points to be observed into the corresponding physical coordinates of the images.

[0080] S1132. Substitute the physical coordinates of the feature points to be observed into the error compensation function to obtain the error compensation value.

[0081] In some embodiments, the fifth-order polynomial function describes the nonlinear relationship between the physical coordinates of the image and the positioning error. The corresponding fitting coefficients have been obtained by solving the error compensation function during the camera calibration stage. Therefore, the physical coordinates of the feature point to be observed and the fitting coefficients can be directly substituted into the error compensation function and calculated by formulas (3) and (4) to obtain the error compensation value.

[0082] S1133. Based on the variable focal length model, the calibration angle value, and the image physical coordinate value of the feature point to be observed, calculate the initial world coordinate value of the feature point to be observed.

[0083] In some embodiments, the physical coordinates of the image of the feature point to be observed can be substituted into the variable focal length model in formula (1) to calculate the corresponding predicted focal length. Then, the predicted focal length and the calibration angle of the downtilt angle are substituted into the nonlinear function in formula (2) to calculate the distance from the feature point to be observed to the reference point (the intersection of the optical center of the monocular vision camera vertically downward and the plane where the ground is located). After that, the initial world coordinates of the feature point to be observed are calculated based on the geometric model and the distance from the feature point to be observed to the reference point.

[0084] S1134. Based on the error compensation value and the initial world coordinates of the feature point to be observed, calculate the target world coordinates of the feature point to be observed.

[0085] In some embodiments, the error compensation value and the initial world coordinate value corresponding to the feature point to be observed can be added together to obtain the target world coordinate value, which can be considered as the positioning coordinates after error compensation.

[0086] It is understandable that after camera calibration, for any feature point to be observed, the image physical coordinates of the feature point to be observed are substituted into the positioning error function already calculated in the calibration stage to obtain the error compensation value. This error compensation value is then used to compensate for the initial world coordinates calculated using the variable focal length model, calibration angle value, and image physical coordinates of the feature point to be observed. This can solve the problem that the world coordinates calculated based on the variable focal length model are inaccurate due to changes in camera height, errors in the calculation of image physical coordinates of the feature point to be measured, etc., thereby improving the positioning accuracy of the feature point to be measured.

[0087] In this embodiment, a calibration image is acquired by a monocular vision camera at a known installation height. This calibration image contains multiple ground feature points with known real-world coordinates. Pixel coordinate values ​​of each ground feature point are extracted and converted into corresponding image physical coordinate values. Based on the image physical coordinate values ​​and the corresponding known real distances, a unified optimization model is constructed. A numerical optimization method combining Newton's method and Brent's method is used to solve the unified optimization model, simultaneously obtaining the calibration angle value of the tilt angle and the parameters of the variable focal length model. The monocular vision camera is calibrated using the calibration angle value of the tilt angle and the parameters of the variable focal length model. In this process, by establishing a unified optimization model related to the monocular camera's tilt angle and the variable focal length model describing the change of equivalent focal length with image physical coordinate values, integrated calibration of the camera's variable focal length and tilt angle can be achieved. Furthermore, the variable focal length model in the unified optimization model changes with the image physical coordinate values, making the focal length no longer constant, taking into account pixel-level differences and lens distortion, thereby improving the calibration accuracy. Furthermore, the camera calibration method provided in this application does not require an inclinometer or lidar, and the calibration process is fully automated, achieving low-cost, lightweight, and high-precision one-time calibration.

[0088] Furthermore, through high-precision one-time calibration, more accurate positioning results of feature points within the camera's field of view can be obtained subsequently using a monocular vision camera, thereby improving the accuracy of speed measurement in the process of vehicle speed measurement based on traffic monitoring and roadside perception.

[0089] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0090] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0091] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0092] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A method for calibrating and measuring the tilt angle of a monocular vision camera, characterized in that, include: Acquire calibration images captured by a monocular vision camera at a known installation height; The calibration image contains multiple ground feature points with known real-world coordinates; Extract the pixel coordinates of each surface feature point and convert each pixel coordinate into its corresponding physical image coordinates; Based on the physical coordinates of the image and the known true distances corresponding to the physical coordinates, a unified optimization model is constructed. The optimization variables of the unified optimization model include: the tilt angle of a unique monocular vision camera, and parameters of a variable focal length model describing the change of the equivalent focal length with the physical coordinates of the image. The known true distance is the distance from the ground feature point corresponding to the physical coordinates of the image to a reference point, where the reference point is the intersection of the optical center of the monocular vision camera perpendicularly downwards with the plane containing the ground. A numerical optimization method combining Newton's method and Brent's method is used to solve the unified optimization model, and the calibration angle value of the downtilt angle and the parameters of the variable focal length model are obtained simultaneously. The monocular vision camera is calibrated using the calibration angle value of the tilt angle and the parameters of the variable focal length model.

2. The method according to claim 1, characterized in that, The variable focal length model is solved in the following way: Using the least squares algorithm, the parameters of the variable focal length model are fitted based on the image physical coordinates of the various ground feature points and the equivalent focal length calculated based on the image physical coordinates; wherein, the variable focal length model is a polynomial function of the image physical coordinates.

3. The method according to claim 2, characterized in that, The variable focal length model is represented as follows: in, The physical coordinates of the image are ( The equivalent focal length corresponding to the ground feature points. These are the parameters in the variable focal length model. All are integers greater than or equal to 0 and less than 4.

4. The method according to claim 1, characterized in that, The geometric relationships upon which the unified optimization model is constructed are described by the following nonlinear functions: in, This represents the known true distance. This indicates the height of the monocular vision camera above the ground. This represents the angle value of the downtilt angle of the monocular vision camera. This indicates that the physical coordinates of the image are ( The predicted focal length is calculated from the ground feature points.

5. The method according to claim 1, characterized in that, After solving the unified optimization model, the following robust optimization steps are also included: Obtain the calibration angle value of the downslope angle based on the preliminary estimate of multiple ground feature points; The calibrated angle values ​​are sorted, and outlier angle values ​​are removed based on the box plot outlier detection method to obtain a set of valid angle values. The average value of the set of effective angle values ​​is used as the initial estimated angle value of the downtilt angle; Based on the initial estimated angle value, the unified optimization model is iteratively optimized and solved to obtain the target calibration angle value of the downtilt angle.

6. The method according to claim 5, characterized in that, The step of iteratively optimizing the unified optimization model based on the initial estimated angle value to obtain the target calibration angle value of the downtilt angle includes: Substitute the initial estimated angle value into the unified optimization model for calculation to obtain the reference specific distance; Based on the difference between the reference specific distance and the average of the known true distances corresponding to the multiple ground feature points, the unified optimization model is iteratively optimized and solved until the convergence condition is met, and the target calibration angle value is obtained.

7. The method according to claim 1, characterized in that, The method further includes: Obtain test feature points with known real-world coordinates and their image physical coordinates; The predicted world coordinates of each test feature point are calculated based on the calibrated angle values, and the positioning error of each test feature point is calculated based on the real world coordinates and the corresponding predicted world coordinates. The positioning error is fitted with the corresponding image physical coordinate values ​​to obtain the error compensation function; Based on the error compensation function, the positioning error of the feature point to be observed is compensated; the feature point to be observed is any point in the image acquired by the monocular vision camera after calibration.

8. The method according to claim 7, characterized in that, The error compensation function is a fifth-order polynomial function of the physical coordinates of the image.

9. The method according to claim 7, characterized in that, The step of compensating for the positioning error of the observed feature point based on the error compensation function includes: Obtain the pixel coordinates of the feature point to be observed, and convert them into the physical coordinates of the image of the feature point to be observed; Substitute the physical coordinates of the observed feature points into the error compensation function to obtain the error compensation value; Based on the variable focal length model, the calibration angle value, and the image physical coordinate value of the feature point to be observed, the initial world coordinate value of the feature point to be observed is calculated. Based on the error compensation value and the initial world coordinates of the feature point to be observed, the target world coordinates of the feature point to be observed are calculated.

Images