Roadside camera calibration method, target positioning method and related equipment

Abstract

The invention provides a roadside camera calibration method, a target positioning method and related equipment. The method comprises the following steps: acquiring preset internal parameters and calibration data of a roadside camera; preprocessing the calibration data to obtain preprocessed calibration data; performing external parameter solving on the preprocessed calibration data by using a preset RANSACPnP algorithm and a Bayesian optimization method to obtain external parameters of the roadside camera; performing second-order curved surface fitting based on the preprocessed calibration data to obtain a global coarse plane model and a plurality of local fine plane models; and determining the internal parameter, the external parameter, the global coarse plane model and each local fine plane model as a calibration result of the roadside camera, thereby achieving the purpose of improving the camera calibration precision and further achieving the purpose of improving the target positioning precision.

Description

Technical Field

[0001] This invention relates to the field of roadside sensing technology, specifically to a roadside camera calibration method, a target localization method, and related equipment. Background Technology

[0002] Camera calibration is the process of determining the camera's intrinsic parameters (such as focal length, principal point position, and distortion coefficients) and extrinsic parameters (the camera's position and attitude in the world coordinate system) through specific mathematical models and algorithms. Its purpose is to establish a precise mapping relationship between pixel coordinates and world coordinates. Visual target localization utilizes the calibrated camera to convert the target's pixel coordinates in an image into world or geographic coordinates, thereby determining the target's position in real space. It typically combines geometric projection models, coordinate transformations, and optimization algorithms, and is one of the core technologies of computer vision in fields such as measurement, navigation, and V2X (vehicle-to-everything).

[0003] Currently, roadside camera calibration methods generally suffer from the following shortcomings: 1. Insufficient modeling accuracy in complex roadside environments: Existing methods are mostly based on a single plane or simple geometric model for camera calibration and positioning, making it difficult to accurately represent complex roadside scenes including slope changes, curved road surfaces, and uneven terrain. This results in large model fitting errors, affecting positioning accuracy. 2. Using a uniform model leads to uneven positioning accuracy for targets at different distances: In the field of view of roadside cameras, the imaging ratios of near, medium, and far-distance targets differ significantly. Traditional calibration methods usually use a uniform model for processing, which cannot perform differentiated optimization for different distance segments. Especially at long distances, perspective distortion and pixel quantization effects can significantly amplify target positioning errors. 3. Insufficient robustness of extrinsic parameter solutions: In actual data acquisition, calibration data is easily affected by noise, occlusion, and outliers. Traditional PnP-based extrinsic parameter solution methods have poor stability when there is outlier data and lack a robust estimation and global optimization extrinsic parameter calculation mechanism, causing calibration results to easily deviate from the true values.

[0004] In summary, the insufficient accuracy of traditional camera calibration, which leads to insufficient target positioning accuracy, is a problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a roadside camera calibration method, a target positioning method, and related equipment to improve camera calibration accuracy and thereby improve target positioning accuracy.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] The first aspect of this invention discloses a roadside camera calibration method, the method comprising:

[0008] Acquire the pre-set intrinsic parameters and calibration data of the roadside camera;

[0009] The calibration data is preprocessed to obtain the preprocessed calibration data;

[0010] Using the preset RANSAC_PnP algorithm and Bayesian optimization method, the extrinsic parameters of the preprocessed calibration data are solved to obtain the extrinsic parameters of the roadside camera;

[0011] Based on the preprocessed calibration data, a second-order surface fitting is performed to obtain a global coarse plane model and multiple local fine plane models;

[0012] The intrinsic parameters, extrinsic parameters, global coarse plane model, and each of the local fine plane models are determined as the calibration results of the roadside camera.

[0013] Optionally, the calibration data is preprocessed, and the preprocessed calibration data includes:

[0014] The calibration data is subjected to deduplication, distortion removal, and coordinate system transformation to obtain the preprocessed calibration data.

[0015] Optionally, after preprocessing the calibration data to obtain preprocessed calibration data, the method further includes:

[0016] Determine whether the preprocessed calibration data has distortion problems and / or coordinate inconsistency problems;

[0017] If so, return to the step of preprocessing the calibration data to obtain the preprocessed calibration data.

[0018] Optionally, the step of using a preset RANSAC_PnP algorithm and Bayesian optimization method to solve for the extrinsic parameters of the preprocessed calibration data to obtain the extrinsic parameters of the roadside camera includes:

[0019] The optimal hyperparameters of the preset RANSAC_PnP algorithm are determined using Bayesian optimization methods.

[0020] Using the optimal hyperparameters and the RANSAC_PnP algorithm, the extrinsic parameters of the calibration data are solved to obtain the extrinsic parameters of the roadside camera.

[0021] Optionally, the second-order surface fitting based on the preprocessed calibration data yields a global coarse plane model and multiple local fine plane models, including:

[0022] Based on the preprocessed calibration data, a second-order surface fitting is performed to obtain a global coarse plane model;

[0023] Based on the horizontal distance from each calibration point to the roadside camera in the preprocessed calibration data, the preprocessed calibration data is divided into multiple sub-intervals;

[0024] Based on each of the sub-intervals, a second-order surface fitting is performed to obtain multiple local fine-plane models.

[0025] A second aspect of this invention discloses a target localization method, the method comprising:

[0026] The calibration results of the roadside camera are obtained, and the pixel coordinates of the target to be located are obtained through the roadside camera; the calibration results are obtained based on any roadside camera calibration method disclosed in the first aspect of the present invention, and the calibration results include: the intrinsic parameters of the roadside camera, the extrinsic parameters of the roadside camera, the global coarse plane model and each local fine plane model;

[0027] A spatial ray model is constructed based on the pixel coordinates, the intrinsic parameters, and the extrinsic parameters.

[0028] Based on the spatial ray model and the global coarse plane model, the initial UTM estimation point of the target to be located is obtained.

[0029] Calculate the distance between the roadside camera and the initial UTM estimation point to obtain the target horizontal distance;

[0030] From each of the aforementioned local fine-faceted models, determine the target local fine-faceted model corresponding to the target horizontal distance;

[0031] Based on the spatial ray model and the target local fine plane model, the target UTM estimation point of the target to be located is obtained;

[0032] Based on the target UTM estimated point, the positioning result of the target to be located is obtained.

[0033] A third aspect of this invention discloses a roadside camera calibration device, the device comprising:

[0034] The first acquisition unit is used to acquire the pre-set intrinsic parameters and calibration data of the roadside camera;

[0035] The preprocessing unit is used to preprocess the calibration data to obtain the preprocessed calibration data;

[0036] The extrinsic parameter solving unit is used to solve the extrinsic parameters of the preprocessed calibration data using the preset RANSAC_PnP algorithm and Bayesian optimization method to obtain the extrinsic parameters of the roadside camera.

[0037] The fitting unit is used to perform second-order surface fitting based on the preprocessed calibration data to obtain a global coarse plane model and multiple local fine plane models.

[0038] The determining unit is used to determine the intrinsic parameters, the extrinsic parameters, the global coarse plane model, and each of the local fine plane models as the calibration results of the roadside camera.

[0039] A fourth aspect of the present invention discloses a target positioning device, the device comprising:

[0040] The second acquisition unit is used to acquire the calibration results of the roadside camera and acquire the pixel coordinates of the target to be located through the roadside camera; the calibration results are obtained based on a roadside camera calibration device disclosed in the third aspect of the present invention, and the calibration results include: the intrinsic parameters of the roadside camera, the extrinsic parameters of the roadside camera, the global coarse plane model and each local fine plane model;

[0041] A construction unit is used to construct a spatial ray model based on the pixel coordinates, the intrinsic parameters, and the extrinsic parameters;

[0042] The coarse localization unit is used to solve for the initial UTM estimation point of the target to be localized based on the spatial ray model and the global coarse plane model; and to calculate the distance between the roadside camera and the initial UTM estimation point to obtain the horizontal distance of the target.

[0043] A fine localization unit is used to determine the target local fine plane model corresponding to the horizontal distance of the target from each of the local fine plane models; and to obtain the target UTM estimation point of the target to be located based on the spatial ray model and the target local fine plane model.

[0044] The conversion unit is used to convert the target UTM estimated point to obtain the positioning result of the target to be located.

[0045] A fifth aspect of this invention discloses an electronic device, comprising:

[0046] Memory, used to store computer programs;

[0047] A processor is configured to execute the computer program to implement any of the roadside camera calibration methods disclosed in the first aspect of the present invention, or a target localization method disclosed in the second aspect of the present invention.

[0048] A sixth aspect of the present invention discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements any of the roadside camera calibration methods disclosed in the first aspect of the present invention, or a target localization method disclosed in the second aspect of the present invention.

[0049] Based on the roadside camera calibration method, target localization method, and related equipment provided in the above embodiments of the present invention, the following steps are taken: Pre-set intrinsic parameters and calibration data of the roadside camera are obtained; the calibration data is pre-processed to obtain pre-processed calibration data; the extrinsic parameters of the roadside camera are obtained by solving the extrinsic parameters of the pre-processed calibration data using a preset RANSAC_PnP algorithm and a Bayesian optimization method; second-order surface fitting is performed based on the pre-processed calibration data to obtain a global coarse plane model and multiple local fine plane models; the intrinsic parameters, extrinsic parameters, global coarse plane model, and each of the local fine plane models are determined as the calibration result of the roadside camera. In this scheme, the RANSAC algorithm is introduced into the traditional PnP solution framework to improve robustness to noise and outliers. Then, Bayesian optimization method is combined to solve the extrinsic parameters, thereby obtaining more stable and accurate camera extrinsic parameters. Furthermore, a second-order surface model is used to represent uneven road surfaces, and a hierarchical modeling strategy of global coarse plane and local fine plane is proposed, which effectively reduces the geometric mismatch error of a single plane model in complex scenes, thereby improving the accuracy of camera calibration and target localization. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0051] Figure 1 This is a flowchart of a roadside camera calibration method disclosed in an embodiment of the present invention;

[0052] Figure 2 This is an example diagram of calibration data disclosed in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of a global coarse plane and a local fine plane disclosed in an embodiment of the present invention;

[0054] Figure 4 This is a flowchart of a target localization method disclosed in an embodiment of the present invention;

[0055] Figure 5 This is an architecture diagram of a roadside camera calibration and target localization system disclosed in an embodiment of the present invention;

[0056] Figure 6 This is a structural diagram of a roadside camera calibration device disclosed in an embodiment of the present invention;

[0057] Figure 7 This is a structural diagram of a target positioning device disclosed in an embodiment of the present invention;

[0058] Figure 8 This is a structural diagram of an electronic device disclosed in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0061] As can be seen from the background technology, current roadside camera calibration methods generally suffer from the following shortcomings: 1. Insufficient modeling accuracy in complex roadside environments: Existing methods are mostly based on a single plane or simple geometric model for camera calibration and positioning, which makes it difficult to accurately represent complex roadside scenes including slope changes, curved road surfaces, and uneven terrain, resulting in large model fitting errors and affecting positioning accuracy; 2. Using a uniform model for processing leads to uneven positioning accuracy for targets at different distances: In the field of view of roadside cameras, the imaging ratio of near, medium, and far-distance targets differs significantly. Traditional calibration methods usually use a uniform model for processing, which cannot perform differentiated optimization for different distance segments. Especially at long distances, perspective distortion and pixel quantization effects will significantly amplify the target positioning error; 3. Insufficient robustness of extrinsic parameter solution: In actual acquisition, calibration data is easily affected by noise, occlusion, and outliers. Traditional PnP-based extrinsic parameter solution methods have poor stability when there is abnormal data and lack a robust estimation and global optimization combined extrinsic parameter calculation mechanism, which makes the calibration results easily deviate from the true value.

[0062] In other words, although existing combined intrinsic and extrinsic parameter visual calibration techniques can achieve target localization capabilities for roadside cameras, they rely on a single plane or simple geometric model, making it difficult to accurately adapt to complex environments such as slope changes and curved road surfaces. In practical applications, the localization accuracy varies greatly for targets at different distances, with significant errors at long distances. Furthermore, the extrinsic parameter solution of this type of method is sensitive to noise and outliers and lacks robust optimization.

[0063] In summary, the insufficient accuracy of traditional camera calibration, which leads to insufficient target positioning accuracy, is a problem that urgently needs to be solved.

[0064] Therefore, this invention discloses a roadside camera calibration method, a target localization method, and related equipment. In this solution, the RANSAC algorithm is introduced into the traditional PnP solution framework to improve robustness to noise and outliers. Then, the extrinsic parameters are solved by combining the Bayesian optimization method to obtain more stable and accurate camera extrinsic parameters. Furthermore, a second-order surface model is used to represent the uneven road surface, and a hierarchical modeling strategy of global coarse plane and local fine plane is proposed to effectively reduce the geometric mismatch error of a single plane model in complex scenes, thereby achieving the goal of improving the accuracy of camera calibration and target localization.

[0065] like Figure 1 The diagram shows a flowchart of a roadside camera calibration method disclosed in an embodiment of the present invention, which includes the following steps:

[0066] Step S101: Obtain the pre-set intrinsic parameters and calibration data of the roadside camera.

[0067] In step S101, the intrinsic parameters (focal length, principal point, distortion coefficient, and other fixed parameters) of the roadside camera that have been calibrated offline are obtained, and the calibration data measured on site are imported simultaneously as the sole input for subsequent extrinsic parameter solving and surface modeling.

[0068] The calibration data consists of pixel-to-world coordinate point pairs.

[0069] like Figure 2 The diagram shown is an example of calibration data disclosed in an embodiment of the present invention.

[0070] For example, Figure 2 All calibration data obtained from field measurements are displayed in a three-dimensional scatter plot. The calibration data is... Figure 2 The 3D point pairs of 51 pixel coordinates and their corresponding UTM (Universal Transverse Mercator) world coordinates.

[0071] Step S102: Preprocess the calibration data to obtain preprocessed calibration data.

[0072] In step S102, since the original calibration data usually has problems such as distortion and coordinate inconsistency, this embodiment of the invention preprocesses the calibration data at the beginning of the calibration stage, including deduplication, distortion removal and unified coordinate system transformation, in order to ensure the accuracy of subsequent calculations.

[0073] Specifically, the calibration data is subjected to deduplication, distortion removal, and coordinate system transformation to obtain preprocessed calibration data.

[0074] Optionally, after preprocessing the calibration data to obtain the preprocessed calibration data, it is determined whether the preprocessed calibration data has distortion problems and / or coordinate inconsistency problems; if so, the process returns to step S102.

[0075] In this embodiment of the invention, the original calibration data collected on site is first cleaned: duplicate points are removed, the pixel coordinates are distorted using camera intrinsic parameters, and then all world coordinates are converted to UTM world coordinates to ensure that they are in the same coordinate system. If the quality inspection finds that there are still residual distortions and / or coordinate jumps, the process is automatically returned to step S102 for reprocessing until the preprocessed calibration data is problem-free before proceeding to the next step.

[0076] Step S103: Using the preset RANSAC_PnP algorithm and Bayesian optimization method, solve the extrinsic parameters of the preprocessed calibration data to obtain the extrinsic parameters of the roadside camera.

[0077] In the specific implementation of step S103, the optimal hyperparameters of the preset RANSAC_PnP algorithm are determined using the Bayesian optimization method; the extrinsic parameters of the calibration data are solved using the optimal hyperparameters and the RANSAC_PnP algorithm to obtain the extrinsic parameters of the roadside camera.

[0078] It should be noted that the RANSAC_PnP algorithm is a robust attitude estimation method that combines the Random Sample Consensus (RANSAC) and the perspective n-point (PnP) problem.

[0079] In the traditional PnP problem, the goal of camera extrinsic parameter calculation is to estimate the rotation matrix by minimizing the reprojection error. Translation vector Given a set of 3D points in the world coordinate system and its corresponding two-dimensional pixel coordinates in the image. The optimization problem can be expressed as:

[0080]

[0081] in, For the projection function of the roadside camera, This is the intrinsic parameter matrix of the roadside camera.

[0082] However, when noise or outliers exist in the observed data, directly optimizing the objective function can easily lead to unstable extrinsic parameter estimation. Therefore, this embodiment of the invention employs the RANSAC_PnP algorithm, which constructs a consensus set through random sampling to estimate the extrinsic parameters, and counts the number of interior points in each iteration, ultimately selecting the solution that maximizes the interior point set.

[0083]

[0084] in, This is used to determine the distance threshold for interior points.

[0085] The performance of the RANSAC algorithm largely depends on the settings of two key hyperparameters: the interior point decision threshold and the interior point decision threshold. With the number of iterations These two parameters are extremely sensitive to the results of external parameter estimation. If If the value is too small, some valid calibration points may be misclassified as outliers, resulting in an insufficient set of points for calculation and unstable fitting; if If the value of M is too large, noise points may be mistakenly included in the inner point set, causing extrinsic parameter shift. Simultaneously, if M is too small, the algorithm may converge to a local optimum prematurely; if M is too large, it will significantly increase computational cost and reduce efficiency. Traditional methods typically rely on manual experience to tune parameters or grid search to determine parameters. However, manual tuning is highly subjective and difficult to maintain consistent performance under different noise scenarios; while grid search is systematic, it is computationally expensive and prone to getting trapped in local optima. Therefore, this embodiment of the invention further introduces a Bayesian optimization method to adaptively adjust the aforementioned hyperparameters. The optimization objective function is defined as:

[0086]

[0087] Bayesian optimization seeks the hyperparameter combination that minimizes the average inlier reprojection error (i.e., maximizes the negative error) under the current parameters. It constructs a probabilistic model (usually a Gaussian process) between the hyperparameters and the objective function, and uses the acquisition function to guide the search in the parameter space.

[0088]

[0089] Ultimately, RANSAC_PnP achieves the optimal hyperparameters determined by Bayesian optimization. The implementation of this method can significantly improve the stability and computational efficiency of extrinsic parameter estimation while maintaining robustness to outliers. In this embodiment of the invention, RANSAC is introduced into the traditional PnP solution framework to improve robustness to noise and outliers, and key hyperparameters (such as the number of iterations and the interior point threshold) are adaptively adjusted through Bayesian optimization to avoid instability caused by manual settings, thereby obtaining more stable and accurate camera extrinsic parameters.

[0090] Step S104: Perform second-order surface fitting based on the preprocessed calibration data to obtain a global coarse plane model and multiple local fine plane models.

[0091] In step S104, a method for layered scene modeling based on a second-order surface model is adopted. Specifically, a second-order surface is used to represent an uneven road surface with variations in slope and curvature. The general expression of the second-order surface model is:

[0092]

[0093] in, The coordinates of the ground point in the world coordinate system. to These are the surface coefficients obtained through least squares fitting. Compared to the traditional single-plane model... The second-order surface model can not only describe the overall slope, but also effectively capture local curvature changes and undulation features.

[0094] In the specific implementation of step S104, a second-order surface fitting is performed based on the preprocessed calibration data to obtain a global coarse plane model (i.e., a global second-order surface model); the preprocessed calibration data is divided into multiple sub-intervals according to the horizontal distance from each calibration point to the roadside camera in the preprocessed calibration data; a second-order surface fitting is performed based on each sub-interval to obtain multiple local fine plane models (i.e., local second-order surface models).

[0095] It should be noted that a global coarse plane model is obtained by performing second-order surface fitting based on the preprocessed calibration data. That is, a global coarse plane model is obtained by performing second-order surface fitting on all calibration data based on the second-order surface model represented by formula (5). Similarly, multiple local fine plane models are obtained by performing second-order surface fitting on each sub-interval. That is, a local fine plane model is obtained by performing second-order surface fitting on each sub-interval based on the second-order surface model represented by formula (5).

[0096] In this embodiment of the invention, in order to achieve more refined scene modeling, a layered scene modeling strategy is adopted: firstly, a global second-order curved surface model describing the overall road surface morphology is fitted based on all calibration data, namely a global coarse plane model.

[0097] Subsequently, based on the horizontal distance between the camera and the target point, the global data was divided into three sub-intervals: near band, mid band, and far band.

[0098] It should be noted that the number of sub-intervals is not fixed at three and can be flexibly adjusted according to scene complexity, camera mounting height, field of view, and the amount of calibration point data. For example, in complex intersection scenes with significant slope changes or large road surface curvature, the number of sub-intervals can be increased to improve fitting accuracy; while in scenes with relatively flat terrain or high calibration point density, the three-partition scheme can achieve a good balance between accuracy and computational efficiency. Therefore, the three-partition scheme is a preferred embodiment of the present invention, but not the only one. Subsequently, second-order surface fitting is performed within each sub-interval to form a local fine-plane model.

[0099] Please see Figure 3 This is a schematic diagram of a global coarse plane and a local fine plane disclosed in an embodiment of the present invention. Figure 3 The global coarse plane and the local fine plane shown are based on Figure 2 The calibration data shown is an example obtained after layered scene modeling.

[0100] In this embodiment of the invention, a hierarchical scene modeling approach is used, with a global surface for overall trend modeling and a local surface for high-precision fitting of different zones in the near, middle, and far distances. This approach can not only grasp the overall trend of the roadside scene at a macro level, but also achieve high-precision fitting at different distances. This effectively overcomes the systematic errors introduced by geometric simplification in complex roadside scenes by traditional single-plane models, thereby significantly improving the adaptability and overall accuracy of the calibration model to complex road conditions.

[0101] Step S105: Determine the intrinsic parameters, extrinsic parameters, global coarse plane model, and each local fine plane model as the calibration results of the roadside camera.

[0102] It should be noted that after obtaining the calibration results, the subsequent target localization stage can directly call the calibration results to quickly map any pixel coordinates to high-precision world coordinates without recalculation.

[0103] Based on the roadside camera calibration method disclosed in the above embodiments of the present invention, this method first preprocesses the data using camera intrinsic parameters and calibration data, then calculates the optimal camera extrinsic parameters using RANSAC_PnP and Bayesian optimization, and completes global second-order surface fitting. To improve accuracy, the data is divided into near-band, mid-band, and far-band according to camera distance, and local second-order surface fitting is performed for each. Finally, the camera intrinsic and extrinsic parameters and the equation parameters of the global coarse plane and the local second-order plane are saved. In this scheme, the RANSAC algorithm is introduced into the traditional PnP solution framework to improve robustness to noise and outliers. Then, the extrinsic parameters are solved using Bayesian optimization, thereby obtaining more stable and accurate camera extrinsic parameters. Furthermore, a second-order surface model is used to represent the uneven road surface, and a layered modeling strategy of global coarse plane and local fine plane is proposed, which effectively reduces the geometric mismatch error of a single plane model in complex scenes, thereby achieving the goal of improving the accuracy of camera calibration and target localization.

[0104] It should be noted that existing target localization methods have a single localization strategy, making it difficult to balance accuracy and efficiency: existing technologies mostly adopt a single-stage localization process, directly using a single fitted plane or surface to solve for coordinates, lacking a hierarchical strategy that combines coarse and fine localization, making it difficult to ensure both computational efficiency and localization accuracy in a wide range of scenarios.

[0105] Therefore, in order to improve the computational efficiency and positioning accuracy of target localization, based on the roadside camera calibration method disclosed in the above embodiments of the present invention, the present invention discloses a target localization method. In this solution, a hierarchical strategy of coarse localization and fine localization is adopted. First, the initial result is obtained quickly through the global surface, and then the local surface is used for fine correction. While ensuring computational efficiency, the final positioning accuracy is significantly improved.

[0106] like Figure 4 The diagram shown is a flowchart of a target localization method disclosed in an embodiment of the present invention. The method includes the following steps:

[0107] Step S201: Obtain the calibration results of the roadside camera and obtain the pixel coordinates of the target to be located through the roadside camera.

[0108] The calibration results are obtained based on any of the roadside camera calibration methods disclosed in the above embodiments of the present invention. The calibration results include: the intrinsic parameters of the roadside camera, the extrinsic parameters of the roadside camera, the global coarse plane model, and each local fine plane model.

[0109] Step S202: Construct a spatial ray model based on pixel coordinates, intrinsic parameters, and extrinsic parameters.

[0110] In step S202, the pixel coordinates of the target in the image are used. As input, a spatial ray model is constructed by combining the camera intrinsic and extrinsic parameters obtained from calibration. Specifically, firstly, the camera intrinsic parameter matrix is ​​used... Normalize the pixel coordinates:

[0111]

[0112] in, For normalized pixel coordinates, further combined with camera extrinsic parameters (rotation matrix) With translation vector Establish a spatial ray model originating from the camera's optical center:

[0113]

[0114] in, This represents a point in space in the world coordinate system where the ray is located. is the scale factor.

[0115] Step S203: Based on the spatial ray model and the global coarse plane model, the initial UTM estimation point of the target to be located is obtained, and the distance between the roadside camera and the initial UTM estimation point is calculated to obtain the horizontal distance of the target.

[0116] Step S203 is the global coarse localization process, and its specific implementation is as follows:

[0117] To obtain the initial position of the target in the world coordinate system, this embodiment of the invention intersects the aforementioned spatial ray model with the global coarse plane model established during the calibration phase, and then uses the ray coordinates... Substituting into the global coarse plane model, the intersection points can be solved. This point is the projection position of the target to be located on the global coarse plane, and is used as the initial UTM coordinate estimate, i.e., the initial UTM estimate point.

[0118] Based on this, to further support subsequent fine-tuning processes, this embodiment of the invention also calculates the target horizontal distance between the projection point and the camera optical center:

[0119]

[0120] in This represents the camera's horizontal coordinates in the world coordinate system. Distance This will serve as the criterion for determining the local fine-plane model, used to select the corresponding target local fine-plane model during the fine localization stage. Through the above coarse localization process, the preliminary spatial coordinates of the target under the global model can be quickly obtained, providing a basis for discrimination and input conditions for subsequent refined localization by region.

[0121] Step S204: From each local fine plane model, determine the target local fine plane model corresponding to the target horizontal distance, and based on the spatial ray model and the target local fine plane model, solve for the target UTM estimation point of the target to be located.

[0122] Step S204 is the local fine-tuning process, and its specific implementation is as follows:

[0123] It should be noted that the coarse localization results obtained under the global surface are... Then, the fine positioning process begins. Since the global second-order surface primarily reflects the overall terrain trend, in real-world scenarios with local slope variations or curvature undulations, the coarse positioning result may still deviate from the target's true position. Therefore, this embodiment of the invention uses the target horizontal distance calculated during the global coarse positioning stage. The target is divided into corresponding distance intervals (near band, middle band, or far band), and the local fine plane model of the target obtained by calibrating the interval is called.

[0124] In the fine-line localization process, this embodiment of the invention performs a secondary intersection between the spatial ray model formed by pixel coordinate back projection and the selected target local fine-line plane model to obtain the corrected three-dimensional coordinates. This is the fine-grained localization result, also known as the target UTM estimation point. Compared to the coarse localization result, this intersection point can more accurately reflect the true projection position of the target under local terrain conditions, effectively suppressing systematic biases caused by the smoothness of the global model.

[0125] In this embodiment of the invention, the initial UTM coordinates of the target are first obtained quickly using a global surface, and then the corresponding local surface is selected for fine correction based on the horizontal distance between the camera and the target, forming a two-level positioning process of coarse positioning + fine positioning. This significantly reduces the impact of perspective distortion and local curvature on positioning accuracy in long-distance imaging, so that the target can maintain stable and high-precision positioning results in different distance ranges.

[0126] Step S205: Based on the target UTM estimated points, convert to obtain the positioning result of the target to be located.

[0127] In step S205, the target UTM estimated point is converted into WGS84 coordinates to obtain the positioning result of the target to be located.

[0128] Among them, the fine positioning results That is, the target UTM estimated point is converted into WGS84 geographic coordinates to output high-precision positioning values.

[0129] Based on the target localization method disclosed in the above embodiments of the present invention, the pixel coordinates of the target to be localized are input, and the mapping relationship from the camera to the world is obtained through ray construction. In the coarse localization stage, the initial UTM estimation point is obtained by finding the intersection point with the global coarse plane, and the horizontal distance to the camera is calculated. In the fine localization stage, a fine plane is selected based on the camera distance threshold, and the final UTM coordinates are obtained by finding the intersection point with the local fine plane. Finally, the UTM coordinates are converted into WGS84 coordinates and output. In this scheme, it is proposed to first use the global surface to quickly obtain the initial UTM coordinates of the target, and then select the corresponding local surface for fine correction according to the horizontal distance between the camera and the target, forming a two-stage localization process of coarse localization + fine localization. This significantly reduces the impact of perspective distortion and local curvature in long-distance imaging on the localization accuracy, so that the target can maintain stable and high-precision localization results within different distance ranges.

[0130] To verify the effectiveness of the roadside camera calibration method and target localization method disclosed in the above embodiments of the present invention, an actual intersection (Intersection A) was selected as the experimental scenario. The performance of the traditional method and the roadside camera calibration method and target localization method proposed in this invention were compared under the same data conditions. The data used in the experiment consisted of intersection point collection data and intersection vehicle test data. The point collection data included 49 reference points, and the vehicle test data included 1319 data points.

[0131] The two methods for experimental comparison are:

[0132] Method 1: The traditional RANSAC_PnP method based on the ground plane equation without Bayesian optimization.

[0133] Method 2: A roadside camera calibration method and target localization method proposed in this invention.

[0134] Data Scenarios method Number of interior points selected for the matrix Positioning error Reprojection error Data at intersection A Method 1 16 0.510m 12.2px Data at intersection A Method 2 6 0.296m 8.55px

[0135] As shown in Table 1, the method of this invention significantly outperforms traditional methods in terms of positioning accuracy and reprojection error. Method 1 uses 16 interior points, while Method 2 uses only 6. Although the number is reduced, the matching quality between these interior points and the real geometric relationship is higher, thus ensuring the stability and accuracy of extrinsic parameter calculation and projection modeling. The final average positioning error decreased from 0.510m to 0.296m, a reduction of approximately 42%; the reprojection error decreased from 12.2px to 8.55px, a reduction of approximately 30%. This fully demonstrates that the method of this invention, by introducing Bayesian optimization to adaptively adjust the hyperparameters of RANSAC_PnP, makes the interior point selection more rigorous, avoiding the incorrect inclusion of points with large deviations in the consistency set. Simultaneously, multi-partition surface modeling effectively improves the accuracy of road surface geometry fitting, ensuring the high consistency between the projected points and the real ground, thereby significantly reducing the overall positioning error.

[0136] The experimental results of collecting data from the sports car are shown in Table 2:

[0137] Data Scenarios method Positioning error Reprojection error Traffic data at intersection A Method 1 1.188m 20.76px Traffic data at intersection A Method 2 0.780m 15.25px

[0138] As shown in Table 2, Method 2 also outperforms the traditional method on dynamic vehicle data. In the vehicle test at intersection A, the average positioning error of Method 1 was 1.188m, while the method of this invention reduced it to 0.780m, a decrease of approximately 34%. Simultaneously, the reprojection error decreased from 20.76px to 15.25px, a reduction of over 26%. These results demonstrate that Method 2 significantly improves positioning accuracy and projection consistency in both static and dynamic scenarios, exhibiting higher robustness and engineering application value.

[0139] like Figure 5 The diagram shown is an architecture diagram of a roadside camera calibration and target positioning system disclosed in an embodiment of the present invention. The system includes a calibration module 100 and a positioning module 200.

[0140] The calibration module 100 is used to execute any of the roadside camera calibration methods disclosed in the above embodiments of the present invention.

[0141] The positioning module 200 is used to execute any of the target positioning methods disclosed in the above embodiments of the present invention.

[0142] Corresponding to the roadside camera calibration method disclosed in the above embodiments of the present invention, such as Figure 6 The diagram shown is a structural diagram of a roadside camera calibration device disclosed in an embodiment of the present invention. The device includes:

[0143] The first acquisition unit 601 is used to acquire the pre-set intrinsic parameters and calibration data of the roadside camera;

[0144] Preprocessing unit 602 is used to preprocess the calibration data to obtain preprocessed calibration data;

[0145] The extrinsic parameter solving unit 603 is used to solve the extrinsic parameters of the preprocessed calibration data using the preset RANSAC_PnP algorithm and Bayesian optimization method to obtain the extrinsic parameters of the roadside camera.

[0146] The fitting unit 604 is used to perform second-order surface fitting based on the preprocessed calibration data to obtain a global coarse plane model and multiple local fine plane models.

[0147] The determining unit 605 is used to determine the intrinsic parameters, the extrinsic parameters, the global coarse plane model, and each of the local fine plane models as the calibration results of the roadside camera.

[0148] Optionally, the preprocessing unit 602 is specifically used for:

[0149] The calibration data is subjected to deduplication, distortion removal, and coordinate system transformation to obtain the preprocessed calibration data.

[0150] Optionally, the device may also include:

[0151] The judgment unit is used to determine whether the preprocessed calibration data has distortion problems and / or coordinate inconsistency problems after preprocessing the calibration data to obtain the preprocessed calibration data;

[0152] If so, return to the step of preprocessing the calibration data to obtain the preprocessed calibration data.

[0153] Optional, extrinsic parameter solver 603, specifically used for:

[0154] The optimal hyperparameters of the preset RANSAC_PnP algorithm are determined using Bayesian optimization methods.

[0155] Using the optimal hyperparameters and the RANSAC_PnP algorithm, the extrinsic parameters of the calibration data are solved to obtain the extrinsic parameters of the roadside camera.

[0156] Optionally, the fitting unit 604 is specifically used for:

[0157] Based on the preprocessed calibration data, a second-order surface fitting is performed to obtain a global coarse plane model;

[0158] Based on the horizontal distance from each calibration point to the roadside camera in the preprocessed calibration data, the preprocessed calibration data is divided into multiple sub-intervals;

[0159] Based on each of the sub-intervals, a second-order surface fitting is performed to obtain multiple local fine-plane models.

[0160] Based on the roadside camera calibration device disclosed in the above embodiments of the present invention, the device first performs data preprocessing using camera intrinsic parameters and calibration data, and then calculates the optimal camera extrinsic parameters using RANSAC_PnP and Bayesian optimization, and completes global second-order surface fitting. To improve accuracy, the data is divided into near-band, mid-band, and far-band according to camera distance, and local second-order surface fitting is performed for each band. Finally, the camera intrinsic and extrinsic parameters and the equation parameters of the global coarse plane and the local second-order plane are saved. In this scheme, the RANSAC algorithm is introduced into the traditional PnP solution framework to improve robustness to noise and outliers. Then, the extrinsic parameters are solved using Bayesian optimization, thereby obtaining more stable and accurate camera extrinsic parameters. Furthermore, a second-order surface model is used to represent the uneven road surface, and a layered modeling strategy of global coarse plane and local fine plane is proposed, which effectively reduces the geometric mismatch error of a single plane model in complex scenes, thereby achieving the goal of improving the accuracy of camera calibration and target localization.

[0161] Corresponding to the target localization method disclosed in the above embodiments of the present invention, such as Figure 7 The diagram shown is a structural diagram of a target positioning device disclosed in an embodiment of the present invention. The device includes:

[0162] The second acquisition unit 701 is used to acquire the calibration results of the roadside camera and acquire the pixel coordinates of the target to be located through the roadside camera; the calibration results are obtained based on a roadside camera calibration device disclosed in the above embodiment of the present invention, and the calibration results include: the intrinsic parameters of the roadside camera, the extrinsic parameters of the roadside camera, the global coarse plane model and each local fine plane model;

[0163] Construction unit 702 is used to construct a spatial ray model based on pixel coordinates, intrinsic parameters, and extrinsic parameters;

[0164] The coarse localization unit 703 is used to solve for the initial UTM estimation point of the target to be localized based on the spatial ray model and the global coarse plane model; and to calculate the distance between the roadside camera and the initial UTM estimation point to obtain the horizontal distance of the target.

[0165] The fine localization unit 704 is used to determine the target local fine plane model corresponding to the horizontal distance of the target from each local fine plane model; and to obtain the target UTM estimation point of the target to be located based on the spatial ray model and the target local fine plane model.

[0166] The conversion unit 705 is used to convert the target UTM estimated points to obtain the positioning result of the target to be located.

[0167] Based on the target localization device disclosed in the above embodiments of the present invention, the pixel coordinates of the target to be localized are input, and the mapping relationship from the camera to the world is obtained through ray construction. In the coarse localization stage, the intersection point with the global coarse plane is obtained to obtain the initial UTM estimation point and the horizontal distance to the camera is calculated. In the fine localization stage, a fine plane is selected based on the camera distance threshold, and the intersection point with the local fine plane is obtained to obtain the final UTM coordinates. Finally, the UTM coordinates are converted into WGS84 coordinates and output. In this scheme, it is proposed to first use the global surface to quickly obtain the initial UTM coordinates of the target, and then select the corresponding local surface for fine correction according to the horizontal distance between the camera and the target, forming a two-stage localization process of coarse localization + fine localization. This significantly reduces the impact of perspective distortion and local curvature in long-distance imaging on the localization accuracy, so that the target can maintain a stable and high-precision localization result in different distance ranges.

[0168] This invention also provides an electronic device, please refer to... Figure 8 The electronic device includes a memory 801 and a processor 802.

[0169] Among them, memory 801 is used to store computer programs;

[0170] The processor 802 is used to execute computer programs, specifically to implement a roadside camera calibration method or a target localization method provided in any embodiment of this application.

[0171] This application also provides a computer storage medium for storing a computer program, which, when executed, is specifically used to implement a roadside camera calibration method or a target positioning method provided in any embodiment of this application.

[0172] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0173] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0174] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for calibrating a roadside camera, characterized in that, The method includes: Acquire the pre-set intrinsic parameters and calibration data of the roadside camera; The calibration data is preprocessed to obtain the preprocessed calibration data; Using the preset RANSAC_PnP algorithm and Bayesian optimization method, the extrinsic parameters of the preprocessed calibration data are solved to obtain the extrinsic parameters of the roadside camera; Based on the preprocessed calibration data, a second-order surface fitting is performed to obtain a global coarse plane model and multiple local fine plane models. The intrinsic parameters, extrinsic parameters, global coarse plane model, and each of the local fine plane models are determined as the calibration results of the roadside camera.

2. The method according to claim 1, characterized in that, The calibration data is preprocessed, and the preprocessed calibration data includes: The calibration data is subjected to deduplication, distortion removal, and coordinate system transformation to obtain the preprocessed calibration data.

3. The method according to claim 1, characterized in that, After preprocessing the calibration data to obtain preprocessed calibration data, the method further includes: Determine whether the preprocessed calibration data has distortion problems and / or coordinate inconsistency problems; If so, return to the step of preprocessing the calibration data to obtain the preprocessed calibration data.

4. The method according to claim 1, characterized in that, The method utilizes a preset RANSAC_PnP algorithm and Bayesian optimization to solve for the extrinsic parameters of the preprocessed calibration data, thereby obtaining the extrinsic parameters of the roadside camera, including: The optimal hyperparameters of the preset RANSAC_PnP algorithm are determined using Bayesian optimization methods. Using the optimal hyperparameters and the RANSAC_PnP algorithm, the extrinsic parameters of the calibration data are solved to obtain the extrinsic parameters of the roadside camera.

5. The method according to any one of claims 1 to 4, characterized in that, The second-order surface fitting based on the preprocessed calibration data yields a global coarse plane model and multiple local fine plane models, including: Based on the preprocessed calibration data, a second-order surface fitting is performed to obtain a global coarse plane model; Based on the horizontal distance from each calibration point to the roadside camera in the preprocessed calibration data, the preprocessed calibration data is divided into multiple sub-intervals; Based on the second-order surface fitting of each sub-interval, multiple local fine-plane models are obtained.

6. A target localization method, characterized in that, The method includes: The calibration results of the roadside camera are obtained, and the pixel coordinates of the target to be located are obtained through the roadside camera; the calibration results are obtained based on the roadside camera calibration method according to any one of claims 1 to 5, and the calibration results include: the intrinsic parameters of the roadside camera, the extrinsic parameters of the roadside camera, the global coarse plane model and each local fine plane model; A spatial ray model is constructed based on the pixel coordinates, the intrinsic parameters, and the extrinsic parameters. Based on the spatial ray model and the global coarse plane model, the initial UTM estimation point of the target to be located is obtained. The horizontal distance to the target is obtained by calculating the distance between the roadside camera and the initial UTM estimation point; From each of the aforementioned local fine-faceted models, determine the target local fine-faceted model corresponding to the target horizontal distance; Based on the spatial ray model and the target local fine plane model, the target UTM estimation point of the target to be located is obtained; Based on the target UTM estimated points, the positioning result of the target to be located is obtained.

7. A roadside camera calibration device, characterized in that, The device includes: The first acquisition unit is used to acquire the pre-set intrinsic parameters and calibration data of the roadside camera; The preprocessing unit is used to preprocess the calibration data to obtain the preprocessed calibration data; The extrinsic parameter solving unit is used to solve the extrinsic parameters of the preprocessed calibration data using the preset RANSAC_PnP algorithm and Bayesian optimization method to obtain the extrinsic parameters of the roadside camera. The fitting unit is used to perform second-order surface fitting based on the preprocessed calibration data to obtain a global coarse plane model and multiple local fine plane models. The determining unit is used to determine the intrinsic parameters, the extrinsic parameters, the global coarse plane model, and each of the local fine plane models as the calibration results of the roadside camera.

8. A target positioning device, characterized in that, The device includes: The second acquisition unit is used to acquire the calibration results of the roadside camera and acquire the pixel coordinates of the target to be located through the roadside camera; the calibration results are obtained based on the roadside camera calibration device of claim 7, and the calibration results include: the intrinsic parameters of the roadside camera, the extrinsic parameters of the roadside camera, the global coarse plane model and each local fine plane model; A construction unit is used to construct a spatial ray model based on the pixel coordinates, the intrinsic parameters, and the extrinsic parameters; The coarse localization unit is used to solve for the initial UTM estimation point of the target to be localized based on the spatial ray model and the global coarse plane model; and to calculate the distance between the roadside camera and the initial UTM estimation point to obtain the horizontal distance of the target. A fine localization unit is used to determine the target local fine plane model corresponding to the horizontal distance of the target from each of the local fine plane models; and to obtain the target UTM estimation point of the target to be located based on the spatial ray model and the target local fine plane model. The conversion unit is used to convert the target UTM estimated point to obtain the positioning result of the target to be located.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement a roadside camera calibration method as described in any one of claims 1 to 5, or a target localization method as described in claim 6.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement a roadside camera calibration method as described in any one of claims 1 to 5, or a target localization method as described in claim 6.

Images