A method for continuous calibration in traffic scenarios without field testing

Abstract

A method for continuous calibration of traffic scenes without field testing includes: acquiring traffic scene images; extracting geometric prior features from the images, whereby the geometric prior features characterize points on the distortion curve in the traffic scene images; defining an error function and optimizing distortion parameters based on the error function, whereby the distortion parameters characterize the geometric offset in the image caused by lens distortion, including radial distortion parameters and tangential distortion parameters; performing orthogonal processing on the acquired images using a nonlinear optimization algorithm based on the optimal distortion parameters; extracting coplanar ground feature points from the orthogonalized images and solving for camera extrinsic parameters, including rotation matrices and translation vectors, based on their corresponding world coordinate system positions; evaluating the calibration error distribution based on simulation or real-world scene data, dynamically adjusting distortion model parameters or optimization strategies, and triggering online parameter updates when pose deviations exceed a threshold. This method simplifies the calibration process and improves calibration efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for continuous calibration of traffic scenes without field calibration. Background Technology

[0002] With the continuous development of intelligent transportation systems and autonomous driving technologies, camera calibration has become a key technology in traffic monitoring and perception systems. Camera calibration is fundamental to ensuring accurate positioning and measurement of visual perception systems, especially in highways, urban roads, and autonomous driving environments, where accurate camera calibration is crucial for ensuring safety and improving traffic efficiency. However, traditional calibration methods typically rely on fieldwork, requiring the use of dedicated calibration boards or control points. These methods are not only time-consuming but also require deployment in real-world environments, limiting their application in large-scale, dynamic environments. Summary of the Invention

[0003] To address the problems existing in the prior art, embodiments of this application provide a method for continuous calibration in traffic scenarios without field testing, a computing device, a computer storage medium, and a product containing a computer program, which can simplify the calibration process and improve calibration efficiency.

[0004] In a first aspect, embodiments of this application provide a camera calibration method without field operations, comprising: acquiring traffic scene images; extracting geometric prior features from the images, wherein the geometric prior features are used to characterize points existing on the distortion curve in the traffic scene images; defining an error function; optimizing distortion parameters based on the error function, wherein the distortion parameters are used to characterize the geometric offset in the image caused by lens distortion, including radial distortion parameters and tangential distortion parameters; performing orthogonal processing on the acquired images using a nonlinear optimization algorithm based on the optimal distortion parameters; extracting ground coplanar feature points in the orthogonal image, and solving for camera extrinsic parameters, including rotation matrix and translation vector, by combining their corresponding world coordinate system positions and using the SolvePnP algorithm; evaluating the calibration error distribution based on simulation or actual scene data, dynamically adjusting distortion model parameters or optimization strategies, and triggering online parameter updates when a pose deviation is detected to exceed a threshold.

[0005] Among some possible implementations, the method also includes: using multi-frame image data fusion and Kalman filtering to suppress noise interference, ensuring the robustness and real-time performance of calibration results in complex environments, including but not limited to rainy / foggy environments and nighttime environments.

[0006] In some possible implementations, a millimeter-wave radar is also mounted on the camera. The method further includes: simultaneously acquiring camera images and millimeter-wave radar point cloud data; extracting feature corner points from the camera images; and identifying static target corner points corresponding to the feature corner points in the millimeter-wave radar point cloud data to generate multiple 2D-3D matching point pairs between image pixel coordinates and radar world coordinates; calculating the homography transformation matrix between the camera image plane and the radar world coordinate system through direct linear transformation based on the matching point pairs; given the camera intrinsic parameters, obtaining the rotation matrix and translation vector of the camera relative to the radar world coordinate system through matrix decomposition based on the homography transformation matrix and the camera intrinsic parameter matrix, as viewpoint correction parameters; and performing perspective transformation on the camera image based on the rotation matrix and translation vector to correct the viewpoint deviation caused by camera pose offset.

[0007] In some possible implementations, a millimeter-wave radar is also mounted on the camera. The method further includes: simultaneously acquiring camera images and millimeter-wave radar point cloud data; extracting feature corner points from the camera images and identifying corresponding static target corner points in the millimeter-wave radar point cloud data to generate multiple 2D-3D matching point pairs between image pixel coordinates and radar world coordinates; calculating the homography transformation matrix between the camera image plane and the radar world coordinate system using direct linear transformation based on the matching point pairs; since the camera's intrinsic parameters are uncertain, constructing a reprojection error function using the homography transformation matrix as the initial value, where the reprojection error function represents the Euclidean distance between the coordinates of the radar world coordinate points projected by the homography transformation matrix and the coordinates of the corresponding feature corner points in the image; iteratively optimizing the parameters of the homography transformation matrix using a nonlinear optimization algorithm to minimize the reprojection error function, and using the optimized and converged homography transformation matrix as the final viewpoint correction parameter; and performing an inverse perspective transformation on the camera image using the optimized homography transformation matrix to correct viewpoint deviations.

[0008] In some possible implementations, the geometric prior features include at least one of lane lines, curbs, and building edges.

[0009] In some possible implementations, dynamic adjustment includes: suppressing noise interference through multi-frame data fusion and Kalman filtering; and triggering recalculation of calibration parameters when the pitch angle error exceeds 0.3 degrees or the translation deviation exceeds a preset threshold.

[0010] Among some possible implementations, the nonlinear optimization algorithm is the Powell method, the least squares method, or the Levenberg-Marquardt algorithm.

[0011] In some possible implementations, the error function is the sum of the geometric distances from each point on the distorted curve to the ideal straight line, which is the line connecting the first and last points on the distorted curve.

[0012] In some possible implementations, the error function is calculated according to the following formula:

[0013]

[0014] In the formula, E represents the error function. It represents the geometric distance from a point on the distorted curve to the ideal straight line.

[0015] In some possible implementations, the geometric distance from a point on the distorted curve to the ideal straight line is calculated using the following formula:

[0016]

[0017] In the formula, The coordinates of points on the distorted curve. The coordinates of the first point on the distortion curve. The coordinates of the last point on the distortion curve.

[0018] Secondly, embodiments of this application provide a computer-readable storage medium including computer-readable instructions, which, when read and executed by a computer, cause the computer to perform the method as described in any of the first aspects.

[0019] Thirdly, embodiments of this application provide a computing device, including a processor and a memory, wherein the memory stores computer program instructions, which, when executed by the processor, perform the method as described in any of the first aspects.

[0020] Fourthly, embodiments of this application provide a product comprising a computer program, which, when the computer program product is run on a processor, causes the processor to perform the method as described in any of the first aspects. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a camera calibration method that eliminates the need for field operations, as provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0025] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0026] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0027] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0028] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0029] Currently, some known calibration methods primarily rely on manually set calibration boards or control points in the field environment. These calibration boards typically have known geometric shapes and features; for example, a checkerboard calibration board, with its black and white checkerboard pattern, provides a large number of corner features. The positions of these corner points in the world coordinate system are known, thus they can be used as control points for calibration. The calibration process mainly includes the following steps: First, the calibration board is manually set up in the field environment, ensuring the accuracy of its position and orientation; second, images of the calibration board are captured using the camera to be calibrated; then, feature pixels on the calibration board are extracted using image processing algorithms; finally, based on the image coordinates of these feature points and the known world coordinates, mathematical models and algorithms are used to calculate the camera's intrinsic and extrinsic parameters. Commonly used mathematical models include camera imaging models and distortion models, such as the Brown distortion model and the Kannala-Brandt distortion model. To solve for the camera's intrinsic and extrinsic parameters, optimization algorithms are usually needed to minimize the prediction error of image feature points, such as nonlinear least squares methods and the Levenberg-Marquardt algorithm.

[0030] However, these traditional calibration methods have many problems and drawbacks. First, the calibration process is cumbersome and inefficient, requiring professionals to carefully set up the calibration board on-site and ensure its position and orientation accuracy. This is not only time-consuming and labor-intensive, but also extremely demanding when deploying traffic monitoring cameras, roadside sensing units, and other equipment on a large scale. Second, existing calibration methods are highly dependent on external calibration objects. If the position or orientation of the calibration object changes, or if the calibration object cannot be accurately identified in practical applications, the accuracy of the calibration will be severely affected. Moreover, in some complex traffic scenarios or inclement weather conditions, the identification of the calibration object may be interfered with, leading to calibration failure or decreased calibration accuracy. For example, in strong light, shadow, rain, snow, or other weather conditions, the corner features of the calibration board may not be accurately captured and extracted, thus affecting the calibration results. In addition, for wide-angle lenses, the distortion range is large, and traditional calibration methods often struggle to fully cover and accurately correct distortion across the entire image range. This results in lower calibration accuracy in image edge areas, affecting the accurate perception of distant objects. Especially in the fields of intelligent transportation and autonomous driving, the demand for high-precision calibration of wide-angle lenses is increasingly urgent, but existing calibration methods have significant shortcomings in meeting this need. Finally, existing calibration methods are usually performed in static environments; once the environment changes, such as vehicle movement or road modifications, the calibration results may no longer be accurate. In real-world traffic scenarios, vehicles are constantly moving, and the road environment changes continuously, requiring calibration methods to adapt to these changes in real time. However, existing calibration methods, due to the need for manual placement of calibration objects and complex calculation processes, struggle to meet the requirements of real-time calibration.

[0031] In view of this, the embodiments of this application provide a camera calibration method that eliminates the need for field operations, which can simplify the calibration process and improve calibration efficiency.

[0032] For example, Figure 1 This illustration shows a flowchart of a camera calibration method without fieldwork provided in an embodiment of this application. Figure 1 As shown, the calibration method may include the following steps:

[0033] S11: Acquire traffic scene images and extract geometric prior features from the images. Geometric prior features are used to characterize points on distorted curves in the traffic scene images.

[0034] In this embodiment, traffic scene images are acquired, and geometric prior features are extracted from the images. Cameras need to be deployed in actual traffic scenes, such as urban roads, highways, or intersections, using wide-angle cameras or radar-visual fusion devices to capture images with a large coverage area. Since traffic scenes contain numerous natural geometric features, such as lane lines, curbs, and building edges, these features should ideally be straight lines, but due to radial and tangential distortion of the camera lens, they appear as curves in the image. After image acquisition, image processing algorithms extract point sets on these distorted curves. Commonly used methods include edge detection (such as the Canny algorithm), Hough transform (for detecting straight lines or curves), and deep learning-based semantic segmentation techniques (such as lane line recognition). For example, for lane lines, the algorithm first performs image grayscale conversion and Gaussian filtering for noise reduction, then finds the lane line contour through edge detection, and then samples discrete points along the contour to form a point set. These point sets are labeled as geometric prior features that "should have been straight lines but are curved due to distortion," and used as input for subsequent error calculation. In other words, geometric prior features refer to objects that should be straight lines in the real world, such as straight lane lines, neat curbs, or the vertical edges of buildings. These features, which should be straight lines, will appear as curves in images due to lens distortion. It is important to note that the accuracy of feature extraction directly affects the subsequent calibration results. Therefore, it is necessary to adjust the algorithm parameters according to the scene characteristics, such as enhancing contrast in environments with large lighting changes, or removing noise interference through morphological operations. The distorted curves are identified and extracted, and the extracted curves are discretized into a series of point coordinate sets {p}. i}, where each point p i Each has its corresponding pixel coordinates (u i , v i ).

[0035] S12: Define the error function and optimize the distortion parameters based on the error function. The distortion parameters are used to characterize the geometric offset in the image caused by lens distortion, including radial distortion parameters and tangential distortion parameters.

[0036] In this embodiment, an error function E is defined to quantify the degree of image distortion, used to quantify the deviation of the distorted curve from the ideal straight line. Specifically, for each identified distorted curve, the first and last points are selected, and these two points are used as reference points to determine an ideal straight line L. The distance from each point to this straight line is calculated, and the sum of these distances is taken as the error. This error function reflects the severity of image distortion under the current distortion parameters. Next, an optimization algorithm is used to adjust the distortion parameters to minimize the error.

[0037] In some possible embodiments, point p i The distance to line L is calculated using the following formula:

[0038]

[0039] In the formula, The distance from a point to a line. The coordinates of points on the distorted curve. The coordinates of the first point on the distortion curve. The coordinates of the last point on the distortion curve are given. The total error is then...

[0040] In the formula, N represents the number of points.

[0041] When optimizing distortion parameters based on the error function, the reward function used is as follows:

[0042]

[0043] In the formula, Reward represents the reward function, and E represents the error. The variance of the distance between points within the same curve is represented by α, β, and γ, which represent the weighting coefficients. The weighting coefficients can be determined through the optimization results of the validation set.

[0044] In some possible implementations, the optimization function chosen is the Powell method. The Powell method is a gradient-free optimization algorithm that iteratively updates radial and tangential distortion parameters by performing a line search along a predetermined direction in the parameter space. In each iteration, the algorithm tries different parameter combinations, calculates the corresponding error values, and ultimately finds the optimal parameters that minimize E. For example, in the initial stage, it is assumed that all distortion parameters are zero (i.e., no distortion). By gradually adjusting the parameters and observing the direction of error change, it eventually converges to the optimal solution. This process requires multiple iterations until the error change is below a set threshold or the maximum number of iterations is reached.

[0045] S13: Based on the optimal distortion parameters, perform orthogonal processing on the acquired image.

[0046] In this embodiment, the original image is orthogonally corrected using optimized distortion parameters. After obtaining the optimal distortion parameters, they need to be applied to the original image to eliminate distortion. The correction process is based on the inverse mapping of the distortion model, such as the Brown-Conrady model. For each pixel (u, v) in the image, it is first normalized to the camera coordinate system, the offsets of radial and tangential distortion are calculated, and then the corrected coordinates are obtained through inverse mapping. Since the corrected coordinates may be non-integer, methods such as bilinear interpolation are used to resample pixel values ​​to ensure image smoothness. In the corrected image, features such as originally curved lane lines and curbs should be restored to straight lines. For example, in a highway scene, curved lane lines captured by a wide-angle camera become parallel straight lines after correction, verifying the accuracy of the distortion parameters. In addition, the correction process needs to handle the distortion in the image edge region to avoid stretching or black edges, which can be solved by expanding the image boundary or cropping invalid areas.

[0047] S14: Extract ground coplanar feature points from the orthogonal image, combine them with their corresponding world coordinate system positions, and solve the camera extrinsic parameters, including rotation matrix and translation vector, using the SolvePnP algorithm.

[0048] In this embodiment, ground coplanar feature points are extracted from the orthogonal image, and their corresponding world coordinate system positions are combined to solve for the camera extrinsic parameters using the SolvePnP algorithm. The orthogonal image eliminates lens distortion, allowing for accurate extraction of geometric feature points on the ground, such as lane line endpoints and curb intersections. The pixel coordinates (u, v) of these feature points in the image are known, and it is assumed that they are located in the Z=0 plane (ground plane) of the world coordinate system. Their world coordinates (X, Y, 0) can be obtained through high-precision maps or manual measurement. Using these correspondences, the SolvePnP algorithm can solve for the camera's rotation matrix R and translation vector t. The algorithm is based on the following perspective projection model:

[0049]

[0050] In the formula, K represents the camera intrinsic parameter matrix, which is obtained through calibration, s represents the scale factor, R represents the rotation matrix, and t represents the translation vector.

[0051] By minimizing the reprojection error (i.e., the deviation between the predicted and actual pixel coordinates), SolvePnP uses optimization methods such as Levenberg-Marquardt to solve for R and t. For example, in an urban intersection scene, the camera mounting height and pitch angle can be analytically derived from the translation vector t and the Euler angles of the rotation matrix. The key to this step is the accurate matching of feature points. If there are errors in the ground feature points (such as lane line wear), outliers need to be removed by averaging multiple pairs of points or using the RANSAC algorithm.

[0052] S15: Evaluate the calibration error distribution based on simulation or real-world scenario data, dynamically adjust distortion model parameters or optimization strategies, and trigger online parameter updates when pose deviation exceeds a threshold.

[0053] In this embodiment, the calibration error distribution is evaluated based on simulation or real-world scenario data, and the distortion model parameters or optimization strategies are dynamically adjusted. After calibration, its accuracy needs to be verified. In a simulation environment, an image with known distortion parameters can be generated using a virtual camera, and the deviation between the calibration results and the true values ​​can be compared. In a real-world scenario, the calibration error can be measured using an object of known size (such as a standard lane width). Error analysis needs to focus on the distortion correction effect in different regions, such as whether the error is uniform between the image center and the edges. If the error is found to be large in the edge region, it may be necessary to expand the distortion model (e.g., by adding a higher-order term k4) or adjust the search range of the optimization algorithm. In addition, the system needs to support a dynamic update mechanism. For example, in traffic monitoring scenarios, the camera may experience pose shifts due to vibration or temperature changes. By monitoring the matching degree between vehicle trajectory and camera perception data in real time (e.g., lane departure warning), when the pitch angle error exceeds 0.3 degrees or the translation deviation exceeds a set threshold, the calibration parameters are automatically recalculated. Dynamic updates can combine historical data with current observations, using weighted averaging or sliding window methods to smooth parameter changes and avoid abrupt changes.

[0054] After obtaining accurate distortion parameters, the original image is corrected using distortion correction methods to restore the objects or edges in the image to their actual geometric shapes, ensuring that effective distortion correction can be achieved throughout the entire area of ​​the image (including the image center and edges).

[0055] In some possible embodiments, the method may also include step S16: suppressing noise interference by fusing multi-frame image data and Kalman filtering to ensure the robustness and real-time performance of calibration results in complex environments (such as rain, fog, and night).

[0056] In this embodiment, multi-frame image data fusion and Kalman filtering are used to suppress noise interference, ensuring the robustness and real-time performance of calibration results in complex environments. In practical applications, single-frame images may be affected by changes in illumination, motion blur, or weather interference (such as rain, fog, or low light at night), impacting calibration accuracy. Therefore, the system needs to fuse calibration results from multiple frames. For example, multiple frames are continuously acquired, and feature extraction, parameter optimization, and extrinsic parameter solving are performed separately. Then, the parameter sequence is smoothed using Kalman filtering. Kalman filtering effectively suppresses random noise by combining current observations with historical state estimates through a prediction-update loop. Taking camera height calibration as an example, assuming its change is slow, the Kalman model can be configured with low process noise, thus ensuring stable calibration results even in rainy or foggy weather. Furthermore, multi-frame fusion can also be used to detect outliers. For example, if a frame's feature point extraction is incorrect due to occlusion, the outlier frame can be removed through time consistency checks. In scenarios with high real-time requirements (such as autonomous driving), the computational efficiency of the algorithm needs to be optimized, for example, by using parallel computing or simplifying the model, to ensure that the calibration update frequency meets the real-time perception requirements. Ultimately, the system can maintain high-precision calibration in complex environments, reduce human intervention, and improve the reliability of traffic monitoring systems.

[0057] The above describes the camera calibration method without field operations provided in this application embodiment. Automatic calibration is achieved by extracting geometric prior features from the scene (such as lane lines, road edges, and other straight lines). The method first acquires traffic scene images, identifies curve features bent due to lens distortion, and discretizes them into point sets. Then, a geometric error function is constructed, the distances from feature points to the ideal straight line are calculated, and optimization algorithms such as Powell are used to solve for the optimal distortion parameters. Next, the parameters are used to perform orthogonal correction on the image to eliminate the influence of distortion. In the orthogonalized image, the camera extrinsic parameters (rotation matrix R and translation vector t) are solved using the SolvePnP algorithm, combining the pixel coordinates of ground feature points and world coordinates. Noise is suppressed through multi-frame data fusion and Kalman filtering. Parameter updates are triggered when the pose deviation exceeds a threshold. The method provided in this application embodiment eliminates the reliance on traditional calibration boards, utilizes the inherent features of traffic scenes to achieve high-precision calibration, is suitable for wide-angle lenses and complex environments, and significantly reduces calibration costs and maintenance difficulty.

[0058] In some possible embodiments, the camera may also experience deviations in the video image area due to external factors (such as vibration, wind, etc.). In this case, the camera's field of view can be restored by additionally setting up a millimeter-wave radar on the camera.

[0059] Specifically, cameras and millimeter-wave radar are installed at the intersection. The millimeter-wave radar can be integrated into the camera or set up separately, ensuring that both can simultaneously acquire images and radar data of the intersection. The camera is used to collect video images of the intersection, and the millimeter-wave radar is used to provide high-precision bird's-eye view (BEV) data. Time synchronization between the camera and the millimeter-wave radar is ensured to accurately match their data in subsequent processing. By analyzing the consistency between the camera images and the millimeter-wave radar data, it is determined whether the camera's video area has deviated due to external factors (such as vibration, wind, etc.). If the camera's field of view shifts, a correction procedure is executed. The execution process of the correction procedure is as follows:

[0060] S21: Extract spatiotemporal corner points from camera images and radar trajectories.

[0061] In this embodiment, spatiotemporal corner points in the image are identified from the camera video stream using feature extraction algorithms (such as Harris corner detection, ORB, SIFT, etc.). The spatial location (pixel coordinates u, v) of the corner points is extracted, and their motion in consecutive frames is tracked to form the object's trajectory. Simultaneously, corner points with the same spatiotemporal attributes are extracted from millimeter-wave radar data. It is worth noting that radar does not directly provide corner points, but they can be obtained by processing its point cloud or target list data. For example, the precise 3D position of a stationary object with obvious angular reflection characteristics (such as the base of a lamppost, the corner of a traffic sign, or the end of a guardrail) in the radar coordinate system. Millimeter-wave radar data provides high-precision object position information, which serves as the spatial information of the corner points. Given the high precision of millimeter-wave radar data, it is used as the near-true value of the corner point spatial information, providing the camera corner points with high-precision corresponding 3D coordinates in the real world. Camera corner points and radar corner points belonging to the same physical target within the same time period are matched. For example, the corner point of a stationary lamppost in an image must be matched with the corner point position of that lamppost in the radar data. In this way, a correspondence is established between 2D image pixels and 3D radar world points.

[0062] S22: Based on the matched spatiotemporal corner points, calculate the homography transformation matrix between the camera image plane and the millimeter-wave radar data plane.

[0063] In this embodiment, a 3x3 homography transformation matrix is ​​calculated based on multiple matched 2D and 3D point pairs. The homography transformation matrix describes the projection mapping relationship (i.e., geometric relationship) between two planes, including rotation, translation, and scaling. The homography transformation matrix can represent the transformation from the radar-detected real-world ground plane (or other planes containing coplanar features) to the camera image plane. By decomposing this homography transformation matrix, the camera's rotation matrix and translation vector can be extracted. The decomposed rotation matrix is ​​converted into a rotation vector using the Rodriguez formula, and this rotation vector is further converted into a quaternion, denoted as the first quaternion.

[0064] S23: Determine the viewing angle correction parameters based on the camera parameters.

[0065] In this embodiment, under ideal conditions, the camera's intrinsic parameters are known. Since the intrinsic parameter K is fixed and known, the rotation matrix and translation vector can be solved more accurately using the Direct Linear Transform (DLT) or PnP algorithm based on the matched 2D-3D point pairs. This rotation matrix is ​​converted into a quaternion, denoted as the second quaternion. The angle difference between this second quaternion and the original first quaternion is calculated. If this angle difference is large, interpolation can be performed between the first and second quaternions during correction, making the correction process smooth and gradual rather than abrupt, avoiding sudden image jumps. The high precision of the radar data ensures the accuracy of the solution. The correction process involves replacing the old, deviated extrinsic parameters (i.e., the rotation matrix and translation vector obtained in step S22) in the system with the newly calculated rotation matrix and translation vector. When the camera's intrinsic parameters cannot be determined, the homography transformation matrix simultaneously incorporates changes in both intrinsic and extrinsic parameters. In this case, the homography transformation can be adjusted based on the error between the spatiotemporal corner points and the radar trajectory, making the error distribution closer to the spatial distribution in the image space. The process involves projecting radar points onto the image using the current homography matrix and adjusting the homography matrix based on the error distribution between this projection and the detected image corner points. Through iterative optimization, an optimal homography matrix is ​​found that minimizes the error and ensures a reasonable distribution. This process essentially involves jointly optimizing intrinsic and extrinsic parameters, but with extrinsic parameter correction as the primary objective. Specifically, a reprojection error function is constructed using the homography matrix as the initial value. This function represents the Euclidean distance between the coordinates of a radar world point projected onto the image using the homography matrix and the coordinates of the corresponding feature corner points in the image. The parameters of the homography matrix are iteratively optimized using a nonlinear optimization algorithm to minimize the reprojection error function. The converged homography matrix is ​​then used as the final viewpoint correction parameter.

[0066] S24: Perform real-time perspective transformation on the camera image based on the correction parameters.

[0067] In this embodiment, the finalized correction parameters (which may be a new extrinsic rotation matrix and translation vector, or the entire homography transformation matrix) are used to perform a real-time perspective transformation on the camera image. After the transformation, the viewpoint of the image is corrected, as if the camera has returned to the correct position and angle. This allows subsequent image-based perception algorithms (such as vehicle detection and lane line recognition) to continue to function normally, unaffected by physical disturbances to the camera.

[0068] The camera calibration method without field operations proposed in this application is applied to traffic monitoring scenarios such as highway ramp merging areas and urban intersections. Wide-angle cameras or radar-visual fusion devices (covering a range of 200 meters) deployed on the main road and ramps serve as target perception units. All vehicles (including CAVs and HDVs) communicate with roadside units (RSUs) through onboard units (OBUs) to upload vehicle location and image data in real time. Based on multiple frames of traffic scene images (including lane lines, road edges, and other natural features) collected by the RSUs, the control center automatically extracts the set of curve points in the images that are "straight lines in reality but distorted in the image". It samples and constructs a nonlinear optimization model by region (central area, transition area, edge area) and jointly solves the radial and tangential distortion parameters to complete the camera intrinsic parameter calibration. Subsequently, using the geometric prior of ground coplanar points (such as lane line endpoints) (Z=0 plane) and world coordinates in a high-precision map, the solvePnP algorithm is used to automatically calculate the camera extrinsic parameters (pitch angle, yaw angle, installation height). The calibration results are sent to the CAVs and traffic management platform through the RSUs. During dynamic operation, the system monitors the matching degree between the CAV trajectory and the camera's perceived data in real time. If a pose deviation is detected (such as a pitch angle error > 0.3°), the calibration parameters are updated online. By combining multi-frame data fusion and Kalman filtering to suppress noise, the system ensures calibration accuracy in complex environments such as rain, fog, and nighttime. This method can reduce traffic interruptions caused by calibration operations and enable the deployment of an all-weather, zero-intervention traffic perception system.

[0069] This method requires no fieldwork. Based on the inherent geometric features of traffic scenarios (such as the straightness of lane lines and curbs, and the coplanarity of the ground), it automatically extracts distortion feature points covering the entire image from multiple frames. This eliminates the need for manual labeling or the placement of calibration objects, avoiding the high-cost operations of closing roads and erecting calibration boards in traditional methods. Since no fieldwork is required, this invention eliminates the need to purchase high-precision tools such as calibration boards, 3D calibration targets, and total stations, significantly reducing the calibration cost per device. Simultaneously, it reduces the frequency of manual inspections and maintenance, lowering manpower requirements and improving the universality of the calibration. This method is applicable to traffic monitoring, roadside detection, and radar-based systems, providing a stable and reliable calibration solution, especially in harsh environments or where field calibration is not feasible.

[0070] By installing cameras and millimeter-wave radar at intersections, and utilizing the high-precision bird's-eye view (BEV) data provided by the millimeter-wave radar as a near-true value, combined with deep reinforcement learning algorithms, deviations in camera perspective caused by external factors are corrected in real time. When dynamic scenarios such as frequent vehicle movement or pedestrian crossings occur, this technology can accurately capture the motion characteristics of objects through spatiotemporal corner point matching and trajectory optimization. Even without strict time synchronization, it can use radar data as a reference to quickly calculate the correction parameters for the camera perspective, ensuring the accuracy and stability of the monitored images. This not only improves the efficiency and reliability of traffic monitoring but also provides more accurate data support for traffic violation detection, traffic flow analysis, and intelligent traffic management, effectively enhancing the traffic efficiency and safety at intersections. By using the high-precision spatial information provided by millimeter-wave radar, combined with feature points in the camera images, perspective correction is automatically completed without manual field calibration, avoiding the high-cost operations of traditional methods such as closing roads and erecting calibration boards. By utilizing spatiotemporal corner matching and homography transformation calculation, it can accurately capture the motion characteristics of objects and maintain high-precision view correction even in complex dynamic scenes. It can adapt to different environmental changes and ensure stable correction effects in complex environments such as rain, fog, and night.

[0071] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Furthermore, in some possible implementations, each step in the above embodiments may be selectively executed according to actual circumstances; it may be partially or fully executed, without limitation here. All or part of any feature of any embodiment of this application can be freely and arbitrarily combined without contradiction. The combined technical solutions are also within the scope of this application.

[0072] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0073] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0074] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0075] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0076] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0077] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

Claims

1. A method for continuous calibration in traffic scenarios without field testing, characterized in that, The method includes: Traffic scene images captured by a camera are acquired, and geometric prior features are extracted from the images. The geometric prior features are used to characterize points existing on distorted curves in the traffic scene images. The geometric prior features include at least one of lane lines, road edges, and building edges. Define an error function and optimize the distortion parameters based on the error function. The distortion parameters are used to characterize the geometric offset in the image caused by lens distortion, including radial distortion parameters and tangential distortion parameters. Based on the optimal distortion parameters, a nonlinear optimization algorithm is used to perform orthogonal processing on the acquired images; In the orthogonal image, ground coplanar feature points are extracted, and combined with their corresponding world coordinate system positions, the camera extrinsic parameters, including rotation matrix and translation vector, are solved by the SolvePnP algorithm. Based on simulation or real-world scenario data, evaluate the calibration error distribution, dynamically adjust distortion model parameters or optimization strategies, and trigger online parameter updates when pose deviation exceeds a threshold. A camera and a millimeter-wave radar are installed, with the millimeter-wave radar set up independently of the camera. The system synchronously acquires camera images and millimeter-wave radar point cloud data, extracts feature corner points from the camera images, and identifies corresponding static target corner points in the millimeter-wave radar point cloud data to generate multiple 2D-3D matching point pairs between image pixel coordinates and radar world coordinates. Based on these matching point pairs, a homography transformation matrix between the camera image plane and the radar world coordinate system is calculated using a direct linear transformation. Given the camera's intrinsic parameters, the rotation matrix and translation vector of the camera relative to the radar world coordinate system are obtained through matrix decomposition based on the homography transformation matrix and the camera's intrinsic parameter matrix, serving as the viewpoint correction parameters. Based on the rotation matrix and translation vector, a perspective transformation is performed on the camera image to correct the viewpoint deviation caused by the camera pose offset. When the camera's intrinsic parameters cannot be determined, a reprojection error function is constructed using the homography matrix as the initial value. The reprojection error function represents the Euclidean distance between the coordinates of the radar world coordinate point projected by the homography matrix and the coordinates of the corresponding feature corner point in the image. The parameters of the homography matrix are iteratively optimized using a nonlinear optimization algorithm to minimize the reprojection error function. The optimized and converged homography matrix is ​​used as the final viewpoint correction parameter. The optimized homography matrix is ​​then used to perform an inverse perspective transformation on the camera image to correct the viewpoint deviation.

2. The method according to claim 1, characterized in that, The method further includes: using multi-frame image data fusion and Kalman filtering to suppress noise interference, ensuring the robustness and real-time performance of calibration results in complex environments, including but not limited to rainy / foggy environments and nighttime environments.

3. The method according to claim 1, characterized in that, The dynamic adjustment includes: Noise interference is suppressed by multi-frame data fusion and Kalman filtering; When the pitch angle error exceeds 0.3 degrees or the translation deviation exceeds the preset threshold, the calibration parameters are recalculated.

4. The method according to claim 1, characterized in that, The nonlinear optimization algorithm is the Powell method, the least squares method, or the Levenberg-Marquardt algorithm.

5. The method according to claim 1, characterized in that, The error function is the sum of the geometric distances from each point on the distorted curve to the ideal straight line, and the ideal straight line is the line connecting the first point and the last point on the distorted curve.

6. The method according to claim 5, characterized in that, The reward function used for optimizing distortion parameters based on the error function is as follows: In the formula, Reward represents the reward function, and E represents the error. The variance of the distance between points within the same curve is represented by α, β, and γ, which represent the weighting coefficients. The weighting coefficients are determined through the optimization results of the validation set.

7. The method according to claim 6, characterized in that, The error function is calculated according to the following formula: In the formula, E represents the error function. It represents the geometric distance from a point on the distorted curve to the ideal straight line.

8. The method according to claim 7, characterized in that, The geometric distance from a point on the distorted curve to the ideal straight line is calculated using the following formula: In the formula, The coordinates of points on the distorted curve. The coordinates of the first point on the distortion curve. The coordinates of the last point on the distortion curve.

Images