Camera external parameter calibration method based on virtual lane line and virtual object

Abstract

The invention belongs to the technical field of computer vision, and particularly relates to a camera external parameter calibration method based on a virtual lane line and a virtual object. The method comprises the following steps: designing a virtual object generator, constructing a vehicle coordinate system and a world coordinate system according to vehicle IMU data and configuration file parameters, and generating a virtual object and a virtual lane line in the world coordinate system; capturing a projection of a three-dimensional point on a virtual object on an image plane of the camera, and calculating a rotation matrix and a translation vector of the camera so as to accurately calibrate a roll angle, a pitch angle, a height and a yaw angle; optimizing a yaw angle, a roll angle and a pitch angle by adopting an LM algorithm; and constructing a coordinate transformation matrix according to the external parameters, and performing coordinate system adjustment to obtain a calibrated camera coordinate system. The virtual object generator is used for calibration from the virtual environment level, dependence on the actual environment is avoided, calibration precision, repeatability and stability are improved, and efficient calibration can be carried out in various complex environments.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a method for calibrating the external parameters of a camera based on a virtual generator and a virtual environment. Background Art

[0002] As a key environmental perception sensor, the accuracy of the external parameter calibration of an in-vehicle camera directly affects the vehicle's ability to understand the surrounding environment and make decisions during driving. The external parameter calibration of a camera is to accurately determine the position information and attitude information of the camera relative to a reference coordinate system (such as a vehicle coordinate system). During the camera installation process, the camera may not be perfectly aligned with the predetermined vehicle coordinate system, and even a small deviation in the angle during manual installation may cause rotational errors. Therefore, external parameter calibration is crucial for applications such as autonomous vehicles that need to understand the relationship between the camera's perspective and the surrounding environment.

[0003] The yaw angle, roll angle, and pitch angle represent the misalignment between the camera coordinate system and the vehicle coordinate system. The purpose of calibration is to measure these angles and make the two coordinate systems as parallel as possible by applying the rotation matrix composed of these three angles. Traditional camera external parameter calibration methods usually rely on external environmental features, such as parallel lane markings or vertical poles, to generate vanishing points through these features and then calculate the external parameters of the camera. This method works well under ideal conditions, but in practical applications, it is affected by the environment and weather, and the repeatability is poor. A camera external parameter calibration method that can be independent of actual environmental conditions is of great significance for improving the accuracy and reliability of calibration. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for calibrating the external parameters of a camera based on virtual lane lines and virtual objects to overcome the dependence on the actual environment of traditional methods and improve the flexibility and accuracy of the calibration process.

[0005] The method for calibrating the external parameters of a camera based on virtual lane lines and virtual objects provided by the present invention accurately calibrates the external parameters of the camera based on projection points and vanishing points by generating virtual lane lines and virtual objects; the specific steps are as follows:

[0006] (1) Generation of virtual lane lines and virtual objects:

[0007] Design a virtual object generator; based on IMU data (yaw angle, roll angle, pitch angle, and velocity components vx, vy, vz in the x, y, and z directions) and configuration file parameters (lane width, object height, etc.), construct a world coordinate system and a vehicle coordinate system, generate virtual lane lines and virtual objects in the world coordinate system, and further generate vanishing points formed by the intersection of the extended lines of parallel lane lines;

[0008] (2) Calibration of roll angle, pitch angle and height: By capturing three-dimensional points on the virtual object and projecting them onto the camera image plane, the rotation matrix and translation vector of the camera are calculated using the PnP algorithm (Perspective-n-Point) [1] to accurately calibrate the roll angle, pitch angle and height;

[0009] (3) Calibration of yaw angle: Normalize the image plane and calibrate the yaw angle through the vanishing point;

[0010] (4) Parameter optimization: Use the Levenberg-Marquardt algorithm (abbreviated as LM algorithm) to optimize the parameters of the camera's yaw angle, roll angle, pitch angle and height to minimize the root mean square error (RMSD) between the observed points and the predicted points;

[0011] (5) Coordinate adjustment: Finally, construct a coordinate transformation matrix based on the calibrated external parameters of the camera, perform coordinate system adjustment, obtain the calibrated camera coordinate system, and achieve the alignment and calibration of the camera coordinate system and the vehicle coordinate system.

[0012] In step (1), the virtual object generator includes using the Transformer model to extract temporal features and perform context modeling on the acquired IMU data, and effectively capturing the complex patterns of vehicle motion through the multi-head self-attention mechanism, thereby improving the accuracy and real-time performance of virtual lane line and virtual object generation.

[0013] In step (1), the generation of virtual lane lines and virtual objects based on the virtual object generator is as follows:

[0014] (1.1) Acquire IMU data, including yaw angle, roll angle, pitch angle, and velocity components vx, vy, vz in the x, y, and z directions, and read parameters (vehicle parameters, virtual lane / virtual target parameters) from the configuration file;

[0015] (1.2) Construct coordinate systems (world coordinate system and vehicle coordinate system) based on the IMU data;

[0016] (1.3) Construct virtual objects and virtual lanes in the world coordinate system based on the vehicle attitude information;

[0017] (1.4) Project the virtual lane line points onto the camera image plane, and generate a vanishing point according to the intersection points of the projection lines for camera external parameter calibration.

[0018] In step (2), the specific steps for calibrating the roll angle, pitch angle and height are as follows:

[0019] (2.1) In a virtual environment, use a virtual object / virtual calibration target with a known position in the world coordinate system, such as a checkerboard or a plane generated by a virtual object generator;

[0020] (2.2) Use the internal parameters of the camera (focal length, principal point, etc.) to capture the two-dimensional projection point P of the virtual object on the camera image plane image ;

[0021] (2.3) According to the formula P image = D·K·(R CW P world + T CW ), use the PnP algorithm to calculate the rotation matrix R CW from the world coordinate system to the camera coordinate system and the translation vector T CW , where P world is the three-dimensional point in the world coordinate system, K is the internal parameter matrix of the camera, and D is the scaling factor of the perspective projection;

[0022] After the above steps, a relatively accurate roll angle, pitch angle, and height aligned with the vehicle coordinate system can be obtained. However, the yaw angle still needs to be further refined and calibrated.

[0023] In step (3), the calibration of the yaw angle is specifically as follows:

[0024] (3.1) Keep the vehicle parallel to the virtual lane line, and the orientation of the camera is (sin(yaw), 0, cos(yaw));

[0025] (3.2) Normalize the camera image plane so that the vanishing point is at the center (0, 0), and the position of the vanishing point with a yaw angle error is (tan(yaw), 0);

[0026] (3.3) Calculate the yaw angle and apply the calculated yaw angle to the camera.

[0027] In step (3), the parameter optimization method based on the Levenberg-Marquardt algorithm is specifically as follows:

[0028] (4.1) Define the optimization objective as finding the yaw angle, roll angle, and pitch angle of the camera model parameters that minimize the root mean square difference (RMSD) between the observation point y i and the predicted position p(x i | yaw, roll, pitch), and the formula is:

[0029]

[0030] This is a non-linear least squares problem;

[0031] (4.2) Solve the non - linear least - squares problem using the Levenberg - Marquardt algorithm. First, calculate the Jacobian matrix J of the RMSD with respect to the yaw angle, roll angle, and pitch angle, and then update the yaw angle, roll angle, and pitch angle. The update formula is:

[0032] (J T J + λI) -1 J T (y i - P(x i |yaw, roll, pitch));

[0033] (4.3) Repeat steps 4.1 and 4.2 until convergence.

[0034] In step (5), the coordinate adjustment is specifically as follows: The external parameter matrix obtained is:

[0035]

[0036] where R yaw , R pitch , R roll , T height are the calibrated yaw angle, pitch angle, roll angle, and height, and the coordinate transformation matrix is constructed as:

[0037] R = R yaw × R pitch × R roll ,

[0038] The final coordinate transformation formula is:

[0039] Coordinate adjust = R × Coordinate original + T height ,

[0040] where Coordinate original is the original camera coordinate system, and Coordinate adjust is the calibrated camera coordinate system. Compared with the prior art, the present invention mainly has the following features and advantages:

[0041] (1) The method for generating virtual lane lines and virtual objects proposed by the present invention is customizable. The virtual lane lines and virtual objects can be accurately positioned and generated, ensuring the accuracy of alignment, positioning, and distance, and improving the accuracy and reliability of calibration.

[0042] (2) The virtual object generator proposed by the present invention can configure the virtual environment according to different types of virtual objects, support calibration tests under non - ideal conditions, and enhance the robustness and adaptability of the system.

[0043] (3) The camera extrinsic parameter calibration method based on virtual lane lines and virtual objects proposed by the present invention uses virtual targets to avoid problems related to distortion or inaccuracy of physical objects, has environmental independence, and improves the flexibility of the calibration process.

[0044] (4) Conducting calibration experiments in a virtual environment proposed by the present invention can be repeated multiple times under controllable conditions to ensure the stability and consistency of the calibration results, and is suitable for large-scale production and deployment. Description of the Drawings

[0045] Figure 1 It is a flowchart of the working process of the virtual generator of the present invention.

[0046] Figure 2 It is a schematic diagram of the yaw angle calibration scenario based on the vanishing point of the present invention.

[0047] Figure 3 It is a schematic diagram of the rotation directions of the parameter matrices of the present invention.

[0048] Figure 4 It is a schematic diagram of the coordinate adjustment of the present invention. Detailed Embodiment

[0049] The present invention will be further introduced below through embodiments in combination with the drawings.

[0050] The camera extrinsic parameter calibration method based on virtual lane lines and virtual objects provided by the present invention generates lane lines and objects in a simulation environment through a virtual generator, and uses virtual targets to calibrate the extrinsic parameters of the camera, thereby avoiding the influence of uncertain factors in the actual environment. The specific steps are as follows:

[0051] (1) Use a virtual generator to construct a virtual lane and virtual objects, and then generate a vanishing point;

[0052] The present invention uses deep learning and the Transformer model as the virtual generator to generate virtual lane lines and virtual objects that conform to the actual driving scenario, uses the self-attention mechanism of the Transformer to optimize the generation process of virtual objects, predicts and models the future vehicle trajectory, and applies generative adversarial network (GAN) technology to enhance the authenticity and complexity of the virtual environment;

[0053] In the virtual generator, a pre-trained model is used to generate virtual lanes and virtual objects, specifically including: using the pre-trained AgentFormer[3] to model the lane line trajectory and virtual object trajectory, and training 3D-GAN[4] to generate virtual lane lines and virtual objects. The 3D-GAN generator network contains 5 convolutional layers with a kernel size of 4*4*4, which upsamples the noise vector. The discriminator uses 5 3D convolutional layers. The adversarial loss function is L = logD(x) + log(1 - D(G(z))), where logD(x) is the binary cross-entropy loss or classification loss, log(1 - D(G(z))) is the adversarial loss, D(x) is the output of the discriminator network, G(z) is the output of the generator network, and z is the latent vector from the probability space p(Z).

[0054] The working process of the virtual generator is as Figure 1 shown and includes the following steps:

[0055] (1.1) Obtain real-time IMU data, and read vehicle parameters and configuration parameters of virtual lanes and virtual objects from the configuration file.

[0056] (1.2) Construct the world coordinate system and vehicle coordinate system based on the IMU data to ensure the consistency of the virtual environment and vehicle dynamic information.

[0057] (1.3) Based on the vehicle's pose information, generate virtual objects and virtual lane lines with predefined positions and poses in the world coordinate system. Among them, the virtual lane line trajectory and virtual object trajectory predicted by the pre-trained AgentFormer from the IMU sequence data are used as the model input of 3D-GAN to generate more realistic 3D virtual lane lines and virtual objects.

[0058] (1.4) Use the generated virtual lane lines to calculate the intersection points of the extended lane lines, which are the vanishing points, as the reference for subsequent camera extrinsic parameter calibration. (2) Camera extrinsic parameter calibration;

[0059] The camera extrinsic parameter calibration process includes two sub-steps: using the PnP algorithm to calibrate the roll angle, pitch angle, and height; calibrating the yaw angle based on the vanishing point. Among them:

[0060] (1) Calibration of the roll angle, pitch angle, and height based on the PnP algorithm. The specific process is as follows

[0061] The input is the virtual calibration target generated by the virtual object generator, such as a checkerboard or a plane; assume the virtual lane line generated by the virtual generator is: lane left= [(-1.75, 0.0, 0.0), (-1.75, 1.0, 0.0), (-1.75, 2.0, 0.0),..., (-1.75, 50.0, 0.0), lane right = [(1.75, 0.0, 0.0), (1.75, 1.0, 0.0), (1.75, 2.0, 0.0),...,(1.75, 50.0, 0.0)]; Assume 2 virtual objects are generated, and each object is represented by a center point and length, width and height: objects = [(20.0, 0.0, 0.0, 4.0, 1.8, 1.5), (35.0, -2.0, 0.0, 3.0, 1.5, 1.2)]

[0062] (1.1) Use virtual objects with known positions in the world coordinate system in the virtual environment; Assume the virtual object 20m ahead is selected, and its 3D corner points are:

[0063] (18.0, -0.9, 0.0), (18.0, 0.9, 0.0), (22.0, -0.9, 0.0), (22.0, 0.9, 0.0), (18.0, -0.9, 1.5), (18.0, 0.9, 1.5), (22.0, -0.9, 1.5), (22.0, 0.9, 1.5).

[0064] (1.2) Use the FindChessboardCorners function [2] of the OpenCV library to extract the coordinates of the two-dimensional projection points of the three-dimensional points on the virtual object in the camera image plane; Assume the resolution of the camera internal parameters is 640*480, the principal point is (320, 240), and the focal length is 800 pixels. Finally, the OpenCV extracts the two-dimensional projection points of the object as (310, 400), (330, 400), (300, 380), (340, 380), (312, 350), (328, 350), (302, 330), (338, 330).

[0065] (1.3) Input the known three-dimensional virtual points and their corresponding two-dimensional projection points into the PnP algorithm, and combine the camera internal parameters to calculate the external parameters of the camera relative to the virtual object, including the rotation matrix and the translation vector. The formula is:

[0066] P image = D·K·(R CW P world + T CW ); (1)

[0067] After this step, a relatively accurate roll angle, pitch angle and height aligned with the vehicle coordinate system can be obtained, but the yaw angle still needs to be further refined and calibrated. The formula is:

[0068] P image = D·K·(R adjust |T adjust )(R CW P world + T CW ), (2)

[0069] Wherein, R adjust is the rotation matrix after refinement adjustment, T adjust is the translation vector after refinement adjustment, and R adjust |T adjust combines the two to form the adjustment part of the external camera parameters.

[0070] The finally calculated rotation matrix R = [[0.999, -0.015, 0.037], [0.016, 0.999, -0.010], [-0.037, 0.011, 0.999]], the translation vector T = [[-0.2], [1.45], [-20.0]], and the final roll angle pitch = arcsin(-R 31 ) = -2.0°, height = 1.45m.

[0071] (2) Yaw angle calibration based on the vanishing point, the scenario is as Figure 2 shown, and the extension lines of parallel lane lines intersect to form a vanishing point. The yaw angle calibration process is as follows:

[0072] The input is two virtual lane lines parallel to the vehicle and the vanishing point generated by them.

[0073] (2.1) Keep the vehicle parallel to the virtual lane lines, and the orientation of the camera (sin(yaw), 0, cos(yaw)) is determined by the yaw angle;

[0074] (2.2) Apply an image processing algorithm to normalize the image captured by the camera, adjust the image coordinate system, so that the vanishing point in the ideal case is located at the center of the image. Any offset caused by the yaw angle will cause the position of the vanishing point to change relative to the center, offset to the position of (tan(yaw), 0);

[0075] (2.3) Measure the offset of the vanishing point relative to the center of the image, and calculate the yaw angle (yaw) of the camera through geometric relationships. Apply the calculated yaw angle parameter to the external camera parameter model to adjust the orientation of the camera to correct the yaw error. Therefore,

[0076] The present invention provides a virtual calibration target with a precisely known position through a virtual object generator, and calibrates the roll angle, pitch angle, and height of a camera by combining the PnP algorithm and the internal parameters of the camera; then, through virtual lane lines and vanishing points, combined with image normalization processing and geometric analysis, the yaw angle of the camera is accurately measured and corrected, realizing comprehensive external parameter calibration.

[0077] (3) Parameter optimization;

[0078] The goal of parameter optimization is to minimize the root mean square difference (RMSD) between the actual observation point y in the image plane i and the predicted position p(x i |yaw, roll, pitch) according to the current camera model parameters. The parameter optimization formula is defined as the following formula (3). By minimizing the error function E, the optimal external parameters of the camera can be obtained, making the attitude and position of the camera model in the vehicle coordinate system more accurate.

[0079]

[0080] This is a non - linear least - squares problem.

[0081] The Levenberg Marquardt algorithm is used to solve the above non - linear least - squares problem, including the following steps:

[0082] (3.1) Initialize the values of the yaw angle, roll angle, and pitch angle, and set the initial damping factor λ;

[0083] (3.2) For each observation point y i Calculate the predicted position P(x i |yaw, roll, pitch), and calculate the error:

[0084] e i = y i - P(x i |yaw, roll, pitch); (4)

[0085] Calculate the total error:

[0086]

[0087] (3.3) Construct the Jacobian matrix J. Each column of J corresponds to the partial derivative of the error with respect to the yaw angle, roll angle, and pitch angle, and calculate the partial derivative of the error e i with respect to each parameter

[0088] (3.4) According to the Levenberg - Marquardt algorithm, the calculation formula for updating the step size Δ is:

[0089] (J T J + λI) -1 J T [e1, e2, …, e n T , where λI is the product of the damping factor and the identity matrix, used to control the adjustment of the step size.

[0090] (3.5) Update each parameter according to the calculated step size Δ:

[0091] [yaw, roll, pitch] T ← [yaw, roll, pitch] T + Δ;

[0092] (3.6) Repeat steps 3.1 to 3.5 until the error E converges.

[0093] The present invention optimizes the external parameters of the camera by using the Levenberg - Marquardt algorithm, minimizing the root - mean - square error between the observation point and the predicted position, and significantly improving the accuracy of the yaw angle, roll angle, and pitch angle of the camera model. The optimized external parameters are yaw = 10.05°, roll = 1.0°, pitch = - 2.05°, height = 1.50m

[0094] (IV) Coordinate transformation

[0095] The present invention finally obtains the following yaw angle, pitch angle, roll angle matrix, and height vector:

[0096]

[0097] Among them, the rotation directions of each matrix are as Figure 3 shown. The yaw angle matrix is used for the transformation of rotation around the Z - axis (vertical axis) of the vehicle coordinate system, the pitch angle matrix is used for the transformation of rotation around the Y - axis (lateral axis) of the vehicle coordinate system, the roll angle matrix is used for the transformation of rotation around the X - axis (front - rear axis) of the vehicle coordinate system, h x , h y , h z respectively represent the translation amounts of the camera in the X, Y, and Z - axis directions of the vehicle coordinate system.

[0098] Furthermore, the rotation matrix R and displacement vector T of the camera relative to the vehicle coordinate system are obtained, where the rotation matrix R is composed of the yaw angle, pitch angle, and roll angle:

[0099] R = R yaw × R pitch × R roll .

[0100] The coordinate adjustment process, as​Figure 4 As shown, the formula is:

[0101] Coordinate adjust =R×Coordinate orignal +T height .

[0102] Where Coordinate orignal represents the original coordinate point in the camera coordinate system, and Coordinate adjust represents the adjusted coordinate point, which has been transformed into the vehicle coordinate system. The final position of the camera in the vehicle coordinate system is T=(0, 1.50, 0).

[0103] The present invention comprehensively forms the rotation matrix and displacement vector of the camera relative to the vehicle coordinate system. Using these parameters, coordinate adjustment is implemented to achieve high-precision alignment between the camera coordinate system and the vehicle coordinate system.

[0104] References

[0105] [1] Fischler MA, Bolles RC. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381 - 395.

[0106] [2] Camera Calibration and 3D Reconstruction—OpenCV 2.4.13.7 documentation

[0107] [3] Khrylx / AgentFormer: [ICCV 2021] Official PyTorch Implementation of "AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting".

[0108] [4]Wu J, Zhang C, Xue T, et al. Learning a probabilistic latent space of object shapes via 3d generative - adversarial modeling[J]. Advances in neural information processing systems, 2016, 29。

Claims

1. A method for calibrating the external parameters of a camera based on virtual lane lines and virtual objects, characterized in that, Precisely calibrate the extrinsic parameters of the camera based on the projection points and vanishing points by generating virtual lane lines and virtual objects. The specific steps are as follows: (1) Generation of virtual lane lines and virtual objects: Design a virtual object generator, and construct a world coordinate system and a vehicle coordinate system according to the IMU data and configuration file parameters. Generate virtual lane lines and virtual objects in the world coordinate system, and further generate the vanishing point formed by the intersection of the extension lines of the parallel lane lines. The IMU data includes yaw angle, roll angle, pitch angle, and velocity components vx, vy, vz in the x, y, and z directions. The configuration file parameters include lane width, object height, vehicle parameters, and virtual lane / virtual target parameters. (2) Calibration of roll angle, pitch angle, and height: Capture the three-dimensional points on the virtual object and project them onto the camera image plane, and use the PnP algorithm to calculate the rotation matrix and translation vector of the camera to precisely calibrate the roll angle, pitch angle, and height. (3) Calibration of yaw angle: Normalize the image plane and calibrate the yaw angle through the vanishing point. (4) Parameter optimization: Use the Levenberg-Marquardt algorithm to optimize the parameters of the yaw angle, roll angle, pitch angle, and height of the camera to minimize the root mean square error (RMSD) between the observation points and the predicted points. (5) Coordinate adjustment: Finally, construct a coordinate transformation matrix based on the calibrated extrinsic parameters of the camera, perform coordinate system adjustment, obtain the calibrated camera coordinate system, and achieve the alignment and calibration of the camera coordinate system and the vehicle coordinate system.

2. The camera external parameter calibration method according to claim 1, wherein, The virtual generator described in step (1) includes using a Transformer model to extract temporal features and perform context modeling on the acquired IMU data, and capturing the complex patterns of vehicle motion through the multi-head self-attention mechanism, thereby improving the accuracy and real-time performance of virtual lane line and virtual object generation.

3. The method for calibrating the external parameters of a camera according to claim 2, wherein, The specific process of generating virtual lane lines and virtual objects based on the virtual object generator in step (1) is as follows: (1.1) Obtain the IMU data, including yaw angle, roll angle, pitch angle, and velocity components vx, vy, vz in the x, y, and z directions, and read the parameters from the configuration file, including lane width, object height, vehicle parameters, and virtual lane / virtual target parameters. (1.2) Construct a world coordinate system and a vehicle coordinate system according to the IMU data. (1.3) Based on the vehicle pose information, construct virtual objects and virtual lanes in the world coordinate system. (1.4) Project the virtual lane line points onto the camera image plane, and generate a vanishing point according to the intersection points of the projection lines for camera extrinsic parameter calibration.

4. The method for calibrating the external parameters of a camera according to claim 3, characterized in that, The specific steps of the roll angle, pitch angle, and height calibration described in step (2) are as follows: (2.1) In the virtual environment, use virtual objects / virtual calibration targets with known positions in the world coordinate system, including checkerboards or planes generated by the virtual object generator. (2.2) Use the internal parameters of the camera to capture the two-dimensional projection point P of the virtual object on the camera image plane image ; ( 2.3) According to the formula P image = D·K·(R CW P world + T CW ), use the PnP algorithm to calculate the rotation matrix P CW from the world coordinate system to the camera coordinate system and the translation vector T CW , where P world is a three-dimensional point in the world coordinate system, K is the internal parameter matrix of the camera, and D is the scaling factor of the perspective projection; After the above steps, obtain relatively accurate roll angle, pitch angle, and height aligned with the vehicle coordinate system. For the yaw angle, further refined calibration is required.

5. The method for calibrating the external parameters of a camera according to claim 4, wherein The calibration of the yaw angle described in step (3) is specifically as follows: (3.1) Keep the vehicle parallel to the virtual lane line, and the orientation of the camera is (sin(yaw), 0, cos(yaw)); (3.2) Normalize the camera image plane so that the vanishing point is at the center (0, 0), and the position of the vanishing point with yaw angle error is (tan(yaw), 0); (3.3) Calculate the yaw angle and apply the calculated yaw angle to the camera.

6. The method for calibrating the external parameters of a camera according to claim 5, characterized in that, The parameter optimization based on the Levenberg - Marquardt algorithm described in step (4) is specifically as follows: (4.1) Define the optimization objective as finding the camera model parameter yaw angle, roll angle, and pitch angle that minimize the root mean square difference (RMSD) between the observation point y i and the predicted position p(x i |yaw, roll, pitch), and the formula is (4.2) Solve the nonlinear least squares problem using the Levenberg Marquardt algorithm. First, calculate the Jacobian matrix J of the RMSD with respect to the yaw angle, roll angle, and pitch angle, and then update the yaw angle, roll angle, and pitch angle. The update formula is (J T J + λI) -1 J T (y i - P(x i | yaw, roll, pitch)); (4.3) Repeat steps 4.1 and 4.2 until convergence.

7. The camera extrinsic parameter calibration method according to claim 1, wherein The coordinate adjustment described in step (5), specifically, the obtained external parameter matrix is: Among them, R yaw , R pitch , R roll , T height are the calibrated yaw angle, pitch angle, roll angle and height. The constructed coordinate transformation matrix is as follows: R = R yaw ×R pitch ×R roll , The final coordinate transformation formula is Coordinate adjust = R × Coordinate original + T height , where Coordinate original is the original camera coordinate system, and Coordinate adjust is the calibrated camera coordinate system.

Images