A camera extrinsic parameter calibration method for a vehicle-mounted panoramic surround-view system

Abstract

The application relates to a camera external parameter calibration method of a vehicle-mounted panoramic all-around viewing system, which comprises the following steps: S1, arranging a calibration site according to the size of a vehicle, and preparing four ground markers and a ruler; S2, placing a vehicle to be calibrated in a calibration area; S3, randomly placing the ground markers in the ROI of the common viewing area, and measuring the world coordinates of one of the ground markers; S4, detecting the position of the corner point of the ground marker in the ROI area of the fisheye image according to a corner point detection algorithm, and removing distortion; S5, numbering the detected corner points; S6, calibrating the initial external parameters of all the cameras, and calculating the 3d world coordinates of the corner points; S7, constructing a least square equation according to the 3d world coordinates, the corner point coordinates and the projection relationship therebetween, performing closed loop BA optimization, and obtaining the final more accurate external parameters. Therefore, the after-sales calibration precision can be improved by randomly laying the ground markers in the ROI of the common viewing area of the two adjacent cameras.

Description

Technical Field

[0001] This invention relates to the field of calibration technology for driver assistance devices, and in particular to a method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system. Background Technology

[0002] In recent years, with the continuous improvement of economic level, the number of motor vehicles in my country has maintained steady growth. This has led to increasingly intense conflicts between people and vehicles, and between vehicles themselves. For example, parking in narrow and congested urban areas and parking lots has become a prominent problem for drivers, easily resulting in collisions and scrapes. Furthermore, at low speeds, obstructions such as the front of the vehicle, windows, and pillars, coupled with the driver's limited field of vision, create blind spots, leading to frequent accidents and posing a significant safety hazard to pedestrians.

[0003] While conventional rearview cameras, reversing radars, and traditional image-based reversing camera systems (with cameras only installed at the rear of the vehicle) offer some assistance, their coverage is limited and they cannot determine the presence of obstacles in blind spots or the distance between obstacles and the vehicle. To avoid the impact of blind spots on driving safety, four fisheye cameras are installed at the front, rear, and under the left and right rearview mirrors of the car. Adjacent cameras share a certain area of ​​view, introducing an in-vehicle surround-view system. This provides a real-time overview of the vehicle's surroundings, displayed on the center console screen. This allows the driver to clearly see whether there are pedestrians, moving objects, non-motorized vehicles, or obstacles around the vehicle, and to understand their relative positions (turning, parking) and distances, helping the driver to easily park and drive at low speeds.

[0004] To synthesize a panoramic surround-view image, real-time projection, stitching, fusion, and rendering of four fisheye cameras are required. The effectiveness of these steps depends on the intrinsic and extrinsic parameter calibration of the four cameras. Intrinsic parameter calibration mainly determines the refractive power of the optical lenses and the distortion caused by the non-perpendicularity of the lens optical axis to the CMOS / CCD sensor plane during installation. This step can be completed during camera manufacturing. Extrinsic parameter calibration mainly calibrates the camera's installation position and attitude parameters. This process needs to be performed after the cameras are installed in the vehicle, either on the automotive assembly line or at a 4S dealership or other repair shop. Extrinsic parameter calibration performed on the automotive assembly line is called factory calibration, while extrinsic parameter calibration performed at a 4S dealership or other repair shop is called after-sales calibration.

[0005] Chinese patent CN105608693A discloses a calibration system and method for a vehicle-mounted panoramic surround view system, comprising: extracting images containing features from images acquired by the panoramic surround view system, denoted as feature images; extracting useful information about the features from the feature images; optimizing calibration parameters based on the useful information about the features; and verifying the accuracy of the calibration results, wherein the calibration results refer to the optimized calibration parameters. This invention treats the optimization of calibration parameters as an optimization problem, establishing an error function based on multi-camera feature projection errors to indicate the calibration error under the current optimization result, thereby making the calibration more accurate and faster, improving the calibration accuracy of the panoramic surround view system; and it can utilize artificial or non-artificial markers to optimize and correct the calibration, improving the calibration efficiency and applicability of the panoramic surround view system.

[0006] Chinese patent CN107133988A discloses a calibration method and system for a camera in a vehicle-mounted panoramic surround view system. The method includes: laying ground markers in a calibration area and setting an external measuring camera above the calibration area, the external measuring camera being able to capture images of the ground markers, the vehicle roof, and the vehicle body; placing the vehicle to be calibrated, equipped with the vehicle-mounted panoramic surround view system, in the calibration area; determining the transformation relationship between the ground coordinate system and the vehicle body coordinate system based on the external measuring camera; and calibrating the camera parameters relative to the vehicle body coordinate system based on the transformation relationship. Using this invention, calibration efficiency can be improved and the accuracy of the calibration results can be guaranteed.

[0007] Currently, there are some methods for calibrating the extrinsic parameters of cameras in vehicle panoramic surround view systems. However, some of these methods require complex calibration boards and require measuring the world coordinates of each corner point of the calibration object, which is cumbersome and inefficient. Others have low calibration accuracy, which affects the surround view effect. Still others require additional cameras, which reduces their applicability. Summary of the Invention

[0008] To address this, the present invention provides a method for calibrating the extrinsic parameters of cameras in a vehicle-mounted panoramic surround view system. This method effectively solves the technical problem in the prior art where ground markers cannot be arbitrarily placed within the ROI of the shared field of view of two adjacent cameras, and the ideal world coordinates of one of the ground markers cannot be measured. Initial extrinsic parameters are then obtained based on principles such as binocular fisheye calibration and triangulation. Finally, the accurate extrinsic parameters are obtained through closed-loop BA optimization to improve the accuracy of after-sales calibration.

[0009] To achieve the above objectives, the present invention provides a method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system, comprising:

[0010] Step S1: Arrange the marking area according to the size of the vehicle, and prepare 4 ground markers and a ruler;

[0011] Step S2: Place the vehicle to be calibrated within the calibration area;

[0012] Step S3: Randomly place ground markers within the shared ROI and measure the world coordinates of one of the ground markers;

[0013] Step S4: Within the ROI region of the fisheye image, detect the positions of the corner points of ground landmarks using a corner detection algorithm and remove distortion.

[0014] Step S5: Number the detected corner points;

[0015] Step S6: Calibrate the initial extrinsic parameters of all cameras and calculate the 3D world coordinates of the corner points;

[0016] Step S7: Based on the 3D world coordinates, corner coordinates and their projection relationship, construct the least squares equation, perform closed-loop BA optimization, and obtain the final more accurate extrinsic parameters.

[0017] Furthermore, the specific steps of step S3 include:

[0018] Step S301: Determine the ROI corresponding to each image captured by the camera;

[0019] Step S302: Randomly place 4 ground markers within the shared ROI area;

[0020] Step S303: Measure the world coordinates of one of the ground markers.

[0021] Furthermore, the specific steps for numbering the detected corner points in step S5 include:

[0022] Step S501: Assuming that 8 corner points have been detected in the front camera image, first calculate the average coordinate x in the x-direction of all corner points. center According to x center Divide the 8 corner points into 2 groups;

[0023] Step S502: For each group of 4 corner points, calculate the average coordinates x in the x and y directions. center ' and y center ' According to x center ' Divide the four corner points into two groups of two.

[0024] Step S503, then according to y center ' The number of each corner point can be obtained.

[0025] Furthermore, the specific steps of step S6 include:

[0026] Step S601: Traverse the shared viewing areas of two adjacent cameras. There are a total of 4 areas: the upper right corner, the lower right corner, the lower left corner, and the upper left corner.

[0027] Step S602: For corner points within the shared field of view of two adjacent cameras, perform corner point matching based on the corner point numbers;

[0028] Step S603: Calculate the initial extrinsic parameters between two adjacent cameras according to the binocular fisheye camera calibration algorithm;

[0029] Step S604: Calculate the 3d coordinates of the corner points according to the principle of triangulation;

[0030] Step S605: For ground marker M0, calculate the initial extrinsic parameters, i.e., the rotation and translation matrix between the vehicle center and the first set of binocular left cameras, based on the detected corner points and 3D world coordinates.

[0031] Step S606: Based on the calibrated initial extrinsic parameters, transform the 3D coordinates of all corner points to 3D world coordinates.

[0032] Furthermore, in step S602, corner point matching is performed based on the corner point number. For any common viewing area, the corner point matching relationship is 4<->3, 5<->0, 6<->1, 7<->2.

[0033] Further, in step S606, the calibrated initial extrinsic parameters are: T FW 、T RF 、T BR 、T LB and T F'L ;

[0034] The transformation matrix from the world coordinate system to the CamF coordinate system of the front-facing camera is: T FW ;

[0035] The transformation matrix from the world coordinate system to the CamR coordinate system of the right-side camera is: T RW =T RF T FW ;

[0036] The transformation matrix from the world coordinate system to the CamB coordinate system of the rear camera is: T BW =T BR T RFT FW ;

[0037] The transformation matrix from the world coordinate system to the CamL coordinate system of the left-side camera is: T LW =T LB T BR T RF T FW ;

[0038] The transformation matrix from the world coordinate system to the virtual vehicle front camera CamF' coordinate system is: .

[0039] Furthermore, in step S606, when transforming the 3D coordinates of all corner points to 3D world coordinates, let the 3D coordinates of a corner point based on the CamF coordinate system of the vehicle's front camera be... X C The transformation matrix from the world coordinate system to the CamF coordinate system of the front-facing camera is: T FW The world coordinates of this corner point X W :

[0040]

[0041] Summarized as follows:

[0042] ;

[0043] In the formula, R represents the rotation matrix and T represents the translation matrix.

[0044] Furthermore, in step S7, the specific formula for constructing the least squares equation is as follows:

[0045] ;

[0046] In the formula, N i For the first i The number of corner points of the chessboard observed by each camera u ij Indicates the first j The corner point is projected onto the first... i Pixel coordinates of each camera s ij K represents the camera's intrinsic parameters, corresponding to the corner scale factor. Indicates the first i The pose of the camera, 𝑃 ij In the first i Within the field of view of the cameraj The world coordinates of each corner point.

[0047] Furthermore, the world coordinates of a certain corner point in the space are: The pixel coordinates of the corner projection are The rotation and translation matrix between the world coordinate system and the camera coordinate system is: r i , t i That is, the pose of the camera is The camera's intrinsic parameter is K, and the relationship between pixel position and spatial point is as follows:

[0048] ;

[0049] Right now:

[0050] .

[0051] Compared with existing technologies, the advantages of this invention are as follows: This invention arbitrarily places ground markers within the Region of Interest (ROI) of two adjacent cameras, measures the ideal world coordinates of one of the ground markers (with the vehicle center as the coordinate system), obtains initial extrinsic parameters based on principles such as binocular fisheye calibration and triangulation, and then obtains the final accurate extrinsic parameters through closed-loop basis optimization (BA). This method simplifies the calibration process, is easy to operate, improves calibration accuracy, and is beneficial for after-sales calibration in various environments.

[0052] Furthermore, this invention determines the ROI corresponding to each camera's captured image and then randomly places four ground markers within the shared ROI of the common field of view. Finally, it measures the world coordinates of one of the ground markers. Since the position of the placed ground markers does not change significantly, the ROI can be set. This not only improves the detection speed of corner points but also reduces false detections of corner points. Attached Figure Description

[0053] Figure 1 This is a flowchart of the camera extrinsic parameter calibration method for the vehicle-mounted panoramic surround view system according to an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of a ground marker in an embodiment of the present invention;

[0055] Figure 3 The ground markers are arranged in the field according to an embodiment of the present invention;

[0056] Figure 4 This refers to the ROI (Region of Interest) corresponding to the front-facing camera image in this embodiment of the invention.

[0057] Figure 5This is a comparison image of corner points in a fisheye image and a corrected image according to an embodiment of the present invention;

[0058] Figure 6 This is a comparison chart of the corner detection accuracy in fisheye images and corrected images implemented in this invention;

[0059] Figure 7 This is a schematic diagram of corner point numbering in an embodiment of the present invention;

[0060] Figure 8 This is a flowchart illustrating the process of calibrating the initial extrinsic parameters of all cameras and calculating 3D world coordinates in an embodiment of the present invention.

[0061] Figure 9 This is a schematic diagram of corner matching in an embodiment of the present invention;

[0062] Figure 10 This is a schematic diagram of closed-loop optimization in an embodiment of the present invention. Detailed Implementation

[0063] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0064] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0065] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0066] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0067] Please see Figure 1The diagram shown is a flowchart illustrating the camera extrinsic parameter calibration method for an in-vehicle panoramic surround view system according to an embodiment of the present invention. The present invention provides a camera extrinsic parameter calibration method for an in-vehicle panoramic surround view system, comprising:

[0068] Step S1: Arrange the marking area according to the size of the vehicle, and prepare 4 ground markers and a ruler;

[0069] In this embodiment, ground markers include, but are not limited to, checkerboard-type objects. Any element or combination of elements of any size with a rectangular structure is acceptable, such as A4 or A0 size sheets of paper, as long as they can effectively detect the corner points of the ground markers. Figure 2 This is a schematic diagram of a ground marker. The marking site should be as flat as possible to accommodate the ground marker and vehicles.

[0070] Step S2: Place the vehicle to be calibrated within the calibration area;

[0071] Step S3: Randomly place ground markers within the shared ROI and measure the world coordinates of one of the ground markers;

[0072] In this embodiment, one of the measured ground markers is denoted as MO, and the world coordinates are measured using the vehicle's center as the coordinate system. For example... Figure 3 As shown, this is the site for the placement of ground markers in an embodiment of the present invention. The ground marker in the upper right corner is M0, and the ground markers in the lower right, lower left, and upper left corners are M1, M2, and M3, respectively. ROI is as follows: Figure 4 The white area in the middle Figure 4 This diagram shows the ROI for corner detection corresponding to the image from the front-facing camera in this embodiment of the invention. The camera positions in the vehicle surround-view system are fixed, and the positions of the ground markers placed therein do not change significantly, so ROI can be set; this not only improves the corner detection speed but also reduces false detections of corner points.

[0073] Step S4: Within the ROI region of the fisheye image, detect the positions of the corner points of ground landmarks using a corner detection algorithm and remove distortion.

[0074] In this embodiment, the corner detection algorithm can be Fast, ORB, Harris, or other algorithms.

[0075] The corner detection algorithm is based on fisheye images for detection, from... Figure 5 It can be observed that Figure 5This is a comparison image of corner points in a fisheye image and a corrected image according to an embodiment of the present invention. At the same corner point within the red rectangle, after fisheye correction, due to the stretching, the corner point is farther from the black squares on both sides, meaning the constraint on the corner point is looser. This not only affects the accuracy of the corner point but also increases the chance of missed detections. Figure 6 It can be seen that, Figure 6 This is a comparison of the corner detection accuracy in fisheye images and corrected images in the implementation of this invention. Corner point 3 and corner point 6 are more accurately detected in the fisheye image (corner point 3 detected in the corrected image is biased towards the upper right corner of the true corner point position, while corner point 6 is biased towards the lower right corner of the true corner point position).

[0076] Step S5: Number the detected corner points;

[0077] like Figure 7 As shown, Figure 7 This is a schematic diagram of corner point numbering in an embodiment of the present invention. The upper diagram shows the corner point numbering of the front camera image, and the lower diagram shows the corner point numbering of the right camera image.

[0078] Step S6: Calibrate the initial extrinsic parameters of all cameras and calculate the 3D world coordinates of the corner points;

[0079] Step S7: Based on the 3D world coordinates, corner coordinates and their projection relationship, construct the least squares equation, perform closed-loop BA optimization, and obtain the final more accurate extrinsic parameters.

[0080] Specifically, this invention involves randomly placing ground markers within the Region of Interest (ROI) of two adjacent cameras and measuring the ideal world coordinates (with the vehicle's center as the coordinate system) of one of the ground markers. Initial extrinsic parameters are obtained based on principles such as binocular fisheye calibration and triangulation. The final, precise extrinsic parameters are then obtained through closed-loop basis optimization (BA). This method simplifies the calibration process, is easy to operate, and improves calibration accuracy, making it beneficial for after-sales calibration in various environments.

[0081] The in-vehicle panoramic surround view system provided in this embodiment consists of four cameras installed at the front, rear, and under the left and right rearview mirrors of the vehicle. The camera parameters include intrinsic parameters (such as focal length, image center, and distortion parameters) and extrinsic parameters (including the rotation and translation matrices of the camera coordinate system relative to other coordinate systems). This invention primarily addresses the calibration of camera extrinsic parameters. Before after-sales calibration, information such as the camera's intrinsic parameters and the ROI area where ground markers are placed has already been obtained. The world coordinate system described in this embodiment is as follows: the projection of the vehicle's center onto the ground is the origin, the x-axis is to the right, the y-axis is forward (towards the front of the vehicle), and the z-axis is upward (towards the roof of the vehicle).

[0082] Specifically, step S3 includes the following steps:

[0083] Step S301: Determine the ROI corresponding to each image captured by the camera;

[0084] Step S302: Randomly place 4 ground markers within the shared ROI area;

[0085] Step S303: Measure the world coordinates of one of the ground markers.

[0086] Specifically, this invention determines the ROI corresponding to each camera's captured image and then randomly places four ground markers within the shared ROI of the field of view. Finally, it measures the world coordinates of one of the ground markers. Since the position of the placed ground markers does not change significantly, the ROI can be set. This not only improves the detection speed of corner points but also reduces false detections of corner points.

[0087] Specifically, the steps for numbering the detected corner points in step S5 include:

[0088] Step S501: Assuming that 8 corner points have been detected in the front camera image, first calculate the average coordinate x in the x-direction of all corner points. center According to x center Divide the 8 corner points into 2 groups;

[0089] In this embodiment, the x-direction of the corner point represents the horizontal direction.

[0090] Step S502: For each group of 4 corner points, calculate the average coordinates x in the x and y directions. center ' and y center ' According to x center ' Divide the four corner points into two groups of two.

[0091] Step S503, then according to y center ' The number of each corner point can be obtained.

[0092] Specifically, such as Figure 8 As shown, this is a flowchart of calibrating the initial extrinsic parameters of all cameras and calculating the 3D world coordinates in an embodiment of the present invention. The specific steps of step S6 include:

[0093] Step S601: Traverse the shared viewing areas of two adjacent cameras. There are a total of 4 areas: the upper right corner, the lower right corner, the lower left corner, and the upper left corner.

[0094] Step S602: For corner points within the shared field of view of two adjacent cameras, perform corner point matching based on the corner point numbers;

[0095] In this embodiment, the corner matching method matches based on the number, such as... Figure 9 As shown, Figure 9 This is a schematic diagram of corner point matching in an embodiment of the present invention. The horizontal part of the image represents the corner point number of the front camera, and the vertical part represents the corner point number of the right camera. The matching relationship is: 4<->3, 5<->0, 6<->1, 7<->2.

[0096] Step S603: Calculate the initial extrinsic parameters between two adjacent cameras according to the binocular fisheye camera calibration algorithm;

[0097] Step S604: Calculate the 3d coordinates of the corner points according to the principle of triangulation;

[0098] In this embodiment, the 3D coordinates of the corner point are based on the first camera of the two adjacent cameras.

[0099] Step S605: For ground marker M0, the initial extrinsic parameters are calculated using the PNP algorithm based on the detected corner points and 3D world coordinates. This is the rotation and translation matrix between the vehicle center and the first set of binocular left cameras. In this embodiment, the left camera is CamF.

[0100] Step S606: Based on the calibrated initial extrinsic parameters, transform the 3D coordinates of all corner points to 3D world coordinates.

[0101] In this embodiment, the calibrated initial extrinsic parameters are: T FW 、T RF 、T BR 、T LB and T F'L .

[0102] Specifically, in step S602, corner point matching is performed based on the corner point number. For any common viewing area, the corner point matching relationship is 4<->3, 5<->0, 6<->1, 7<->2.

[0103] Specifically, in step S606, the calibrated initial extrinsic parameters are: T FW 、T RF 、T BR 、T LB and T F'L ;

[0104] The transformation matrix from the world coordinate system to the CamF coordinate system of the front-facing camera is: T FW ;

[0105] The transformation matrix from the world coordinate system to the CamR coordinate system of the right-side camera is: T RW =T RF T FW ;

[0106] The transformation matrix from the world coordinate system to the CamB coordinate system of the rear camera is: T BW =T BR T RF T FW ;

[0107] The transformation matrix from the world coordinate system to the CamL coordinate system of the left-side camera is: T LW =T LB T BR T RF T FW .

[0108] Specifically, in step S606, when transforming the 3D coordinates of all corner points to 3D world coordinates, let the 3D coordinates of a corner point based on the CamF coordinate system of the front-facing camera be... X C The transformation matrix from the world coordinate system to the CamF coordinate system of the front-facing camera is: T FW The world coordinates of this corner point X W :

[0109]

[0110] Summarized as follows:

[0111] ;

[0112] In the formula, R represents the rotation matrix and T represents the translation matrix.

[0113] Specifically, in step S7, the specific formula for constructing the least squares equation is as follows:

[0114] ;

[0115] In the formula, N i For the first i The number of corner points of the chessboard observed by each camera uij Indicates the first j The corner point is projected onto the first... i Pixel coordinates of each camera s ij K represents the camera's intrinsic parameters, corresponding to the corner scale factor. Indicates the first i The pose of the camera, 𝑃 ij In the first i Within the field of view of the camera j The world coordinates of each corner point.

[0116] Specifically, the world coordinates of a corner point in the space are: The pixel coordinates of the corner projection are The rotation and translation matrix between the world coordinate system and the camera coordinate system is: r i , t i That is, the pose of the camera is The camera's intrinsic parameter is K, and the relationship between pixel position and spatial point is as follows:

[0117] ;

[0118] Right now:

[0119] .

[0120] In this embodiment, as Figure 10 As shown, this is a schematic diagram of closed-loop optimization in an embodiment of the present invention. In step S7, closed-loop BA optimization is performed. Based on the binocular fisheye calibration formed by adjacent cameras, a closed loop is formed. Ideally... However, there will be discrepancies in reality. For a virtual front-facing camera, a least-squares equation is constructed for the five cameras based on the reprojection error. For a point in space with world coordinates of... The pixel coordinates of the corner projection are The rotation and translation matrix between the world coordinate system and the camera coordinate system is: That is, the pose of the camera is The camera's intrinsic parameter is K, and the relationship between pixel position and spatial point is as follows:

[0121] ;

[0122] Right now:

[0123] ;

[0124] Since the camera pose and the 3D world coordinates of the corners only have initial values ​​and are subject to noise, this equation contains an error. Therefore,

[0125] We sum the errors, construct a least-squares problem, and then find the optimal camera pose and corner 3D world coordinates to minimize it:

[0126] The preferred embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system, characterized in that, include: Step S1: Arrange the marking area according to the size of the vehicle, and prepare 4 ground markers and a ruler; Step S2: Place the vehicle to be calibrated within the calibration area; Step S3: Randomly place ground markers within the shared ROI and measure the world coordinates of one of the ground markers M0; Step S4: Within the ROI region of the fisheye image, detect the positions of the corner points of ground landmarks using a corner detection algorithm and remove distortion. Step S5: Number the detected corner points; Step S6: Calibrate the initial extrinsic parameters of all cameras and calculate the 3D world coordinates of the corner points; Step S7: Based on the 3D world coordinates, corner coordinates and their projection relationship, construct the least squares equation, perform closed-loop BA optimization, and obtain the final more accurate extrinsic parameters; The specific steps of step S3 include: Step S301: Determine the ROI corresponding to each image captured by the camera; Step S302: Randomly place 4 ground markers within the shared ROI area; Step S303: Measure the world coordinates of one of the ground markers; The specific steps for numbering the detected corner points in step S5 include: Step S501: Assuming that 8 corner points have been detected in the front camera image, first calculate the average coordinate x in the x-direction of all corner points. center According to x center Divide the 8 corner points into 2 groups; Step S502, for each group of 4 corner points, calculate direction and Average coordinates of direction x center ' and y center ' According to x center ' Divide the four corner points into two groups of two. Step S503, then according to y center ' The number of each corner point can be obtained; The specific steps of step S6 include: Step S601: Traverse the shared viewing areas of two adjacent cameras. There are a total of 4 areas: the upper right corner, the lower right corner, the lower left corner, and the upper left corner. Step S602: For corner points within the shared field of view of two adjacent cameras, perform corner point matching based on the corner point numbers; Step S603: Calculate the initial extrinsic parameters between two adjacent cameras according to the binocular fisheye camera calibration algorithm; Step S604: Calculate the 3d coordinates of the corner points according to the principle of triangulation; Step S605: For ground marker M0, calculate the initial extrinsic parameters, i.e., the rotation and translation matrix between the vehicle center and the first set of binocular left cameras, based on the detected corner points and 3D world coordinates. Step S606: Based on the calibrated initial extrinsic parameters, transform the 3D coordinates of all corner points to 3D world coordinates.

2. The method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system according to claim 1, characterized in that, In step S602, corner point matching is performed based on the corner point number. For any common viewing area, the corner point matching relationship is 4<->3, 5<->0, 6<->1, 7<->2.

3. The method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system according to claim 1, characterized in that, In step S606, the calibrated initial extrinsic parameters are: T FW 、T RF 、T BR 、T LB and T F'L ; The transformation matrix from the world coordinate system to the CamF coordinate system of the front-facing camera is: T FW ; The transformation matrix from the world coordinate system to the CamR coordinate system of the right-side camera is: T RW =T RF T FW ; The transformation matrix from the world coordinate system to the CamB coordinate system of the rear camera is: T BW =T BR T RF T FW ; The transformation matrix from the world coordinate system to the CamL coordinate system of the left-side camera is: T LW =T LB T BR T RF T FW ; The transformation matrix from the world coordinate system to the virtual vehicle front camera CamF' coordinate system is: 。 4. The method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system according to claim 1, characterized in that, In step S606, when transforming the 3D coordinates of all corner points to 3D world coordinates, let the 3D coordinates of a corner point based on the CamF coordinate system of the vehicle's front camera be... X C The transformation matrix from the world coordinate system to the CamF coordinate system of the front-facing camera is: T FW The world coordinates of this corner point X W : Summarized as follows: ； In the formula, R represents the rotation matrix and T represents the translation matrix.

5. The method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system according to claim 1, characterized in that, In step S7, the specific formula for constructing the least squares equation is as follows: ； In the formula, N i For the first i The number of corner points of the chessboard observed by each camera u ij Indicates the first j The corner point is projected onto the first... i Pixel coordinates of each camera s ij K represents the camera's intrinsic parameters, corresponding to the corner scale factor. Indicates the first i The pose of the camera, 𝑃 ij In the first i Within the field of view of the camera j The world coordinates of each corner point.

6. The method for calibrating the extrinsic parameters of a camera in a vehicle-mounted panoramic surround view system according to claim 5, characterized in that, The world coordinates of a certain corner point in the space are: The pixel coordinates of the corner projection are The rotation and translation matrix between the world coordinate system and the camera coordinate system is: r i ,t i That is, the pose of the camera is The camera's intrinsic parameter is K, and the relationship between pixel position and spatial point is as follows: ； Right now: 。

Images