Automobile panoramic fisheye external parameter calibration method

Abstract

The invention discloses an automobile panoramic fisheye external parameter calibration and calibration method based on ground feature points. The method comprises the following steps: arranging flat calibration plates in front left, front right, rear right and rear left areas of an automobile, and collecting four paths of fisheye camera images; a user manually selects calibration plate angular points in the overlapped view field of the adjacent cameras and records pixel coordinates; back-projecting the pixel coordinates to world ground coordinates in combination with internal and external parameters of the camera; constructing an error function based on the multi-path ground coordinates of the same physical angular point, and iteratively optimizing the external parameters of each camera through nonlinear least squares until convergence; and finally generating a seamless aerial view and verifying a calibration effect. The method does not need a professional site or measuring equipment, is simple and convenient to operate, has high precision and strong robustness, and is suitable for on-site rapid calibration.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and vehicle surround view systems, and more particularly to a method for calibrating the extrinsic parameters of automotive fisheye cameras using ground feature points. This method is suitable for correcting extrinsic parameter offsets caused by changes in the operating environment after factory calibration, as well as for recalibrating extrinsic parameters after camera replacement or accidents. Background Art

[0002] Currently, automotive surround view systems primarily utilize four fisheye cameras, capturing images of the vehicle's surroundings and stitching them together to create a bird's-eye view, providing the driver with a 360-degree field of view to assist with driving and parking. Existing technology typically involves precise calibration of each camera at the factory to ensure accurate extrinsic parameters. However, due to the long-term effects of road vibrations, collisions, and camera repairs and replacements, the relative mounting positions of the fisheye cameras and the vehicle are susceptible to slight offsets. This can cause changes in the extrinsic parameters and lead to errors in the stitched image.

[0003] Furthermore, traditional calibration methods often rely on specialized sites and equipment, making them unsuitable for the rapid, on-site calibration required in 4S dealerships and other after-sales service environments, increasing usage and maintenance costs. Consequently, a robust and user-friendly external calibration method is urgently needed to achieve real-time correction under various operating conditions and ensure high-precision display for panoramic imaging systems. Summary of the Invention

[0004] The technical problem to be solved by the present invention is that the existing external parameter calibration method of the automobile panoramic surround view system usually relies on professional calibration sites, dedicated measurement equipment and manual measurement. The calibration process is complicated and inefficient, and it is difficult to meet the fast, simple and high-precision calibration requirements on the after-sales site.

[0005] In order to solve the above problems, the present invention provides a method for calibrating extrinsic parameters of a panoramic fisheye image of an automobile, comprising the following steps:

[0006] S1. Place calibration plates with clearly visible calibration corners in the front left, front right, rear right, and rear left areas of a horizontally stationary vehicle, respectively, and activate the onboard surround view system so that images captured by the four fisheye cameras (front, left, right, and rear) are displayed in real time on the same display screen.

[0007] S2. The user selects the calibration corner points in the image through an input device. Each calibration corner point is selected in two adjacent images to establish a cross-machine correspondence relationship, and the pixel coordinates of the calibration corner points are recorded.

[0008] S3. Convert the pixel coordinates into normalized coordinates in the camera coordinate system based on the pixel coordinates and the intrinsic parameters of the corresponding camera, and convert the normalized coordinates into ground coordinates in the world coordinate system in combination with the corresponding extrinsic parameters; wherein each of the calibration corner points obtains a ground coordinate from two cameras respectively;

[0009] S4. Calculate the deviation of the ground coordinates of each calibrated corner point obtained by the two cameras, and summarize all the deviations to construct an error function. With the goal of minimizing the error function, iteratively fine-tune the extrinsic parameters of each camera until the error function converges.

[0010] S5. Solidify the external parameters obtained after the iterative optimization and load them into the panoramic surround view system to regenerate a seamless bird's-eye view of the vehicle.

[0011] The beneficial effect of the present invention is that the present invention can complete high-precision calibration of the external parameters of multiple fisheye cameras by simply laying out a calibration plate and selecting corner points in the vehicle-mounted surround view interface. No professional site or measurement equipment is required, the operation is fast and robust, and the stitching effect can be verified in real time in a bird's-eye view, which not only reduces costs but also meets the needs of on-site after-sales rapid calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention will be further described below with reference to the accompanying drawings and examples.

[0013] Figure 1 It is a schematic diagram of a top view of a vehicle and an arrangement of calibration plates in an embodiment of the automobile panoramic fisheye extrinsic parameter calibration method of the present invention.

[0014] Figure 2 The present invention is a flow chart of an embodiment of a method for calibrating extrinsic parameters of a panoramic fisheye car. DETAILED DESCRIPTION

[0015] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0016] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0017] like Figure 1-2 As shown, the present invention provides an embodiment of a method for calibrating extrinsic parameters of a panoramic fisheye image of an automobile. In this embodiment, the hardware constituting the vehicle surround view system includes:

[0018] Fisheye cameras: Four fisheye cameras are located at the front (F), left (L), right (R), and rear (B) positions of the vehicle;

[0019] processor, running a real-time operating system;

[0020] Human-computer interaction interface: Users can view images collected by four fisheye cameras through the human-computer interaction interface;

[0021] Storage and network module: used to save internal parameters, external parameters and calibration results.

[0022] The method of this embodiment includes the following steps:

[0023] S1. Place calibration plates with clearly visible calibration corners in the front left, front right, rear right, and rear left areas of a horizontally stationary vehicle. Activate the onboard surround view system, displaying the real-time images captured by the four fisheye cameras for the front, left, right, and rear views on a single display screen. In this embodiment, the calibration plates are white A4 paper. However, the material and form of the calibration plates are not limited to ordinary A4 paper. Any flat material with distinct corners can be used as an alternative, such as: white A4 paper or matte paper with a non-reflective finish; plastic floor mats / films pre-printed with a high-contrast checkerboard or zebra pattern; black and white grids attached to PVC, acrylic, or rigid plastic; weather-resistant cardboard or non-wrinkle composite cardboard. Regardless of the material used, the following requirements must be met: a flat surface with no creases, wrinkles, or warping; clear boundaries and high contrast for the calibration corners, ensuring reliable detection in the fisheye camera image; and avoid using highly reflective or transparent materials to prevent light spots and light transmission interference. Through the above-mentioned multiple optional solutions, the present invention can flexibly select the most suitable calibration plate under different field working conditions, thereby ensuring the robustness and accuracy of the calibration process.

[0024] In this example, three calibration plates are arranged in a straight line at intervals of 0.5 m, 0.8 m, and 1.2 m along the longitudinal direction of the vehicle. In other embodiments of the present invention, the number of calibration plates can be selected according to actual needs (e.g., 3 to 6 plates), and the distances d1, d2, ..., d between each calibration plate and the vehicle reference point (e.g., the center of the vehicle body) are N

[0025] It is not limited to a fixed value and can be arbitrarily selected within the range of 0.2m to 2.0m according to the vehicle size, camera resolution and on-site spatial conditions to ensure sufficient overlapping fields of view and geometric constraints at different depths.

[0026] S2. The user selects at least 10 calibration corner points in the image displayed on the human-computer interaction interface. Each calibration corner point is selected in two adjacent images to establish a cross-machine correspondence relationship. The kth calibration corner point selected by the user in the i-th image is expressed as a two-dimensional pixel coordinate vector:

[0027] p {i,k} =(u {i,k} ,v {i,k} ) T ,

[0028] And expand to homogeneous coordinates:

[0029]

[0030] For any calibration corner point k in the overlapping area of ​​the field of view of the i-th and j-th cameras, its homogeneous coordinates are recorded in both images. and

[0031] S3, the homogeneous coordinates of the kth calibration corner point of the i-th camera Convert to normalized coordinates:

[0032]

[0033] Among them, K i is the intrinsic parameter matrix of the i-th camera, in the form of:

[0034]

[0035] And transform the normalized coordinates of the calibration corner points into the ground coordinates in the world coordinate system:

[0036]

[0037] in, is the inverse matrix of the extrinsic parameter matrix of the i-th camera. The form of the extrinsic parameter matrix is:

[0038] R i With T i are the rotation matrix and translation vector of the i-th camera respectively;

[0039] S4. For any pair of adjacent cameras i and j, the same calibration corner point k in the overlapping area of ​​their field of view is marked by the world coordinates obtained by the two cameras respectively. and Compute their residual vectors:

[0040]

[0041] Summarize the residual vectors of all adjacent camera pairs (i, j) and all calibrated corner points k to construct the error function:

[0042] Among them, N i,j is the number of calibrated corner points in the overlap region between camera i and camera j, and ||·||2 represents the Euclidean norm, i.e., the geometric distance between two points. With the goal of minimizing the error function, the extrinsic parameters of each camera are iteratively fine-tuned until the error function converges.

[0043] S5. After the optimization converges, all the latest extrinsic parameters of the cameras are written into the calibration configuration file and stored in the non-volatile memory. The bird's-eye view generation module is called to read the four dedistorted images in sequence, back-project them onto the world plane according to the solidified extrinsic parameters, map the plane coordinates to BEV image coordinates according to the established pixel mapping ratio, and use the weighted fusion algorithm to slide the window to smooth the stitching boundary.

[0044] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0045] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A method for calibrating extrinsic parameters of a panoramic fisheye image of an automobile, characterized in that: The following steps are involved: S1. Place calibration plates with clearly visible calibration corners in the front left, front right, rear right, and rear left areas of a horizontally stationary vehicle, respectively, and activate the onboard surround view system so that images captured by the four fisheye cameras (front, left, right, and rear) are displayed in real time on the same display screen. S2. The user selects the calibration corner points in the image through an input device. Each calibration corner point is selected in two adjacent images to establish a cross-machine correspondence relationship, and the pixel coordinates of the calibration corner points are recorded. S3. Convert the pixel coordinates into normalized coordinates in the camera coordinate system based on the pixel coordinates and the intrinsic parameters of the corresponding camera, and convert the normalized coordinates into ground coordinates in the world coordinate system in combination with the corresponding extrinsic parameters; wherein each of the calibration corner points obtains a ground coordinate from two cameras respectively; S4. Calculate the deviation of the ground coordinates of each calibrated corner point obtained by the two cameras, and summarize all the deviations to construct an error function. With the goal of minimizing the error function, iteratively fine-tune the extrinsic parameters of each camera until the error function converges. S5. Solidify the external parameters obtained after the iterative optimization and load them into the panoramic surround view system to regenerate a seamless bird's-eye view of the vehicle.

2. The method for calibrating the external parameters of a panoramic fisheye image of an automobile according to claim 1, wherein: In step S2, the kth calibration corner point selected by the user in the i-th image is represented as a two-dimensional pixel coordinate vector: p {i,k} =(u {i,k} ,v {i,k} ) T , And expand to homogeneous coordinates: For any calibration corner point k in the overlapping area of ​​the field of view of the i-th and j-th cameras, its homogeneous coordinates should be recorded in both images. and 3. The method for calibrating the external parameters of a panoramic fisheye image of an automobile according to claim 2, wherein: In step S3, the homogeneous coordinates of the kth calibrated corner point of the i-th camera are Convert to normalized coordinates: Among them, K i is the intrinsic parameter matrix of the i-th camera, in the form of: And convert the normalized coordinates of the calibration corner points to the ground coordinates in the world coordinate system: in, is the inverse matrix of the extrinsic parameter matrix of the i-th camera. The form of the extrinsic parameter matrix is: R i With T i are the rotation matrix and translation vector of the i-th camera respectively.

4. The method for calibrating the external parameters of a panoramic fisheye image of an automobile according to claim 3, wherein: In step S4, the specific process of constructing the error function and optimizing the external parameters includes: For any pair of adjacent cameras i and j with the same calibration corner point k in the overlapping area of ​​their field of view, the world coordinates obtained by the two cameras are respectively and Compute their residual vectors: Summarize the residual vectors of all adjacent camera pairs (i, j) and all calibrated corner points k to construct the error function: Among them, N i,j is the number of all calibrated corner points in the overlapping area of ​​camera i and camera j, and ‖·‖2 represents the Euclidean norm, that is, the geometric distance between two points.

5. The method for calibrating extrinsic parameters of a panoramic fisheye image of an automobile according to any one of claims 1 to 4, characterized in that: In step S1, three calibration plates are arranged in a straight line at intervals of 0.5 meters, 0.8 meters and 1.2 meters respectively in the front left area, front right area, rear right area and rear left area of ​​the vehicle in a horizontally stationary state along the longitudinal direction of the vehicle.

6. The method for calibrating extrinsic parameters of a panoramic fisheye image of an automobile according to any one of claims 1 to 4, characterized in that: The number of selected calibration corner points is at least 10.

Images