Vehicle-mounted around-view camera external parameter determination method and system

Abstract

The invention relates to the technical field of computer vision, in particular to a method and a system for determining external parameters of a vehicle-mounted around-view camera. The method comprises the following steps: S1, controlling each vehicle-mounted all-around camera to shoot a vehicle-mounted all-around camera image with a marker; the markers are respectively placed on the front, back, left and right sides of the vehicle; s2, performing marker detection on each auxiliary image containing a plurality of markers, and constructing a global coordinate system based on relative poses among the markers; the auxiliary image is shot by a distortionless lens camera; s3, based on a PNP algorithm and pre-calibrated internal parameters of the vehicle-mounted all-round view cameras, calculating position coordinates of the vehicle-mounted all-round view cameras in the global coordinate system; s4, a vehicle body coordinate system with the center point of the vehicle as the original point is established based on the position coordinates; and S5, establishing a transformation relation between the global coordinate system and the vehicle body coordinate system. The method can realize accurate calibration of the external parameters of the vehicle-mounted all-round camera under the conditions that the vehicle is parked at will, the marker is placed at will, and common view of adjacent cameras is not needed.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a method and system for determining the extrinsic parameters of an in-vehicle surround-view camera. Background Technology

[0002] The vehicle surround view system captures images using fisheye cameras installed at the front, rear, left, and right of the vehicle, and stitches these images together to create a bird's-eye view, providing the driver with a global view of the vehicle's surroundings. The key to achieving seamless stitching lies in obtaining accurate camera extrinsic parameters, that is, the transformation relationship between the coordinate systems of each camera and the vehicle coordinate system.

[0003] Existing calibration methods include two main approaches. The first is the fixed-position method, which requires the vehicle to be precisely placed in a pre-defined location, and a calibration cloth with specific markings (such as a checkerboard pattern or dots) to be accurately placed in fixed positions around the vehicle. This method demands high skill levels from the operator and has poor flexibility, making it difficult to deploy in space-constrained environments such as 4S stores and repair shops. The second approach is the camera-sharing method, such as the "adjacent cameras acquiring extrinsic parameters from the same marker" scheme proposed in Chinese patent (publication number CN110246184B). This scheme relaxes the requirements for vehicle parking location but mandates that adjacent cameras (such as the left and front cameras) must simultaneously capture the same marker, meaning the marker needs to be placed within the overlapping field of view of adjacent cameras. The relative pose between the cameras is calculated by solving for their observations of the shared marker.

[0004] Existing technologies suffer from three main drawbacks. First, they lack flexibility. The fixed-position method relies on precise physical positioning, and any placement deviation will directly introduce calibration errors, leading to ghosting or misalignment in the stitched images. Second, their applicability is limited. Some solutions rely on 3D markers, which involve complex setups, high costs, and high operational barriers, making them unsuitable for on-site calibration by non-professionals. Third, their calibration accuracy is low. In adjacent camera sharing, markers need to be placed in the overlapping areas of the adjacent cameras' fields of view. However, this area is often located at the camera's edge. Fisheye cameras are prone to vignetting, chromatic aberration, and blurring at the edge, resulting in severe lens distortion that is difficult to correct, leading to low feature point detection accuracy and poor stability in extrinsic parameter calculation. Summary of the Invention

[0005] The present invention aims to provide a method and system for determining the extrinsic parameters of a vehicle-mounted surround-view camera, which can accurately calibrate the extrinsic parameters of the vehicle-mounted surround-view camera when the vehicle is parked arbitrarily, the markers are placed arbitrarily, and there is no need for adjacent cameras to share the same view.

[0006] To address the aforementioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for determining the extrinsic parameters of an in-vehicle surround-view camera, comprising the following steps: S1: Control each vehicle surround view camera to capture images of the vehicle surround view camera with a marker; the markers are placed at the front, rear, left and right sides of the vehicle respectively; S2: Perform marker detection on each auxiliary image containing multiple markers, and construct a global coordinate system based on the relative poses between the markers; the auxiliary images are captured by a distortion-free lens camera; S3: Based on the PNP algorithm and the pre-calibrated intrinsic parameters of the vehicle surround view camera, calculate the position coordinates of each vehicle surround view camera in the global coordinate system; S4: Establish a vehicle coordinate system with the vehicle center point as the origin based on the coordinates of each position; S5: Establish the transformation relationship between the global coordinate system and the vehicle body coordinate system to determine the extrinsic parameters of the vehicle surround view camera and perform image stitching.

[0007] By adopting the above technical solution, when determining the extrinsic parameters of the vehicle-mounted surround-view camera, the vehicle is first parked between four markers, and the vehicle can be parked arbitrarily, as can the markers. Then, multiple auxiliary images are captured using a distortion-free camera to calculate the relative poses between the markers. Next, the transformation relationship between each marker and the master marker coordinate system (global coordinate system) is determined, i.e., a global coordinate system is constructed, transforming each marker to the global coordinate system. Then, images are captured by each vehicle-mounted surround-view camera to determine the position coordinates of each camera in the global coordinate system. Finally, a vehicle body coordinate system is established based on these position coordinates, with the vehicle's center point as the origin. The transformation relationship between the global coordinate system and the vehicle body coordinate system is then established. After this transformation relationship is established, the corresponding extrinsic parameters are obtained, thus enabling the stitching of the vehicle-mounted surround-view view. Because the vehicle can be parked arbitrarily, the markers can be placed arbitrarily, and adjacent cameras do not need to share a view, this method offers higher calibration accuracy and flexibility.

[0008] Optionally, the markers are four ArUco markers of known size.

[0009] Optionally, the auxiliary images shall be at least four, and each image shall contain at least two or more markers.

[0010] Optionally, S2 includes: S2-1: Calculate the size factor for each marker. : in, Representative markers The pixel coordinates of a corner point on the x-axis of the image coordinate system; This represents the pixel coordinate difference of the diagonal point on the y-axis; the marker with the largest size factor in each auxiliary image is selected as the primary marker, and its corresponding local 3D coordinate system is... Tie; S2-2: Construct a local coordinate system for the main marker based on the pixel coordinates of its corner points and its known actual size. The projection transformation matrix to the image pixel coordinate system backprojects the corner pixel coordinates of other markers in the auxiliary image to the image pixel coordinate system. The system was used to obtain the markers in the system. The three-dimensional coordinates are used in the system; for each pair of markers in the auxiliary image, the relative transformation matrix is ​​calculated. , Represents a non-primary identifier ID; Represents the primary identifier ID; S2-3: Define confidence weights : Retain the relative transformation matrix with the highest confidence weight; S2-4: with Using the local 3D coordinate system as the global coordinate system, the markers are transformed using a relative transformation matrix. , , The coordinates are transformed sequentially to Tie: in, Representative markers Switch to Global coordinates after the system; Representative markers Coordinates in its own local coordinate system ; Representative markers To marker The relative transformation matrix.

[0011] Optionally, S3 includes: For each vehicle-mounted surround-view camera image acquired, the PNP algorithm, combined with pre-calibrated intrinsic parameters of the vehicle-mounted surround-view camera, is used to solve the coordinate system of the vehicle-mounted surround-view camera to... The first transformation matrix of the system : in, represent 3. Rotation matrix; represent 1. Translation vector ( ); 0 represents 3. Zero vector; Based on the first transformation matrix, the derivation of the vehicle-mounted surround view camera... Position coordinates in the system : in, Represents a rotation matrix The transpose of the transpose yields the final result. exist The position coordinates in the system.

[0012] Optionally, S4 includes: Based on four vehicle surround view cameras The positional relationship within the system, calculating the vehicle's center point. Coordinates in the system Establish a vehicle coordinate system with the vehicle center point as the origin. Solve for the vehicle body coordinate system Compared to Yaw angle of the system ,get Tie The second transformation matrix of the system : in, , , The center point of the vehicle is at The coordinates of the x, y, and z axes in the system.

[0013] Optionally, S5 includes: Combining the first transformation matrix and the second transformation matrix, we can obtain the transformation of any point from the vehicle body coordinate system. Transformation formula to the global coordinate system: in, This represents the coordinates of the point in the global coordinate system; Transformation of representative points to vehicle coordinate system The coordinates after.

[0014] Optionally, a method for determining the extrinsic parameters of a vehicle-mounted surround-view camera further includes: controlling the display module to display in real time the auxiliary images, marker detection results, and parameter calculation results acquired by the distortion-free lens camera.

[0015] In a second aspect, the present invention provides a vehicle-mounted surround-view camera extrinsic parameter determination system for implementing a vehicle-mounted surround-view camera extrinsic parameter determination method as described in the first aspect, comprising: a processing device, a vehicle-mounted surround-view camera, a display module, markers, and a distortion-free lens camera, wherein the processing device is connected to the vehicle-mounted surround-view camera and the display module respectively.

[0016] In summary, the present invention has at least the following beneficial technical effects: 1. High flexibility: This invention achieves "vehicles can be parked anywhere and markers can be placed anywhere" through the technical approach of "multi-directional acquisition by auxiliary cameras + global coordinate unification": vehicles do not need to be in a fixed position, and markers only need to be located in the center of the corresponding camera's field of view (no need for common viewing). The auxiliary camera can establish global coordinate association by shooting around the vehicle.

[0017] 2. Improved Calibration Accuracy: Existing technologies require placing markers in the overlapping areas of the fields of view of adjacent vehicle cameras (often at the edge of the fisheye camera image). However, these edge areas are prone to problems such as vignetting, color aberration, and increased distortion, leading to high feature point detection errors (typically ≥5%). This invention, through a design that "preferentially places markers in the center of the single camera's field of view," avoids the degraded image quality areas at the edges. It utilizes the image center region, which has low distortion and high resolution, combined with the high robustness of ArUco markers, reducing the feature point detection error to ≤1.2%.

[0018] 3. Lowered operational threshold and controllable cost: This invention uses four low-cost ArUco planar markers, which only need to be flat on the ground without any adjustment of their orientation. Common smartphones can be used as auxiliary equipment (no need to purchase specialized equipment). The entire process is highly automated, requiring no professional technicians, making it more suitable for large-scale scenarios such as after-sales maintenance at large agricultural vehicle 4S stores and vehicle factory inspections. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a method for determining the extrinsic parameters of a vehicle-mounted surround-view camera according to an embodiment of the present invention. Figure 2 The diagram shows an actual implementation example of the placement of markers and vehicles.

[0020] Figure 3 This is a schematic diagram of coordinate system transformation. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] The terminology used in the following embodiments of the present invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification and appended claims of the present invention, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in the present invention refers to and includes any or all possible combinations of one or more of the listed items. The terms “first” and “second” are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of the present invention, unless otherwise stated, “a plurality” means two or more.

[0023] This invention provides a method for determining the extrinsic parameters of a vehicle-mounted surround-view camera.

[0024] refer to Figure 1-3 A method for determining the extrinsic parameters of a vehicle-mounted surround-view camera includes the following steps: S1: Control each vehicle surround view camera to capture images of the vehicle surround view camera with a marker; the marker is placed at the front, rear, left and right sides of the vehicle respectively.

[0025] The markers are four ArUco markers of known size, denoted as follows: (Corresponding to the forward-looking camera) ), (Corresponding to the left-view camera) ), (Corresponding to rearview camera) ), (Corresponding to the right-view camera) ).

[0026] Each marker is placed in the center of the field of view of the corresponding vehicle-mounted surround view camera. It is not necessary to ensure that any two vehicle-mounted surround view cameras have a common marker, and the vehicle can be parked anywhere in the calibrated scene without a fixed pose.

[0027] S2: Perform marker detection on each auxiliary image containing multiple markers, and construct a global coordinate system based on the relative poses between the markers; the auxiliary images are captured by a distortion-free lens camera.

[0028] Using a distortion-free lens camera eliminates the need for additional intrinsic parameter calibration, thus preventing the introduction of errors. When capturing auxiliary images, take pictures from four positions around the vehicle: left front, right front, left rear, and right rear. Each auxiliary image should contain at least two markers, prioritizing the coverage of adjacent marker pairs (e.g., [image description needed]). and , and , and , and The markers must be complete and clear, and at least four valid images must be collected for each item.

[0029] Specifically, S2 includes the following steps: S2-1: Perform marker detection on each auxiliary image, obtaining the ID and corner pixel coordinates of each marker. Calculate the size factor of each marker. The calculation formula is: The size factor is used to characterize the observation quality of the marker in the auxiliary image; Representative markers The pixel coordinates of a corner point on the x-axis of the image coordinate system; This represents the pixel coordinate difference of the diagonal point on the y-axis.

[0030] The marker with the largest size factor in each auxiliary image is selected as the primary marker, and its corresponding local 3D coordinate system is: (z=0, the markers are in the same plane).

[0031] S2-2: Construct a local coordinate system for the main marker based on the pixel coordinates of its corner points and its known actual size. The projection transformation matrix is ​​used to project the image pixel coordinates. Using the inverse transformation of this projection transformation matrix, the corner pixel coordinates of other markers in the auxiliary image are back-projected to the image pixel coordinate system. The system was used to obtain the markers in the system. The three-dimensional coordinates are given. For each pair of markers in the auxiliary image (preferably adjacent marker pairs), their relative transformation matrix is ​​calculated. ( Represents a non-primary identifier ID; (Represents the primary tag ID), characterizing the tag. To the main marker The relationship between rotation and translation.

[0032] S2-3: Define confidence weights Used to evaluate the observation quality of the relative transformation matrix: For pairwise observations of uniformly labeled pairs, the relative transformation matrix with the highest confidence weight is retained to ensure data reliability.

[0033] S2-4: with Using the local 3D coordinate system as the global coordinate system, the relative transformation matrix is ​​used to... , , The coordinates are transformed sequentially to This system achieves global uniformity of coordinates for all markers. The conversion formula is: in, Representative markers Switch to Global coordinates after the system; Representative markers Coordinates in its own local coordinate system ; Representative markers To marker The relative transformation matrix.

[0034] S3: Based on the PNP (Perspective-n-Point) algorithm and the pre-calibrated intrinsic parameters of the vehicle surround view camera, calculate the position coordinates of each vehicle surround view camera in the global coordinate system.

[0035] For each vehicle-mounted surround-view camera image acquired, the PNP algorithm, combined with pre-calibrated intrinsic parameters of the vehicle-mounted surround-view camera, is used to solve the coordinate system of the vehicle-mounted surround-view camera to... The first transformation matrix of the system (global coordinate system) : in, represent 3. Rotation matrix; represent 1. Translation vector ( ); 0 represents 3. Zero vector.

[0036] Based on the first transformation matrix, the derivation of the vehicle-mounted surround view camera... Position coordinates in the system : in, Represents a rotation matrix The transpose of the transpose yields the final result. exist The position coordinates in the system.

[0037] S4: Establish a vehicle coordinate system with the vehicle center point as the origin based on the coordinates of each position.

[0038] Based on four vehicle surround view cameras The positional relationship within the system, calculating the vehicle's center point. Coordinates in the system Establish a vehicle coordinate system with the vehicle center point as the origin. Solve for the vehicle body coordinate system. Compared to Yaw angle of the system ,get Tie The second transformation matrix of the system : in, , , The center point of the vehicle is at The coordinates of the x, y, and z axes in the system.

[0039] S5: Establish the transformation relationship between the global coordinate system and the vehicle body coordinate system to determine the extrinsic parameters of the vehicle surround view camera and perform image stitching.

[0040] Combining the first transformation matrix and the second transformation matrix, we can obtain the transformation of any point from the vehicle body coordinate system. Transformation formula to the global coordinate system: in, This represents the coordinates of the point in the global coordinate system; Transformation of representative points to vehicle coordinate system The coordinates after.

[0041] A method for determining extrinsic parameters of a vehicle-mounted surround-view camera further includes a control display module that displays in real time auxiliary images acquired by a distortion-free lens camera, marker detection results, and parameter calculation results. When the integrity of markers in the auxiliary image is less than 90%, a prompt to re-acquire data is issued.

[0042] Based on the same inventive concept, embodiments of the present invention also provide a system for determining the extrinsic parameters of an in-vehicle surround-view camera.

[0043] A vehicle-mounted surround-view camera extrinsic parameter determination system includes: a processing device, a vehicle-mounted surround-view camera, a display module, markers, and a distortion-free lens camera (for auxiliary imaging).

[0044] The processing device is connected to both the vehicle-mounted surround-view camera and the display module, and is capable of acquiring auxiliary images captured by a distortion-free lens camera. The processing device includes a computing device comprising one or more processors and memory, configured to perform all or core steps of the method described in this invention. The processing device can be the vehicle's own in-vehicle infotainment system or a dedicated surround-view system controller.

[0045] The markers consist of at least four non-coplanar visual markers with known geometric dimensions, and each marker has a unique identification ID, such as an ArUco code.

[0046] A distortion-free lens camera is a standalone image acquisition device for capturing markers from multiple angles; it can be a smartphone or a high-resolution digital camera equipped with a distortion-free lens.

[0047] Various variations and specific examples of the methods provided in the above embodiments are applicable to the vehicle surround view camera extrinsic parameter determination system of this embodiment. Through the foregoing detailed description of the vehicle surround view camera extrinsic parameter determination method, those skilled in the art can clearly understand the implementation method of the vehicle surround view camera extrinsic parameter determination system of this embodiment. For the sake of brevity, it will not be described in detail here.

[0048] The above description of the embodiments is only used to provide a detailed introduction to the technical solution of the present invention. However, the description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention, and should not be construed as a limitation of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for determining the extrinsic parameters of a vehicle-mounted surround-view camera, characterized in that, Includes the following steps: S1: Control each vehicle surround view camera to capture images of the vehicle surround view camera with a marker; the markers are placed at the front, rear, left and right sides of the vehicle respectively; S2: Perform marker detection on each auxiliary image containing multiple markers, and construct a global coordinate system based on the relative poses between the markers; The auxiliary images were captured by a distortion-free lens camera; S3: Based on the PNP algorithm and the pre-calibrated intrinsic parameters of the vehicle surround view camera, calculate the position coordinates of each vehicle surround view camera in the global coordinate system; S4: Establish a vehicle coordinate system with the vehicle center point as the origin based on the coordinates of each position; S5: Establish the transformation relationship between the global coordinate system and the vehicle body coordinate system to determine the extrinsic parameters of the vehicle surround view camera and perform image stitching.

2. The method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in claim 1, characterized in that, The markers are four ArUco markers of known size.

3. The method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in claim 1, characterized in that, The auxiliary images must be at least 4 images, and each image must contain at least 2 or more markers.

4. The method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in claim 3, characterized in that, S2 includes: S2-1: Calculate the size factor for each marker. : in, Representative markers The pixel coordinates of a corner point on the x-axis of the image coordinate system; This represents the pixel coordinate difference of the diagonal point on the y-axis; the marker with the largest size factor in each auxiliary image is selected as the primary marker, and its corresponding local 3D coordinate system is... Tie; S2-2: Construct a local coordinate system for the main marker based on the pixel coordinates of its corner points and its known actual size. The projection transformation matrix to the image pixel coordinate system backprojects the corner pixel coordinates of other markers in the auxiliary image to the image pixel coordinate system. The system was used to obtain the markers in the system. The three-dimensional coordinates are used in the system; for each pair of markers in the auxiliary image, the relative transformation matrix is ​​calculated. , Represents a non-primary identifier ID; Represents the primary identifier ID; S2-3: Define confidence weights : Retain the relative transformation matrix with the highest confidence weight; S2-4: with Using the local 3D coordinate system as the global coordinate system, the markers are transformed using a relative transformation matrix. , , The coordinates are transformed sequentially to Tie: in, Representative markers Switch to Global coordinates after the system; Representative markers Coordinates in its own local coordinate system ; Representative markers To marker The relative transformation matrix.

5. The method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in claim 4, characterized in that, S3 includes: For each vehicle-mounted surround-view camera image acquired, the PNP algorithm, combined with pre-calibrated intrinsic parameters of the vehicle-mounted surround-view camera, is used to solve the coordinate system of the vehicle-mounted surround-view camera to... The first transformation matrix of the system : in, represent 3. Rotation matrix; represent 1. Translation vector ( ); 0 represents 3. Zero vector; Based on the first transformation matrix, the derivation of the vehicle-mounted surround view camera... Position coordinates in the system : in, Represents a rotation matrix The transpose of the transpose yields the final result. exist The position coordinates in the system.

6. The method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in claim 5, characterized in that, S4 includes: Based on four vehicle surround view cameras The positional relationship within the system, calculating the vehicle's center point. Coordinates in the system Establish a vehicle coordinate system with the vehicle center point as the origin. Solve for the vehicle body coordinate system Compared to Yaw angle of the system ,get Tie The second transformation matrix of the system : in, , , The center point of the vehicle is at The coordinates of the x, y, and z axes in the system.

7. The method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in claim 6, characterized in that, S5 includes: Combining the first transformation matrix and the second transformation matrix, we can obtain the transformation of any point from the vehicle body coordinate system. Transformation formula to the global coordinate system: in, This represents the coordinates of the point in the global coordinate system; Transformation of representative points to vehicle coordinate system The coordinates after.

8. The method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in claim 1, characterized in that, Also includes: The control display module displays in real time the auxiliary images, marker detection results, and parameter calculation results acquired by the distortion-free lens camera.

9. A vehicle-mounted surround-view camera extrinsic parameter determination system, characterized in that, A method for determining the extrinsic parameters of a vehicle-mounted surround-view camera as described in any one of claims 1-8, comprising: a processing device, a vehicle-mounted surround-view camera, a display module, markers, and a distortion-free lens camera, wherein the processing device is connected to the vehicle-mounted surround-view camera and the display module respectively.

Images