A method and device for calibrating external parameters based on a vehicle-mounted reversing image system

Abstract

The application provides a kind of based on the external parameter calibration method and device of vehicle reversing image system.The method comprises: determining the intersection of multiple preset calibration lines on the ground behind the vehicle in the multiple calibration intersection coordinates of world coordinate system, and in response to the rear view image obtained based on the rear view camera of the vehicle, obtain the multiple reference intersection coordinates of the multiple reference lines in image pixel coordinate system based on the rear view image;The external parameter of the rear view camera is obtained based on the multiple calibration intersection coordinates in world coordinate system and the multiple reference intersection coordinates in image pixel coordinate system.The application obtains the external parameter of the rear view camera by the multiple calibration intersection coordinates of the intersection of multiple preset calibration lines on the ground behind the vehicle in world coordinate system and the multiple reference intersection coordinates of the multiple reference lines in image pixel coordinate system in rear view image.The calibration process is simple, the requirement of calibration site and calibration tool is low, the operation is simple, the cost is low, and it is easy to apply.

Description

Technical Field

[0001] This application relates to the field of camera calibration technology, and more specifically, to a method, apparatus, medium, and electronic device for calibrating external parameters based on an in-vehicle reversing camera system. Background Technology

[0002] In vehicle reversing camera systems, calibrating the intrinsic and extrinsic parameters of the rearview camera is a crucial prerequisite for accurate distance measurement and positioning using computer vision. The intrinsic parameters of the rearview camera are determined by the inherent properties of the camera itself, while the extrinsic parameters are determined by the camera's mounting position and angle.

[0003] Generally, the intrinsic parameters of a camera are calibrated using the Zhang Zhengyou calibration method, while extrinsic parameters are usually calibrated using many different methods. However, most current extrinsic parameter calibration methods have relatively strict requirements for the calibration site and calibration tools.

[0004] Therefore, this application provides an external parameter calibration method based on an in-vehicle reversing camera system to solve the above-mentioned technical problems. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, medium, and electronic device for calibrating external parameters based on a vehicle reversing camera system, which can solve at least one of the aforementioned technical problems. The specific solution is as follows:

[0006] According to a specific embodiment of this application, in a first aspect, this application provides a method for calibrating external parameters based on an in-vehicle reversing camera system, including:

[0007] Determine the coordinates of the intersection points of multiple pre-set calibration lines on the ground behind the vehicle in the world coordinate system.

[0008] And in response to acquiring a rear view image based on a vehicle-based rear-view camera, obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, wherein the rear view image includes at least the image of the multiple calibration lines and the image of the multiple reference lines generated by the rear-view camera, and the image of the multiple reference lines coincides with the image of the multiple calibration lines;

[0009] The extrinsic parameters of the rear-view camera are obtained based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system.

[0010] Optionally, the plurality of calibration lines include: a first calibration line, a second calibration line, and a third calibration line, wherein the first calibration line and the second calibration line are both parallel to the Yw axis direction in the vehicle coordinate system, the first calibration line and the second calibration line are located on both sides of the vehicle body, and the distance between the first calibration line and the second calibration line is equal to the width of the vehicle body; the projection of the preset position of the vehicle's rear bumper on the ground is 1 meter away from the third calibration line behind the vehicle body, and the third calibration line is perpendicular to the first calibration line and the second calibration line. The plurality of calibration intersection point coordinates include the first calibration intersection point coordinates and the second calibration intersection point coordinates; the images of the plurality of reference lines include: the image of the first reference line that coincides with the image of the first calibration line, the image of the second reference line that coincides with the image of the second calibration line, and the image of the third reference line that coincides with the image of the third calibration line; the plurality of reference intersection point coordinates include: the first reference intersection point where the first reference line intersects the third reference line, the second reference intersection point where the second reference line intersects the third reference line, and the third reference intersection point where the first reference line intersects the second reference line.

[0011] Optionally, obtaining the extrinsic parameters of the rear-view camera based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system includes:

[0012] The coordinates of multiple reference intersection points in the image pixel coordinate system are normalized to a preset normalization plane to obtain multiple intersection point projection coordinates. The preset normalization plane refers to the plane where Zc equals 1 in the camera coordinate system. The multiple intersection point projection coordinates include at least the vanishing point coordinates. The vanishing point coordinates are the projection coordinates of the third reference intersection point coordinates in the preset normalization plane.

[0013] The extrinsic parameters of the rear-view camera are obtained based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system.

[0014] Optionally, obtaining the extrinsic parameters of the rear-view camera based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system includes:

[0015] ZXZ Euler angles are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system, wherein ZXZ Euler angles represent the rotation matrix of the first extrinsic parameter of the rear-view camera.

[0016] The translation vector characterizing the second extrinsic parameter of the rear-view camera is obtained by substituting the rotation matrix of the rear-view camera, the coordinates of multiple calibration intersection points in the world coordinate system, and the projected coordinates of multiple intersection points in the preset normalized plane into a preset linear equation.

[0017] Optionally, obtaining the Z×Z Euler angles based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated coordinates of multiple intersection points in the world coordinate system includes:

[0018] The direction vectors of each axis direction in the camera coordinate system are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane, the coordinates of multiple calibrated intersection points in the world coordinate system, and the preset perspective projection relationship between each calibrated intersection point coordinate and the corresponding projection coordinates of the intersection point.

[0019] ZXZ Euler angles are obtained based on the direction vectors of each axis in the camera coordinate system.

[0020] Optionally, the preset linear equation is:

[0021]

[0022] Where u represents the coordinate in the Xc-axis direction in the preset normalized plane, v represents the coordinate in the Yc-axis direction in the preset normalized plane, x represents the coordinate in the Xw-axis direction in the world coordinate system, y represents the coordinate in the Yw-axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.

[0023] Optionally, after acquiring the rear-view image using the vehicle-based rear-view camera, the process includes:

[0024] Before obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, the intrinsic parameters and distortion parameters of the rear view camera are calibrated based on the Zhang Dingyou calibration method.

[0025] The rear view image is corrected based on the intrinsic parameters and distortion parameters of the rear view camera.

[0026] According to a specific embodiment of this application, in a second aspect, this application provides an external parameter calibration device based on a vehicle reversing camera system, comprising:

[0027] The coordinate acquisition unit is used to determine the coordinates of multiple calibration intersection points in the world coordinate system of the intersection points of multiple preset calibration lines on the ground behind the vehicle.

[0028] And in response to acquiring a rear view image based on a vehicle-based rear-view camera, obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, wherein the rear view image includes at least the image of the multiple calibration lines and the image of the multiple reference lines generated by the rear-view camera, and the image of the multiple reference lines coincides with the image of the multiple calibration lines;

[0029] The extrinsic parameter acquisition unit is used to obtain the extrinsic parameters of the rear-view camera based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system.

[0030] Optionally, the plurality of calibration lines include: a first calibration line, a second calibration line, and a third calibration line, wherein the first calibration line and the second calibration line are both parallel to the Yw axis direction in the vehicle coordinate system, the first calibration line and the second calibration line are located on both sides of the vehicle body, and the distance between the first calibration line and the second calibration line is equal to the width of the vehicle body; the projection of the preset position of the vehicle's rear bumper on the ground is 1 meter away from the third calibration line behind the vehicle body, and the third calibration line is perpendicular to the first calibration line and the second calibration line. The plurality of calibration intersection point coordinates include the first calibration intersection point coordinates and the second calibration intersection point coordinates; the images of the plurality of reference lines include: the image of the first reference line that coincides with the image of the first calibration line, the image of the second reference line that coincides with the image of the second calibration line, and the image of the third reference line that coincides with the image of the third calibration line; the plurality of reference intersection point coordinates include: the first reference intersection point where the first reference line intersects the third reference line, the second reference intersection point where the second reference line intersects the third reference line, and the third reference intersection point where the first reference line intersects the second reference line.

[0031] Optionally, obtaining the extrinsic parameters of the rear-view camera based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system includes:

[0032] The coordinates of multiple reference intersection points in the image pixel coordinate system are normalized to a preset normalization plane to obtain multiple intersection point projection coordinates. The preset normalization plane refers to the plane where Zc equals 1 in the camera coordinate system. The multiple intersection point projection coordinates include at least the vanishing point coordinates. The vanishing point coordinates are the projection coordinates of the third reference intersection point coordinates in the preset normalization plane.

[0033] The extrinsic parameters of the rear-view camera are obtained based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system.

[0034] Optionally, obtaining the extrinsic parameters of the rear-view camera based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system includes:

[0035] ZXZ Euler angles are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system, wherein ZXZ Euler angles represent the rotation matrix of the first extrinsic parameter of the rear-view camera.

[0036] The translation vector characterizing the second extrinsic parameter of the rear-view camera is obtained by substituting the rotation matrix of the rear-view camera, the coordinates of multiple calibration intersection points in the world coordinate system, and the projected coordinates of multiple intersection points in the preset normalized plane into a preset linear equation.

[0037] Optionally, obtaining the Z×Z Euler angles based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated coordinates of multiple intersection points in the world coordinate system includes:

[0038] The direction vectors of each axis direction in the camera coordinate system are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane, the coordinates of multiple calibrated intersection points in the world coordinate system, and the preset perspective projection relationship between each calibrated intersection point coordinate and the corresponding projection coordinates of the intersection point.

[0039] ZXZ Euler angles are obtained based on the direction vectors of each axis in the camera coordinate system.

[0040] Optionally, the preset linear equation is:

[0041]

[0042] Where u represents the coordinate in the Xc-axis direction in the preset normalized plane, v represents the coordinate in the Yc-axis direction in the preset normalized plane, x represents the coordinate in the Xw-axis direction in the world coordinate system, y represents the coordinate in the Yw-axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.

[0043] Optionally, after acquiring the rear-view image using the vehicle-based rear-view camera, the process includes:

[0044] Before obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, the intrinsic parameters and distortion parameters of the rear view camera are calibrated based on the Zhang Dingyou calibration method.

[0045] The rear view image is corrected based on the intrinsic parameters and distortion parameters of the rear view camera.

[0046] According to a specific embodiment of this application, in a third aspect, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the external parameter calibration method based on the vehicle reversing image system as described in any of the preceding claims.

[0047] According to a specific embodiment of this application, in a fourth aspect, this application provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the extrinsic parameter calibration method based on the vehicle reversing image system as described in any of the preceding claims.

[0048] Compared with the prior art, the above-described solutions of this application have at least the following beneficial effects:

[0049] This application provides a method, apparatus, medium, and electronic device for extrinsic parameter calibration of a vehicle-mounted reversing camera system. The method includes: determining the coordinates of multiple calibration intersection points in a world coordinate system of the intersection points of multiple preset calibration lines on the ground behind the vehicle; and, in response to acquiring a rear-view image based on a vehicle-based rear-view camera, obtaining the coordinates of multiple reference intersection points of the multiple reference lines in an image pixel coordinate system based on the rear-view image; and obtaining the extrinsic parameters of the rear-view camera based on the coordinates of the multiple calibration intersection points in the world coordinate system and the coordinates of the multiple reference intersection points in the image pixel coordinate system. This application obtains the extrinsic parameters of the rear-view camera by using the coordinates of multiple calibration intersection points in a world coordinate system of the intersection points of multiple preset calibration lines on the ground behind the vehicle, and the coordinates of multiple reference lines in the rear-view image in an image pixel coordinate system. The calibration process is simple, has low requirements for calibration site and tools, is easy to operate, low in cost, and easy to apply. Attached Figure Description

[0050] Figure 1 A flowchart of an external parameter calibration method based on an in-vehicle reversing camera system according to an embodiment of this application is shown;

[0051] Figure 2 A schematic diagram showing the coordinates of multiple calibrated intersection points in the world coordinate system according to an embodiment of this application is provided;

[0052] Figure 3 A schematic diagram showing the coordinates of multiple reference intersection points in the image pixel coordinate system according to an embodiment of this application is provided;

[0053] Figure 4 This diagram illustrates the relationship between the coordinates of multiple calibration intersection points in the world coordinate system and the camera coordinate system according to an embodiment of this application.

[0054] Figure 5 A schematic diagram showing the projected coordinates of multiple intersection points in a preset normalized plane according to an embodiment of this application is provided.

[0055] Figure 6 A schematic diagram illustrating the conversion of ZXZ Euler angles according to an embodiment of this application is shown;

[0056] Figure 7 A unit block diagram of an external parameter calibration device based on an in-vehicle reversing camera system according to an embodiment of this application is shown.

[0057] Explanation of reference numerals in the attached figures:

[0058] CA - Rearview Camera;

[0059] A1 - First calibration line, B1 - Second calibration line, C1 - Third calibration line, PA1 - Coordinates of the first calibration intersection point, PB1 - Coordinates of the second calibration intersection point;

[0060] A2 - First reference line, B2 - Second reference line, C2 - Third reference line, PA2 - Coordinates of the first reference intersection point, PB2 - Coordinates of the second reference intersection point, P2 - Coordinates of the third reference intersection point;

[0061] PA3 - Projected coordinates of the first intersection point, PB3 - Projected coordinates of the second intersection point, P3 - Coordinates of the vanishing point. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0064] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0065] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0066] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0067] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0068] It should be noted that any symbols and / or numbers present in the specification that are not marked in the accompanying drawings are not reference numerals.

[0069] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0070] The embodiments provided in this application are embodiments of an external parameter calibration method based on an in-vehicle reversing camera system.

[0071] The following is combined with Figure 1 The embodiments of this application will be described in detail.

[0072] Step S101: Determine the coordinates of multiple calibration intersection points of multiple preset calibration lines on the ground behind the vehicle in the world coordinate system, and in response to the rear view image acquired by the rear view camera CA based on the vehicle, obtain the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image.

[0073] The rear view image includes at least the images of the multiple calibration lines and the images of the multiple reference lines generated by the rear-view camera CA, wherein the images of the multiple reference lines coincide with the images of the multiple calibration lines.

[0074] The ground refers to a flat surface. In this embodiment, markings are made on a flat surface.

[0075] The rearview camera CA is positioned at the center of the vehicle's rear bumper.

[0076] The key to the technical implementation of this application's embodiments lies in establishing a projection mapping from spatial objects to camera images. In the field of computer vision research, this is generally represented by a camera imaging model. The pinhole camera model is a commonly used camera imaging model. The establishment of the pinhole camera model involves four coordinate systems: a two-dimensional image pixel coordinate system, an image physical coordinate system, a camera coordinate system, and a world coordinate system.

[0077] Image pixel coordinates are used to represent the image obtained through an optical system and photoelectric conversion elements (such as CCD and CMOS) and stored in the computer's storage device. It uses pixels as the unit of measurement, and the size of the image is defined as its resolution. This coordinate system has its origin at the top left corner of the image, with rows along the u-axis and columns along the v-axis.

[0078] The physical coordinate system of an image is established to describe the pinhole imaging relationship. Its origin is the intersection of the optical axis of the optical system and the imaging plane of the photoelectric conversion element (such as CCD, CMOS), and the x-axis and y-axis are parallel to the u-axis and v-axis of the image pixel coordinate system, respectively. The physical coordinate system of the image and the image pixel coordinate system are called the image coordinate system.

[0079] The camera coordinate system, established on the camera, changes as the camera moves; therefore, it is a local coordinate system. The camera coordinate system consists of the Xc, Yc, and Zc axes. The origin of the camera coordinate system is the optical center Oc of the optical system. The Xc-Oc-Yc plane is parallel to the image coordinate system plane, and the Xc and Yc axes are parallel to the u and v axes of the image coordinate system, respectively. The Zc axis of the camera coordinate system is the optical axis of the optical system, and together with the Xc and Yc axes, forms a right-handed coordinate system.

[0080] The world coordinate system is established based on the measurement environment and objectives. Once established, it does not change with camera movement; therefore, it is sometimes also called the global coordinate system. The world coordinate system consists of the Xw, Yw, and Zw axes.

[0081] In some specific embodiments, such as Figure 2 As shown, the multiple calibration lines include: a first calibration line A1, a second calibration line B1, and a third calibration line C1. The first calibration line A1 and the second calibration line B1 are both parallel to the Yw axis in the vehicle coordinate system. The first calibration line A1 and the second calibration line B1 are located on opposite sides of the vehicle body, and the distance between the first calibration line A1 and the second calibration line B1 is equal to the width of the vehicle body. The projection of the preset position of the vehicle's rear bumper onto the ground is 1 meter away from the third calibration line C1 behind the vehicle body, and the third calibration line C1 is perpendicular to the first calibration line A1 and the second calibration line B1. The multiple calibration intersection coordinates include the first calibration intersection coordinate PA1 and the second calibration intersection coordinate PB1.

[0082] Two calibration lines, A1 and B1, are affixed to the ground at the extended line of the vehicle's outer contour (i.e., in the Yw direction of the vehicle coordinate system). A third calibration line, C1, is affixed perpendicularly to A1 and B1, 1 meter behind the rear bumper. The intersection of A1 and C1 is the first calibration intersection point coordinate PA1 in the world coordinate system, and the intersection of B1 and C1 is the second calibration intersection point coordinate PB1 in the world coordinate system.

[0083] like Figure 3 As shown, the images of the multiple reference lines include: the image of the first reference line A2 that coincides with the image of the first calibration line A1, the image of the second reference line B2 that coincides with the image of the second calibration line B1, and the image of the third reference line C2 that coincides with the image of the third calibration line C1; the coordinates of the multiple reference intersection points include: the coordinates of the first reference intersection point PA2 where the first reference line A2 intersects the third reference line C2, the coordinates of the second reference intersection point PB2 where the second reference line B2 intersects the third reference line C2, and the coordinates of the third reference intersection point P2 where the first reference line A2 intersects the second reference line B2.

[0084] In some specific embodiments, after the vehicle-based rearview camera (CA) acquires the rearview image, it includes:

[0085] Step S101-1: Before obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, the intrinsic parameters and distortion parameters of the rear view camera CA are calibrated based on the Zhang Dingyou calibration method.

[0086] Step S101-2: Correct the rear view image based on the intrinsic parameters and distortion parameters of the rear view camera CA.

[0087] This specific embodiment calibrates the intrinsic and distortion parameters of the rear-view camera CA using the Zhang Dingyou calibration method, ensuring that the rear-view camera CA can obtain a distortion-free rear view image. This ensures that the coordinates of multiple reference lines in the rear view image accurately correspond to the coordinates of multiple calibration intersection points in the world coordinate system, thereby obtaining accurate extrinsic parameters.

[0088] Step S102: Obtain the extrinsic parameters of the rear-view camera CA based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system.

[0089] This embodiment of the application obtains the extrinsic parameters of the rear-view camera CA by using the coordinates of multiple calibration intersection points in the world coordinate system of the intersection points of multiple preset calibration lines on the ground behind the vehicle, and the coordinates of multiple reference intersection points in the image pixel coordinate system of multiple reference lines in the rear view image. The calibration process is simple, has low requirements for calibration site and calibration tools, is easy to operate, low in cost, and easy to apply.

[0090] In some specific embodiments, obtaining the extrinsic parameters of the rear-view camera CA based on multiple calibration intersection point coordinates in the world coordinate system and multiple reference intersection point coordinates in the image pixel coordinate system includes:

[0091] Step S102-1: Normalize the coordinates of multiple reference intersection points in the image pixel coordinate system to a preset normalization plane to obtain the projection coordinates of multiple intersection points.

[0092] The preset normalized plane refers to the plane where Zc equals 1 in the camera coordinate system. The multiple intersection point projection coordinates include at least the vanishing point coordinates P3, which are the projection coordinates of the third reference intersection point coordinates P2 in the preset normalized plane.

[0093] like Figure 3 As shown, the projection coordinates of the third reference intersection point P2 onto the preset normalized plane are also the vanishing point coordinates P3.

[0094] Step S102-2: Obtain the extrinsic parameters of the rear-view camera CA based on the projected coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system.

[0095] This specific embodiment normalizes the coordinates of multiple reference intersection points in the image pixel coordinate system to a preset normalized plane in the camera coordinate system, which simplifies the complexity of data processing, simplifies complex data, and improves the efficiency of extrinsic parameter calibration.

[0096] In some specific embodiments, obtaining the extrinsic parameters of the rear-view camera CA based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system includes:

[0097] Step S102-2-1: Obtain ZXZ Euler angles based on the projected coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system.

[0098] Wherein, ZXZ Euler angles represent the rotation matrix of the first extrinsic parameter of the rear-view camera CA.

[0099] In some specific embodiments, obtaining Z×Z Euler angles based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated coordinates of multiple intersection points in the world coordinate system includes:

[0100] Step S102-2-1-1: Based on the projection coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system, as well as the preset perspective projection relationship between the coordinates of each calibrated intersection point and the corresponding projection coordinates of the intersection point, the direction vectors of each axis direction in the camera coordinate system are obtained.

[0101] Step S102-2-1-2: Obtain the ZXZ Euler angles based on the direction vectors of each axis in the camera coordinate system.

[0102] For example, such as Figures 2-5 As shown, since both the first calibration line A1 and the second calibration line B1 are parallel to the Yw axis in the world coordinate system, according to the geometric meaning of the vanishing point, the vector in the camera coordinate system... The Yw axis is parallel to the world coordinate system; in the image physical coordinate system, the coordinates of the first physical intersection point of the first physical line and the third physical line are (x0, y0), and the coordinates of the second physical intersection point of the second physical line and the third physical line are (x1, y1); combining the coordinates of the first calibration intersection point PA1 and the second calibration intersection point PB1 in the world coordinate system, and the projection coordinates of the first intersection point PA3 and the second intersection point PB3; according to the perspective projection relationship, in the camera coordinate system, the origin Oc, the coordinates of the first calibration intersection point PA1, and the coordinates of the first reference intersection point PA2 are collinear, and the origin Oc, the coordinates of the second calibration intersection point PB1, and the coordinates of the second reference intersection point PB2 are also collinear, then... y1, 1}; can be used to calculate the normal vector of plane OcPA1PB1. Since the third calibration line C1 lies in the plane OcPA1PB1, the line OcNab is also perpendicular to the third calibration line C1; since the third calibration line C1 is parallel to the Xw axis, the line OcNab must lie in the plane OwYwZw; therefore, the direction vector of the Xw axis can be expressed as... The direction vector of Zw can be calculated from the direction vectors of Xw and Yw. exist Figure 6 The straight line Nab is the intersection of the Xw-Yw plane and the Xc-Yc plane before and after rotation, therefore the vector... It must be perpendicular to the Zw and Zc axes. At this point, the answer can be found. Figure 6 Given the direction vectors of each vector, the Z×Z Euler angles can be calculated using the geometric interpretation of the dot product:

[0103]

[0104]

[0105]

[0106] At this point, the rotation matrix R can be obtained from Euclidean algorithm.

[0107] Step S102-2-2: Substitute the rotation matrix of the rear-view camera CA, the coordinates of multiple calibration intersection points in the world coordinate system, and the projection coordinates of multiple intersection points in the preset normalized plane into the preset linear equation to obtain the translation vector characterizing the second extrinsic parameter of the rear-view camera CA.

[0108] In some specific embodiments, the preset linear equation is:

[0109]

[0110] Where u represents the coordinate in the Xc-axis direction in the preset normalized plane, v represents the coordinate in the Yc-axis direction in the preset normalized plane, x represents the coordinate in the Xw-axis direction in the world coordinate system, y represents the coordinate in the Yw-axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.

[0111] Since the calibration process is performed on the plane of world coordinate system Zw=0, the aforementioned preset linear equation can be obtained within the preset normalized plane. Finally, the translation vector of the second extrinsic parameter of the rear-view camera CA is obtained.

[0112] This application also provides an apparatus embodiment that follows the above embodiments, used to implement the method steps described in the above embodiments. The interpretation of the same names is the same as that in the above embodiments, and the same technical effects are achieved. Therefore, it will not be repeated here.

[0113] like Figure 7 As shown, this application provides an external parameter calibration device 700 based on a vehicle reversing camera system, comprising:

[0114] The coordinate acquisition unit 701 is used to determine the coordinates of multiple calibration intersection points in the world coordinate system of the intersection points of multiple preset calibration lines on the ground behind the vehicle.

[0115] And in response to acquiring a rear view image based on a vehicle-based rear-view camera, obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, wherein the rear view image includes at least the image of the multiple calibration lines and the image of the multiple reference lines generated by the rear-view camera, and the image of the multiple reference lines coincides with the image of the multiple calibration lines;

[0116] The extrinsic parameter acquisition unit 702 is used to obtain the extrinsic parameters of the rear-view camera based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system.

[0117] Optionally, the plurality of calibration lines include: a first calibration line, a second calibration line, and a third calibration line, wherein the first calibration line and the second calibration line are both parallel to the Yw axis direction in the vehicle coordinate system, the first calibration line and the second calibration line are located on both sides of the vehicle body, and the distance between the first calibration line and the second calibration line is equal to the width of the vehicle body; the projection of the preset position of the vehicle's rear bumper on the ground is 1 meter away from the third calibration line behind the vehicle body, and the third calibration line is perpendicular to the first calibration line and the second calibration line. The plurality of calibration intersection point coordinates include the first calibration intersection point coordinates and the second calibration intersection point coordinates; the images of the plurality of reference lines include: the image of the first reference line that coincides with the image of the first calibration line, the image of the second reference line that coincides with the image of the second calibration line, and the image of the third reference line that coincides with the image of the third calibration line; the plurality of reference intersection point coordinates include: the first reference intersection point where the first reference line intersects the third reference line, the second reference intersection point where the second reference line intersects the third reference line, and the third reference intersection point where the first reference line intersects the second reference line.

[0118] Optionally, obtaining the extrinsic parameters of the rear-view camera based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system includes:

[0119] The coordinates of multiple reference intersection points in the image pixel coordinate system are normalized to a preset normalization plane to obtain multiple intersection point projection coordinates. The preset normalization plane refers to the plane where Zc equals 1 in the camera coordinate system. The multiple intersection point projection coordinates include at least the vanishing point coordinates. The vanishing point coordinates are the projection coordinates of the third reference intersection point coordinates in the preset normalization plane.

[0120] The extrinsic parameters of the rear-view camera are obtained based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system.

[0121] Optionally, obtaining the extrinsic parameters of the rear-view camera based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system includes:

[0122] ZXZ Euler angles are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system, wherein ZXZ Euler angles represent the rotation matrix of the first extrinsic parameter of the rear-view camera.

[0123] The translation vector characterizing the second extrinsic parameter of the rear-view camera is obtained by substituting the rotation matrix of the rear-view camera, the coordinates of multiple calibration intersection points in the world coordinate system, and the projected coordinates of multiple intersection points in the preset normalized plane into a preset linear equation.

[0124] Optionally, obtaining the Z×Z Euler angles based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated coordinates of multiple intersection points in the world coordinate system includes:

[0125] The direction vectors of each axis direction in the camera coordinate system are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane, the coordinates of multiple calibrated intersection points in the world coordinate system, and the preset perspective projection relationship between each calibrated intersection point coordinate and the corresponding projection coordinates of the intersection point.

[0126] ZXZ Euler angles are obtained based on the direction vectors of each axis in the camera coordinate system.

[0127] Optionally, the preset linear equation is:

[0128]

[0129] Where u represents the coordinate in the Xc-axis direction in the preset normalized plane, v represents the coordinate in the Yc-axis direction in the preset normalized plane, x represents the coordinate in the Xw-axis direction in the world coordinate system, y represents the coordinate in the Yw-axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system.

[0130] Optionally, after acquiring the rear-view image using the vehicle-based rear-view camera, the process includes:

[0131] Before obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, the intrinsic parameters and distortion parameters of the rear view camera are calibrated based on the Zhang Dingyou calibration method.

[0132] The rear view image is corrected based on the intrinsic parameters and distortion parameters of the rear view camera.

[0133] This embodiment of the application obtains the extrinsic parameters of the rear-view camera by using the coordinates of multiple calibration intersections in the world coordinate system of the intersections of multiple preset calibration lines on the ground behind the vehicle, and the coordinates of multiple reference intersections in the image pixel coordinate system of multiple reference lines in the rear-view image. The calibration process is simple, has low requirements for calibration site and calibration tools, is easy to operate, low in cost, and easy to apply.

[0134] This embodiment provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method steps described in the above embodiment.

[0135] This application provides a non-volatile computer storage medium storing computer-executable instructions that can perform the steps described in the above embodiments.

[0136] Finally, it should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0137] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for calibrating extrinsic parameters based on a vehicle reversing camera system, characterized in that, include: Determine the coordinates of the intersection points of multiple pre-set calibration lines on the ground behind the vehicle in the world coordinate system. And in response to acquiring a rear view image based on a vehicle-based rear-view camera, obtaining the coordinates of multiple reference intersection points of multiple reference lines in the image pixel coordinate system based on the rear view image, wherein the rear view image includes at least the images of the multiple calibration lines and the images of the multiple reference lines generated by the rear-view camera, and the images of the multiple reference lines coincide with the images of the multiple calibration lines; The extrinsic parameters of the rear-view camera are obtained based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system. The extrinsic parameters of the rear-view camera are obtained based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system, including: The coordinates of multiple reference intersection points in the image pixel coordinate system are normalized to a preset normalization plane to obtain multiple intersection point projection coordinates. The preset normalization plane refers to the plane where Zc equals 1 in the camera coordinate system. The multiple intersection point projection coordinates include at least the vanishing point coordinates. The vanishing point coordinates are the projection coordinates of the third reference intersection point coordinates in the preset normalization plane. The extrinsic parameters of the rear-view camera are obtained based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system. The process of obtaining the extrinsic parameters of the rear-view camera based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated intersection coordinates in the world coordinate system includes: ZXZ Euler angles are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system, wherein ZXZ Euler angles represent the rotation matrix of the first extrinsic parameter of the rear-view camera. The rotation matrix of the rear-view camera, the coordinates of multiple calibration intersection points in the world coordinate system, and the projected coordinates of multiple intersection points in the preset normalized plane are substituted into a preset linear equation to obtain the translation vector characterizing the second extrinsic parameter of the rear-view camera. The process of obtaining Z×Z Euler angles based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated coordinates of multiple intersection points in the world coordinate system includes: The direction vectors of each axis direction in the camera coordinate system are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane, the coordinates of multiple calibrated intersection points in the world coordinate system, and the preset perspective projection relationship between each calibrated intersection point coordinate and the corresponding projection coordinates of the intersection point. ZXZ Euler angles are obtained based on the direction vectors of each axis in the camera coordinate system. The preset linear equation is: ； Where u represents the coordinate in the Xc axis direction in the preset normalized plane, v represents the coordinate in the Yc axis direction in the preset normalized plane, x represents the coordinate in the Xw axis direction in the world coordinate system, y represents the coordinate in the Yw axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system. The multiple calibration lines include: a first calibration line, a second calibration line, and a third calibration line. The first and second calibration lines are both parallel to the Yw axis in the vehicle coordinate system. The first and second calibration lines are located on opposite sides of the vehicle body, and the distance between them is equal to the width of the vehicle body. The projection of the preset position of the vehicle's rear bumper onto the ground is 1 meter away from the third calibration line behind the vehicle body, and the third calibration line is perpendicular to both the first and second calibration lines. The multiple calibration intersection point coordinates include the first calibration intersection point coordinates and the second calibration intersection point coordinates; the images of the multiple reference lines include: the image of the first reference line that coincides with the image of the first calibration line, the image of the second reference line that coincides with the image of the second calibration line, and the image of the third reference line that coincides with the image of the third calibration line; the multiple reference intersection point coordinates include: the first reference intersection point where the first reference line intersects the third reference line, the second reference intersection point where the second reference line intersects the third reference line, and the third reference intersection point where the first reference line intersects the second reference line.

2. The method according to claim 1, characterized in that, After the vehicle-based rearview camera acquires the rearview image, it includes: Before obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, the intrinsic parameters and distortion parameters of the rear view camera are calibrated based on the Zhang Dingyou calibration method. The rear view image is corrected based on the intrinsic parameters and distortion parameters of the rear view camera.

3. A extrinsic parameter calibration device based on a vehicle reversing camera system, characterized in that, include: The coordinate acquisition unit is used to determine the coordinates of multiple calibration intersection points in the world coordinate system of the intersection points of multiple preset calibration lines on the ground behind the vehicle. And in response to acquiring a rear view image based on a vehicle-based rear-view camera, obtaining the coordinates of multiple reference intersection points of the multiple reference lines in the image pixel coordinate system based on the rear view image, wherein the rear view image includes at least the image of the multiple calibration lines and the image of the multiple reference lines generated by the rear-view camera, and the image of the multiple reference lines coincides with the image of the multiple calibration lines; The extrinsic parameter acquisition unit is used to obtain the extrinsic parameters of the rear-view camera based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system. The extrinsic parameters of the rear-view camera are obtained based on the coordinates of multiple calibration intersection points in the world coordinate system and the coordinates of multiple reference intersection points in the image pixel coordinate system, including: The coordinates of multiple reference intersection points in the image pixel coordinate system are normalized to a preset normalization plane to obtain multiple intersection point projection coordinates. The preset normalization plane refers to the plane where Zc equals 1 in the camera coordinate system. The multiple intersection point projection coordinates include at least the vanishing point coordinates. The vanishing point coordinates are the projection coordinates of the third reference intersection point coordinates in the preset normalization plane. The extrinsic parameters of the rear-view camera are obtained based on the projected coordinates of multiple intersection points in a preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system. The process of obtaining the extrinsic parameters of the rear-view camera based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated intersection coordinates in the world coordinate system includes: ZXZ Euler angles are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane and the coordinates of multiple calibrated intersection points in the world coordinate system, wherein ZXZ Euler angles represent the rotation matrix of the first extrinsic parameter of the rear-view camera. The rotation matrix of the rear-view camera, the coordinates of multiple calibration intersection points in the world coordinate system, and the projected coordinates of multiple intersection points in the preset normalized plane are substituted into a preset linear equation to obtain the translation vector characterizing the second extrinsic parameter of the rear-view camera. The process of obtaining Z×Z Euler angles based on the projected coordinates of multiple intersection points in a preset normalized plane and the calibrated coordinates of multiple intersection points in the world coordinate system includes: The direction vectors of each axis direction in the camera coordinate system are obtained based on the projected coordinates of multiple intersection points in the preset normalized plane, the coordinates of multiple calibrated intersection points in the world coordinate system, and the preset perspective projection relationship between each calibrated intersection point coordinate and the corresponding projection coordinates of the intersection point. ZXZ Euler angles are obtained based on the direction vectors of each axis in the camera coordinate system. The preset linear equation is: ； Where u represents the coordinate in the Xc axis direction in the preset normalized plane, v represents the coordinate in the Yc axis direction in the preset normalized plane, x represents the coordinate in the Xw axis direction in the world coordinate system, y represents the coordinate in the Yw axis direction in the world coordinate system, and t represents the translation vector in the camera coordinate system. The multiple calibration lines include: a first calibration line, a second calibration line, and a third calibration line. The first and second calibration lines are both parallel to the Yw axis in the vehicle coordinate system. The first and second calibration lines are located on opposite sides of the vehicle body, and the distance between them is equal to the width of the vehicle body. The projection of the preset position of the vehicle's rear bumper onto the ground is 1 meter away from the third calibration line behind the vehicle body, and the third calibration line is perpendicular to both the first and second calibration lines. The multiple calibration intersection point coordinates include the first calibration intersection point coordinates and the second calibration intersection point coordinates; the images of the multiple reference lines include: the image of the first reference line that coincides with the image of the first calibration line, the image of the second reference line that coincides with the image of the second calibration line, and the image of the third reference line that coincides with the image of the third calibration line; the multiple reference intersection point coordinates include: the first reference intersection point where the first reference line intersects the third reference line, the second reference intersection point where the second reference line intersects the third reference line, and the third reference intersection point where the first reference line intersects the second reference line.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 2.

5. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 2.

Images