Camera extrinsic parameter calibration method and device

By setting four calibration points that form quadrilaterals on the lane line, measuring and shooting their position information, and calculating camera external parameter information, the problems of complex calculations and high environmental requirements in the prior art are solved, and the simplification of camera external parameter calibration is achieved. and flexibility.

CN118941637APending Publication Date: 2024-11-12BEIJING JINGWEI HIRAIN TECH CO INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310538139.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The calculation process of the existing camera external parameter calibration method is complex and has high environmental requirements. It requires a checkerboard calibration plate or circular array calibration plate and needs to meet the calibration plate to appear completely in the camera field of view, and multiple feature points of the world coordinate system need to be extracted.

Method used

Four calibration points that can form a quadrilateral are set on the lane line. By measuring their position information and pixel coordinates on the captured image, combining world coordinates and pixel coordinates, the camera's external parameter information is calculated, which simplifies the calibration process and reduces the environment. Require.

Benefits of technology

The simple calculation process of camera external parameter calibration is realized, reducing the environmental requirements. It only requires the help of two lane lines on the road, without the need for calibration plates, which improves the flexibility and convenience of calibration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941637A_ABST
    Figure CN118941637A_ABST
Patent Text Reader

Abstract

The invention provides a camera external parameter calibration method and device, and relates to the field of camera calibration, and the method comprises the steps: measuring the position information of four calibration points in the visual field range of a vehicle-mounted camera relative to the vehicle-mounted camera, the four calibration points being points located on two lane lines, and every two connection lines being capable of forming a quadrangle; determining coordinates of the four calibration points in a world coordinate system according to the position information of the four calibration points relative to the vehicle-mounted camera; images shot by the vehicle-mounted camera for the four calibration points are obtained, and pixel coordinates of the four calibration points on the images are determined; and according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates of the four calibration points on the image, external parameter information of the vehicle-mounted camera is determined. According to the method, the external parameters of the camera can be obtained through calculation only by simply setting four calibration points, the calculation process is simple, only two lane lines on a road are needed in the calibration process, a calibration plate is not needed, and the requirement for the calibration environment is lowered.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of camera calibration, and in particular to a camera extrinsic calibration method and device. Background Art

[0002] Before the camera is officially put into use, it needs to be calibrated with external parameters to determine the transformation relationship between the pixel coordinate system and the world coordinate system.

[0003] In the prior art, when calibrating the external parameters of a camera, a black and white checkerboard calibration plate or a circular array calibration plate is generally used. When using a checkerboard calibration plate or a circular array calibration plate to calibrate a camera, it is necessary to satisfy that the checkerboard calibration plate or the circular array calibration plate must appear completely in the camera field of view, and multiple feature points in the world coordinate system need to be extracted. However, this method has the problems of complex calculation process and high requirements on the conditions of the calibration environment. Summary of the invention

[0004] In view of this, the present application provides a camera extrinsic calibration method and device, which is used to solve the problems of complex calculation process and high requirements on calibration environment conditions in the prior art. The technical solution is as follows:

[0005] A camera extrinsic calibration method, comprising:

[0006] Measure the position information of four calibration points within the field of view of the vehicle camera relative to the vehicle camera, wherein the four calibration points are points located on the two lane lines and the lines connecting any two of them can form a quadrilateral;

[0007] According to the position information of the four calibration points relative to the vehicle-mounted camera, the coordinates of the four calibration points in the world coordinate system are determined;

[0008] Obtain the images taken by the vehicle-mounted camera of the four calibration points, and determine the pixel coordinates of the four calibration points on the image respectively;

[0009] The external parameter information of the vehicle-mounted camera is determined according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

[0010] Optionally, the vehicle where the on-board camera is located is parallel to the two lane lines and is located in the center of the two lane lines;

[0011] Measure the position information of four calibration points within the field of view of the vehicle camera relative to the vehicle camera, including:

[0012] Measure the vertical distances between the four calibration points and the vehicle-mounted camera;

[0013] Measure the lengths of the two line segments that intersect the two lane lines when the four calibration points form a quadrilateral; measure the distances between the wheels on both sides of the vehicle and the corresponding lane lines;

[0014] Measure the height of the vehicle-mounted camera from the ground;

[0015] Measure the offset distance between the vehicle camera and the center line of the vehicle body.

[0016] Optionally, the four calibration points are points with obvious texture features and / or manually marked points, and the four calibration points form a rectangle.

[0017] Optionally, determining pixel coordinates of four calibration points on the image includes:

[0018] The image is dedistorted by using the internal reference information of the vehicle-mounted camera to obtain a dedistorted image; and the pixel coordinates of the four calibration points on the dedistorted image are determined.

[0019] Optionally, the external parameter information of the vehicle-mounted camera is determined according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image, including:

[0020] Obtain a pre-established mapping relationship between world coordinates and pixel coordinates;

[0021] Substitute the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image into the mapping relationship for calculation to obtain the rotation parameters and translation parameters of the vehicle-mounted camera.

[0022] Optionally, also include:

[0023] Based on the external parameter information of the vehicle-mounted camera, the calculated coordinates of the preset ground points on the image taken by the vehicle-mounted camera in the vehicle coordinate system are calculated;

[0024] Get the real coordinates of the preset ground point in the vehicle coordinate system;

[0025] Calculate the error value between the calculated coordinates of the preset ground point in the vehicle coordinate system and the real coordinates of the preset ground point in the vehicle coordinate system;

[0026] When the error value is less than a preset error threshold, it is determined that the external parameter information verification has passed; otherwise, it is determined that the external parameter information verification has failed.

[0027] Optionally, based on the external parameter information of the vehicle-mounted camera, calculating the calculated coordinates of a preset ground point on the image captured by the vehicle-mounted camera in the vehicle coordinate system includes:

[0028] Determine the pixel coordinates of the preset ground point on the image captured by the vehicle camera;

[0029] Substituting the determined pixel coordinates and external parameter information into the mapping relationship between the world coordinates and the pixel coordinates for calculation, the coordinates of the preset ground point in the world coordinate system are obtained;

[0030] The calculated coordinates of the preset ground point in the vehicle coordinate system are calculated based on the offset between the world coordinates and the vehicle coordinates, and the coordinates of the preset ground point in the world coordinate system.

[0031] Optionally, based on the external parameter information of the vehicle-mounted camera, calculating the calculated coordinates of a preset ground point on the image captured by the vehicle-mounted camera in the vehicle coordinate system includes:

[0032] Determine the pixel coordinates of the preset ground point on the image captured by the vehicle camera;

[0033] Based on the determined pixel coordinates, the calculated coordinates of the preset ground point in the vehicle coordinate system are obtained by querying from a pre-established lookup table;

[0034] The process of establishing the lookup table includes:

[0035] Determine the pixel coordinates of all ground points on the image taken by the vehicle camera respectively;

[0036] Substitute the determined pixel coordinates and external parameter information into the mapping relationship between world coordinates and pixel coordinates for calculation, and obtain the coordinates of all ground points in the world coordinate system;

[0037] According to the offset between the world coordinates and the vehicle coordinates, and the coordinates of all ground points in the world coordinate system, the calculated coordinates of all ground points in the vehicle coordinate system are calculated;

[0038] A lookup table is constructed based on the calculated coordinates of all ground points in the vehicle coordinate system and the pixel coordinates of all ground points in the image taken by the vehicle camera.

[0039] Optionally, calculating the error value between the calculated coordinates of the preset ground point in the vehicle coordinate system and the real coordinates of the preset ground point in the vehicle coordinate system includes:

[0040] Calculate a first distance difference between the x-axis calculated coordinates of the preset ground point in the vehicle coordinate system and the x-axis real coordinates, and calculate a second distance difference between the y-axis calculated coordinates of the preset ground point in the vehicle coordinate system and the y-axis real coordinates;

[0041] The first distance difference value is divided by the true x-axis coordinate, and the quotient is used as the first error value, and the second distance difference value is divided by the true y-axis coordinate, and the quotient is used as the second error value.

[0042] A camera extrinsic parameter calibration device, comprising:

[0043] A position information measurement unit, used to measure the position information of four calibration points within the field of view of the vehicle-mounted camera relative to the vehicle-mounted camera, wherein the four calibration points are points located on two lane lines and the lines connecting two of them can form a quadrilateral;

[0044] A world coordinate determination unit, used to determine the coordinates of the four calibration points in the world coordinate system according to the position information of the four calibration points relative to the vehicle-mounted camera;

[0045] A pixel coordinate determination unit, used to obtain an image taken by the vehicle-mounted camera of the four calibration points, and determine the pixel coordinates of the four calibration points on the image respectively;

[0046] The external parameter information determination unit is used to determine the external parameter information of the vehicle-mounted camera according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

[0047] Through the above technical solutions, it can be known that the camera extrinsic parameter calibration method provided by the present application sets four calibration points that can be connected in pairs to form a quadrilateral on the two lane lines within the field of view of the vehicle-mounted camera in advance. By measuring the position information of the four calibration points relative to the vehicle-mounted camera, the coordinates of the four calibration points in the world coordinate system can be calculated. By taking pictures of the four calibration points by the vehicle-mounted camera, the pixel coordinates of the four calibration points on the captured image can be obtained. Combining the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the captured image, the extrinsic parameter information of the vehicle-mounted camera can be determined. The present application only needs to simply set four calibration points to obtain the camera extrinsic parameters by calculation. The calculation process is simple. The calibration process of the present application only needs to use the two lane lines on the road, and no calibration plate is required, which reduces the requirements for the calibration environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of a flow chart of a camera extrinsic calibration method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of a calibration scenario including two lane lines provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of four calibration points and position information required to be measured provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of a flow chart of another camera extrinsic calibration method provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of the structure of a camera extrinsic calibration device provided in an embodiment of the present application;

[0054] Figure 6 A hardware structure block diagram of a camera extrinsic calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] The present application provides a camera extrinsic parameter calibration method, and the camera extrinsic parameter calibration method provided by the present application is described in detail through the following embodiments.

[0057] See also Figure 1 , shows a schematic flow chart of a camera extrinsic calibration method provided in an embodiment of the present application, and the camera extrinsic calibration method may include:

[0058] Step S101: measuring position information of four calibration points within the field of view of the vehicle-mounted camera relative to the vehicle-mounted camera, wherein the four calibration points are points located on two lane lines and lines connecting any two of them can form a quadrilateral.

[0059] The vehicle-mounted camera in this application refers to a vehicle-mounted front-view camera.

[0060] Specifically, this application can be found in Figure 2 The calibration is performed in the calibration scene shown in the figure, which includes two lane lines. Before calibration, four calibration points need to be set on the two lane lines on the road in advance. The four calibration points need to be within the field of view of the vehicle camera and the lines connecting any two of them can form a quadrilateral, that is, every two calibration points in the four calibration plates are located on one lane line.

[0061] The present application can measure the position information of the four calibration points relative to the vehicle-mounted camera, so as to determine the coordinates of the four calibration points in the world coordinate system based on the measured position information.

[0062] Optionally, the four calibration points can be points with obvious texture features, for example, the intersection of the inner edge of the lane line and the cement joint can be used as the calibration point; optionally, the four calibration points can also be manually marked points, for example, when the two lane lines on the ground do not have obvious texture features as calibration points, marks can be manually drawn on the ground.

[0063] Of course, the four calibration points may be selected in other ways, as long as the four calibration points can form a quadrilateral and their pixel coordinate positions can be clearly found in the image within the field of view of the vehicle-mounted camera. This application does not make any specific limitations on this.

[0064] Step S102: Determine the coordinates of the four calibration points in the world coordinate system according to the position information of the four calibration points relative to the vehicle-mounted camera.

[0065] Specifically, the present application can perform calculations based on the position information of the four calibration points relative to the vehicle-mounted camera to obtain the coordinates of the four calibration points in the world coordinate system.

[0066] Optionally, for the convenience of calculation, the present application may make the vehicle where the onboard camera is located parallel to the two lane lines and in the center of the two lane lines, such as Figure 2 As shown; Optionally, the present application can also connect the above four calibration points in pairs to form a rectangle, for example, see Figure 3 The four calibration points A, B, C, and D shown in FIG. 5 are connected in pairs to form a rectangle, thereby simplifying the calculation.

[0067] Optional, such as Figure 3 As shown in the figure, the black rectangle represents the on-board camera installed on the vehicle. The position information of the four calibration points measured in the above steps relative to the on-board camera includes: the vertical distances between the four calibration points and the on-board camera (taking calibration point A as an example, the vertical distance between calibration point A and the on-board camera is dA; similarly, the vertical distances measured in this application also include dB, dC and dD), the lengths of the two line segments intersecting the two lane lines when the four calibration points form a quadrilateral (i.e. Figure 3 The lengths of the AB and CD segments shown in the figure), the distances between the wheels on both sides of the vehicle and the corresponding lane lines (i.e. Figure 3 d1 and d2 shown), the height of the vehicle-mounted camera from the ground (denoted as h in this application), the offset distance of the vehicle-mounted camera from the center line of the vehicle body (i.e. Figure 3 ΔX shown).

[0068] Then, this step can be calculated based on dA, dB, dC, dD, AB segment length, CD segment length, d1, d2, h and ΔX measured in the previous step to obtain the coordinates of the four calibration points in the world coordinate system.

[0069] Step S103: Obtain images of the four calibration points taken by the vehicle-mounted camera, and determine the pixel coordinates of the four calibration points on the image respectively.

[0070] Camera extrinsic calibration requires obtaining the coordinates of the calibration points in the world coordinate system and the pixel coordinate system at the same time. To this end, the embodiment of the present application also requires the on-board camera to take images of the four calibration points and obtain the pixel coordinates of the four calibration points on the image.

[0071] Step S104: Determine the external parameter information of the vehicle-mounted camera according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

[0072] Specifically, in the embodiment of the present application, when the intrinsic parameters of the camera are known, the extrinsic parameter information of the vehicle-mounted camera can be obtained by calculation based on the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

[0073] Here, the external parameter information of the vehicle-mounted camera includes a rotation parameter R and a translation parameter T.

[0074] The camera extrinsic parameter calibration method provided by the present application sets four calibration points that can be connected in pairs to form a quadrilateral on the two lane lines within the field of view of the vehicle-mounted camera in advance. The coordinates of the four calibration points in the world coordinate system can be calculated by measuring the position information of the four calibration points relative to the vehicle-mounted camera. The four calibration points are photographed by the vehicle-mounted camera to obtain the pixel coordinates of the four calibration points on the captured image. The extrinsic parameter information of the vehicle-mounted camera can be determined by combining the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the captured image. The present application only needs to set four calibration points to obtain the camera extrinsic parameters by calculation. The calculation process is simple. The calibration process of the present application only needs to use the two lane lines on the road, and no calibration plate is required, which reduces the requirements for the calibration environment.

[0075] In an optional embodiment, considering that the image taken by the vehicle-mounted camera may be distorted, resulting in errors in the coordinates of several pixel points on the image, in order to obtain the accurate pixel coordinates of the four calibration points, the present application can dedistort the image through the internal reference information of the vehicle-mounted camera when determining the pixel coordinates of the four calibration points on the image, to obtain the dedistorted image, and then determine the pixel coordinates of the four calibration points on the dedistorted image.

[0076] The formula for “dedistorting the image using the internal reference information of the vehicle-mounted camera” is shown in the following formula (1).

[0077]

[0078] Where (x, y) represents the pixel coordinates of the calibrated point on the image before distortion removal, r 2 =x 2 +y 2 , represents the square of the distance between the calibration point and the origin of the coordinate system on the image before distortion removal, k1, k2, k3, p1 and p2 are the internal reference information of the vehicle-mounted camera, among which k1, k2 and k3 are the radial distortion coefficients of the camera, p1 and p2 are the tangential distortion parameters of the camera, (x corrected ,y corrected ) are the pixel coordinates of the calibrated points on the dedistorted image.

[0079] The pixel coordinates of the four calibration points on the dedistorted image can be calculated by the above formula (1). The pixel coordinates are used as the pixel coordinates for calculating the camera extrinsic parameters in step S104 in the embodiment of the present application.

[0080] On this basis, the process of "step S104, determining the external parameter information of the vehicle-mounted camera according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image" may include:

[0081] Step S1041: Acquire a pre-established mapping relationship between world coordinates and pixel coordinates.

[0082] Here, the world coordinates refer to the coordinates in the world coordinate system, and the pixel coordinates refer to the coordinates in the pixel coordinate system. The mapping relationship obtained in this step can be found in formula (2).

[0083]

[0084] In the formula, z c is a scale factor, which indicates the distance from the center of the pixel coordinate system, i.e., the optical center, to the origin of the camera coordinate system; (u, v) indicates the pixel coordinates. In the embodiment of the present application, u specifically refers to the aforementioned x corrected , v specifically refers to the aforementioned y corrected ; dx, dy, f, u0 and v0 are all internal parameters of the vehicle-mounted camera and are known a priori. Among them, (u0, v0) is the coordinate of the center point of the image in the pixel coordinate system, and f x and f y are the image distances in the x and y directions of the pixel coordinate system, respectively; f represents the focal length; dx and dy represent the actual physical lengths of one pixel in the x and y directions (how many meters one pixel corresponds to); (X w , Y w , Z w ) represents the coordinates in the world coordinate system, that is, the coordinate information calculated from the distance information of the measuring point, where Zw=0 of the measuring point.

[0085] Step S1042: Substitute the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image into the mapping relationship for calculation to obtain the rotation parameters and translation parameters of the vehicle-mounted camera.

[0086] The coordinates of the calibration point in the world coordinate system refer to (X w Y w , Z w ), the pixel coordinates of the calibration point on the image refer to (u, v) in formula (2). This application substitutes the specific coordinate values ​​into formula (2) and can be used to calculate the pixel coordinates of the calibration point on the image when f is known. x 、f y , u0 and v0, the rotation parameter R and translation parameter T are calculated.

[0087] In another optional embodiment, the camera extrinsic parameter calibration method provided in the embodiment of the present application can be implemented by a computer program. After the position information is measured in the aforementioned step S101, the measured position information can be input into the configuration file of the computer program for extrinsic parameter calibration, and in step S103, after obtaining the pixel coordinates of the four calibration points on the image respectively, they are input into the corresponding positions of the corresponding variables in the configuration file of the computer program for extrinsic parameter calibration. The computer program can execute step S102 to obtain the coordinates of the four calibration points in the world coordinate system respectively, and then execute step S104 to obtain the extrinsic parameter information of the vehicle-mounted camera.

[0088] The camera extrinsic calibration method provided in this application can be implemented only with the information of lane markings on the ground road, avoiding the use of a large-area calibration plate, greatly reducing the requirements for the calibration environment, and improving the flexibility and convenience of calibration. Ordinary workers rather than professional calibration personnel can also perform the operation, reducing a certain amount of labor costs.

[0089] Taking into account the possibility of inaccurate calibration in the aforementioned calibration process, the embodiment of the present application further provides a method for verifying the external parameter calibration result to verify the reliability of the external parameter calibration result.

[0090] For this purpose, see Figure 4 As shown, the camera extrinsic calibration method provided in the embodiment of the present application may further include the following steps based on the aforementioned steps S101 to S104:

[0091] Step S105: Calculate the coordinates of a preset ground point on the image captured by the vehicle camera in the vehicle coordinate system based on the external parameter information of the vehicle camera.

[0092] Here, the image captured by the vehicle-mounted camera may be the image containing the four calibration points in the aforementioned step S103, or may be other images captured by the vehicle-mounted camera.

[0093] It can be understood that the image taken by the vehicle-mounted camera contains the ground. In this embodiment, several ground points can be selected from the image taken by the vehicle-mounted camera to verify the external parameter calibration results. For the convenience of description, the selected several ground points are defined as preset ground points. For example, the preset ground points can be the four calibration points mentioned above.

[0094] This step can calculate the coordinates of the preset ground point on the image captured by the vehicle camera in the vehicle coordinate system based on the external parameter information of the vehicle camera. In order to distinguish it from the "real coordinates of the preset ground point in the vehicle coordinate system" in the following steps, the coordinates calculated in this step are defined as calculated coordinates. Here, the coordinates in the vehicle coordinate system are world coordinates with the camera as the origin.

[0095] This step of "calculating the coordinates of the preset ground points on the image taken by the vehicle camera in the vehicle coordinate system based on the external parameter information of the vehicle camera" can be implemented in many ways, and the following two ways are provided here but are not limited to.

[0096] The first method is direct calculation, which includes the following steps:

[0097] Step S11: Determine the pixel coordinates of a preset ground point on the image captured by the vehicle-mounted camera.

[0098] The process of this step corresponds to the process of determining the pixel coordinates of the calibration point on the image in the aforementioned step S103. For details, please refer to the introduction in the aforementioned embodiment, which will not be repeated here.

[0099] Step S12: Substitute the determined pixel coordinates and external parameter information into the mapping relationship between the world coordinates and the pixel coordinates for calculation to obtain the coordinates of the preset ground point in the world coordinate system.

[0100] Specifically, the determined pixel coordinates and external parameter information can be substituted into the aforementioned formula (2) to obtain the coordinates of the preset ground point in the world coordinate system.

[0101] Step S13: Calculate the calculated coordinates of the preset ground point in the vehicle coordinate system according to the offset between the world coordinates and the vehicle coordinates, and the coordinates of the preset ground point in the world coordinate system.

[0102] Here, the vehicle coordinates refer to the coordinates in the vehicle coordinate system. In this step, the offset between the world coordinates and the vehicle coordinates can be obtained in advance, and then the calculated coordinates of the preset ground point in the vehicle coordinate system are calculated based on the coordinates of the preset ground point in the world coordinate system and the obtained offset.

[0103] The second method is a table lookup method, which specifically includes the following steps:

[0104] Step S21: Determine the pixel coordinates of a preset ground point on the image captured by the vehicle-mounted camera.

[0105] The process of this step corresponds to the process of determining the pixel coordinates of the calibration point on the image in the aforementioned step S103. For details, please refer to the introduction in the aforementioned embodiment, which will not be repeated here.

[0106] Step S22: Based on the determined pixel coordinates, the calculated coordinates of the preset ground point in the vehicle coordinate system are obtained by querying from a pre-established lookup table.

[0107] Specifically, in this step, a lookup table may be pre-established with the pixel coordinates of the ground point on the image captured by the vehicle camera as the index and the calculated coordinates of the ground point in the vehicle coordinate system as the index content. After the lookup table is established, the calculated coordinates of the preset ground point in the vehicle coordinate system are obtained by querying the lookup table based on the pixel coordinates determined in step S21.

[0108] In this step, the process of establishing the lookup table includes: determining the pixel coordinates of all ground points on the image taken by the vehicle camera respectively; substituting the determined pixel coordinates and external parameter information into the mapping relationship between the world coordinates and the pixel coordinates for calculation, and obtaining the coordinates of all ground points in the world coordinate system respectively; according to the offset between the world coordinates and the vehicle coordinates, and the coordinates of all ground points in the world coordinate system respectively, calculating the calculated coordinates of all ground points in the vehicle coordinate system respectively; based on the calculated coordinates of all ground points in the vehicle coordinate system respectively, and the pixel coordinates of all ground points in the image taken by the vehicle camera respectively, construct a lookup table.

[0109] Step S106: Obtain the real coordinates of the preset ground point in the vehicle coordinate system.

[0110] The process of obtaining the real coordinates of the preset ground point in the vehicle coordinate system in this step is prior art and will not be described in detail here.

[0111] Step S107, calculating the error value between the calculated coordinates of the preset ground point in the vehicle coordinate system and the real coordinates of the preset ground point in the vehicle coordinate system.

[0112] Optionally, the process of this step may include: calculating a first distance difference between the calculated x-axis coordinates and the true x-axis coordinates of the preset ground point in the vehicle coordinate system, and calculating a second distance difference between the calculated y-axis coordinates and the true y-axis coordinates of the preset ground point in the vehicle coordinate system; dividing the first distance difference by the true x-axis coordinates, and taking the quotient as the first error value, and dividing the second distance difference by the true y-axis coordinates, and taking the quotient as the second error value.

[0113] It is worth noting that since the present application selects a preset ground point, its z-axis coordinate in the vehicle coordinate system is 0, therefore, both the z-axis calculated coordinate and the z-axis real coordinate are 0.

[0114] Take an example to illustrate the process of calculating the error value: assuming that the calculated coordinates of the preset ground point in the vehicle coordinate system are (10.5, 11.2, 0), and the real coordinates are (10, 11, 0), then the first error value is Δx = |10.5-10| / 10*100% = 5%, and the second error value is Δy = |11.2-11| / 11*100% = 1.8%.

[0115] Step S108: When the error value is less than a preset error threshold, it is determined that the external parameter information verification has passed; otherwise, it is determined that the external parameter information verification has failed.

[0116] The above error threshold can be set according to actual conditions and is not specifically limited in this application.

[0117] Taking the error threshold of 2% as an example, if the error value is less than 2%, it means that the calibration result of the camera extrinsic parameter is relatively reliable, and it is determined that the extrinsic parameter information verification has passed; conversely, if the error value is greater than or equal to 2%, it means that the calibration result of the camera extrinsic parameter is unreliable, and it is determined that the extrinsic parameter information verification has not passed. Since the aforementioned step S107 calculates two error values, they need to be compared with the error threshold of 2% respectively. Only when both error values ​​are less than the error threshold of 2%, it is determined that the extrinsic parameter information verification has passed.

[0118] For example, in the aforementioned example, Δx=5%, which is greater than the error threshold of 2%. At this time, it is determined that the external parameter information verification has failed.

[0119] Optionally, in this embodiment, if the external parameter information verification passes, the calibrated external parameter information can be transmitted to the downstream. Conversely, if the external parameter information verification fails, a warning is prompted to the user indicating that the calibration result cannot meet the application standard.

[0120] This embodiment combines calibration and verification decisions together, eliminates data with large calibration errors, and improves the accuracy of the calibration results.

[0121] An embodiment of the present application also provides a camera extrinsic parameter calibration device. The camera extrinsic parameter calibration device provided in the embodiment of the present application is described below. The camera extrinsic parameter calibration device described below and the camera extrinsic parameter calibration method described above can be referenced to each other.

[0122] See also Figure 5 , which shows a schematic diagram of the structure of a camera extrinsic calibration device provided in an embodiment of the present application, such as Figure 5As shown, the camera extrinsic parameter calibration device may include: a position information measuring unit 501, a world coordinate determining unit 502, a pixel coordinate determining unit 503 and an extrinsic parameter information determining unit 504.

[0123] The position information measuring unit 501 is used to measure the position information of four calibration points within the field of view of the vehicle-mounted camera relative to the vehicle-mounted camera, wherein the four calibration points are points located on two lane lines and the lines connecting the four points can form a quadrilateral.

[0124] The world coordinate determination unit 502 is used to determine the coordinates of the four calibration points in the world coordinate system according to the position information of the four calibration points relative to the vehicle-mounted camera.

[0125] The pixel coordinate determining unit 503 is used to obtain the image captured by the vehicle-mounted camera of the four calibration points, and determine the pixel coordinates of the four calibration points on the image respectively.

[0126] The external parameter information determination unit 504 is used to determine the external parameter information of the vehicle-mounted camera according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

[0127] The working principle of the camera extrinsic parameter calibration device provided in the present application corresponds to the working principle of the aforementioned camera extrinsic parameter calibration method. For details, please refer to the introduction in the aforementioned embodiment, and will not be repeated here.

[0128] The embodiment of the present application also provides a camera extrinsic calibration device. Optionally, Figure 6 The hardware structure diagram of the camera external parameter calibration device is shown in FIG. Figure 6 , the hardware structure of the camera extrinsic calibration device may include: at least one processor 601, at least one communication interface 602, at least one memory 603 and at least one communication bus 604;

[0129] In the embodiment of the present application, the number of the processor 601, the communication interface 602, the memory 603, and the communication bus 604 is at least one, and the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604;

[0130] The processor 601 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0131] The memory 603 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0132] The memory 603 stores a program, and the processor 601 can call the program stored in the memory 603, and the program is used to:

[0133] Measure the position information of four calibration points within the field of view of the vehicle camera relative to the vehicle camera, wherein the four calibration points are points located on the two lane lines and the lines connecting any two of them can form a quadrilateral;

[0134] According to the position information of the four calibration points relative to the vehicle-mounted camera, the coordinates of the four calibration points in the world coordinate system are determined;

[0135] Obtain the images taken by the vehicle-mounted camera of the four calibration points, and determine the pixel coordinates of the four calibration points on the image respectively;

[0136] The external parameter information of the vehicle-mounted camera is determined according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

[0137] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0138] An embodiment of the present application further provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the camera extrinsic parameter calibration method as described above is implemented.

[0139] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0140] Finally, it should be noted that, in this article, relational terms such as and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprises a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0141] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0142] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A camera extrinsic calibration method, characterized in that: include: Measuring position information of four calibration points within the field of view of the vehicle-mounted camera relative to the vehicle-mounted camera, wherein the four calibration points are points located on two lane lines and two of the four calibration points can form a quadrilateral when connected in pairs; Determine the coordinates of the four calibration points in the world coordinate system according to the position information of the four calibration points relative to the vehicle-mounted camera; Acquire an image of the four calibration points taken by the vehicle-mounted camera, and determine the pixel coordinates of the four calibration points on the image respectively; The external parameter information of the vehicle-mounted camera is determined according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

2. The camera extrinsic calibration method according to claim 1, characterized in that: The vehicle where the on-board camera is located is parallel to the two lane lines and is located in the center of the two lane lines; The measuring of the position information of four calibration points within the field of view of the vehicle-mounted camera relative to the vehicle-mounted camera includes: Measuring the vertical distances between the four calibration points and the vehicle-mounted camera respectively; Measuring the lengths of two line segments intersecting the two lane lines when the four calibration points form a quadrilateral; Measuring the distances between the wheels on both sides of the vehicle and the corresponding side lane lines respectively; Measuring the height of the vehicle-mounted camera from the ground; The offset distance of the vehicle-mounted camera from the center line of the vehicle body is measured.

3. The camera extrinsic calibration method according to claim 1, characterized in that: The four calibration points are points with obvious texture features and / or manually marked points, and the four calibration points form a rectangle.

4. The camera extrinsic calibration method according to claim 1, characterized in that: Determining the pixel coordinates of the four calibration points on the image respectively includes: Dedistorting the image using the internal reference information of the vehicle-mounted camera to obtain a dedistorted image; Determine the pixel coordinates of the four calibration points on the dedistorted image.

5. The camera extrinsic calibration method according to claim 1, characterized in that: The determining the external parameter information of the vehicle-mounted camera according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image respectively includes: Obtain a pre-established mapping relationship between world coordinates and pixel coordinates; The coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image are respectively substituted into the mapping relationship for calculation to obtain the rotation parameters and translation parameters of the vehicle-mounted camera.

6. The camera extrinsic calibration method according to any one of claims 1 to 5, characterized in that: Also includes: Based on the external parameter information of the vehicle-mounted camera, calculating the calculated coordinates of the preset ground point on the image taken by the vehicle-mounted camera in the vehicle coordinate system; Obtaining the real coordinates of the preset ground point in the vehicle coordinate system; Calculating the calculated coordinates of the preset ground point in the vehicle coordinate system and the error value between the real coordinates of the preset ground point in the vehicle coordinate system; When the error value is less than a preset error threshold, it is determined that the external parameter information verification has passed; otherwise, it is determined that the external parameter information verification has failed.

7. The camera extrinsic calibration method according to claim 6, characterized in that: The calculating, based on the external parameter information of the vehicle-mounted camera, the calculated coordinates of the preset ground point on the image captured by the vehicle-mounted camera in the vehicle coordinate system includes: Determine the pixel coordinates of the preset ground point on the image captured by the vehicle-mounted camera; Substituting the determined pixel coordinates and the external parameter information into the mapping relationship between the world coordinates and the pixel coordinates for calculation, to obtain the coordinates of the preset ground point in the world coordinate system; The calculated coordinates of the preset ground point in the vehicle coordinate system are calculated based on the offset between the world coordinates and the vehicle coordinates, and the coordinates of the preset ground point in the world coordinate system.

8. The camera extrinsic calibration method according to claim 6, characterized in that: The calculating, based on the external parameter information of the vehicle-mounted camera, the calculated coordinates of the preset ground point on the image captured by the vehicle-mounted camera in the vehicle coordinate system includes: Determine the pixel coordinates of the preset ground point on the image captured by the vehicle-mounted camera; Based on the determined pixel coordinates, querying from a pre-established lookup table to obtain the calculated coordinates of the preset ground point in the vehicle coordinate system; The process of establishing the lookup table includes: Determine the pixel coordinates of all ground points on the image taken by the vehicle-mounted camera on the image taken by the vehicle-mounted camera; Substituting the determined pixel coordinates and the external parameter information into the mapping relationship between the world coordinates and the pixel coordinates for calculation, to obtain the coordinates of all the ground points in the world coordinate system; Calculating the calculated coordinates of all the ground points in the vehicle coordinate system according to the offset between the world coordinates and the vehicle coordinates and the coordinates of all the ground points in the world coordinate system; The lookup table is constructed based on the calculated coordinates of all the ground points in the vehicle coordinate system and the pixel coordinates of all the ground points in the image captured by the vehicle camera.

9. The camera extrinsic calibration method according to claim 6, characterized in that: The calculating of the calculated coordinates of the preset ground point in the vehicle coordinate system and the error value between the real coordinates of the preset ground point in the vehicle coordinate system includes: Calculating a first distance difference between the x-axis calculated coordinates of the preset ground point in the vehicle coordinate system and the x-axis real coordinates, and calculating a second distance difference between the y-axis calculated coordinates of the preset ground point in the vehicle coordinate system and the y-axis real coordinates; The first distance difference value is divided by the actual x-axis coordinate, and the quotient is used as a first error value; and the second distance difference value is divided by the actual y-axis coordinate, and the quotient is used as a second error value.

10. A camera extrinsic calibration device, characterized in that: include: A position information measuring unit, used to measure the position information of four calibration points within the field of view of the vehicle-mounted camera relative to the vehicle-mounted camera, wherein the four calibration points are points located on two lane lines and the lines connecting the four points can form a quadrilateral; A world coordinate determination unit, used to determine the coordinates of the four calibration points in the world coordinate system according to the position information of the four calibration points relative to the vehicle-mounted camera; A pixel coordinate determination unit, used for acquiring an image taken by the vehicle-mounted camera of the four calibration points, and determining the pixel coordinates of the four calibration points on the image respectively; The external parameter information determination unit is used to determine the external parameter information of the vehicle-mounted camera according to the coordinates of the four calibration points in the world coordinate system and the pixel coordinates on the image.

Citation Information

Patent Citations

  • Method for calibrating external parameters of automobile panoramic camera based on lane line

    CN107103627A

  • On-line calibration method, system and device for vehicle-mounted look-around system

    CN111462244A

  • Vehicle-mounted camera automatic calibration method based on dash lane line

    CN112102413A

  • Automatic driving vehicle-mounted camera pose estimation method and system based on lane line

    CN112927303A

  • Vehicle-mounted panoramic camera calibration method and device based on lane line and storage medium

    CN113963066A