A method for calibrating the extrinsic parameters of a vehicle front-view camera

Abstract

The application discloses a method for calibrating the external parameters of a vehicle front-view camera, which comprises the following steps: acquiring the attitude parameters of the vehicle camera; aiming the vehicle camera at a plurality of marker points according to the attitude parameters to obtain corresponding marker point data; and fitting the marker point data based on the least square method to obtain the position information of the vehicle camera. The camera parameter calibration is realized from a special site specially set for camera parameter calibration to a general site, namely, the attitude parameters of the camera are directly acquired, then the camera is aimed at the marker blocks at different positions, and the position of the camera is calibrated based on the least square method, so that the technical problem of high cost of the external parameter calibration of the vehicle camera is solved, the requirements of the calibration work on the environment and the site are effectively reduced, and the external parameter calibration process of the vehicle camera does not need to make and place the checkerboard.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, and particularly relates to a method for calibrating external parameters of a front-view camera of a vehicle. BACKGROUND

[0002] At present, with the development of intelligent driving technology, cameras are increasingly widely applied on vehicles. A vehicle performs environment perception through a camera installed on the vehicle after calibrating internal and external parameters of the camera.

[0003] The most commonly used method is Zhang Zhengyou camera calibration method, which can conveniently calibrate internal parameters of a camera by shooting a chessboard of a certain size. When calibrating external parameters of the camera, that is, calibrating the position and posture of the camera in the vehicle coordinate system, the position and posture of the chessboard in the vehicle coordinate system need to be determined first.

[0004] However, when the Zhang Zhengyou camera calibration method is used to calibrate external parameters, a special camera calibration site is needed, the chessboard of a certain size and position needs to be placed, and good lighting conditions need to be ensured for recognition of the chessboard, which leads to high cost of calibration of external parameters of the vehicle camera and is inconvenient to implement. SUMMARY

[0005] The main purpose of the present application is to provide a method for calibrating external parameters of a front-view camera of a vehicle, which aims to solve the technical problem of high cost of calibration of external parameters of a vehicle camera, effectively reduces the requirements of the calibration work on the environment and site, and makes the calibration process of external parameters of the vehicle camera not need to make and place a chessboard.

[0006] To achieve the above purpose, the present application provides a method for calibrating external parameters of a front-view camera of a vehicle, which comprises the following steps:

[0007] obtaining posture parameters of a vehicle camera;

[0008] aiming the vehicle camera at preset mark points at different positions according to the posture parameters to obtain corresponding mark point data;

[0009] performing fitting on the mark point data based on a least square method to obtain position information of the vehicle camera.

[0010] Optionally, the step of aiming the vehicle camera at preset mark points at different positions according to the posture parameters to obtain corresponding mark point data comprises the following steps:

[0011] determining distances between a target vehicle to which the vehicle camera belongs and the mark points;

[0012] adjusting a direction of the vehicle-mounted camera to aim at the plurality of mark points through a preset display picture to obtain a corresponding pitch angle;

[0013] obtaining the mark point data based on the distance and a cotangent value of the pitch angle.

[0014] Optionally, the step of determining the distance between the target vehicle to which the vehicle-mounted camera belongs and the mark point comprises:

[0015] respectively acquiring a front axle center, a forward direction, a vertical direction and a preset right-hand rule of the target vehicle to determine an origin, a horizontal axis, a vertical axis and a longitudinal axis to obtain a target vehicle coordinate system;

[0016] determining the distance according to a position of the mark point in the target vehicle coordinate system.

[0017] Optionally, the vehicle-mounted camera is integrated with an acceleration sensor, and the display picture comprises a picture center point, and the step of adjusting the direction of the vehicle-mounted camera to aim at the plurality of mark points through the preset display picture to obtain the corresponding pitch angle comprises:

[0018] adjusting the picture center point to coincide with the mark point and adjusting a component of the acceleration sensor on the horizontal axis to be zero to determine an adjusted camera;

[0019] respectively reading components of the acceleration sensor on the vertical axis and the longitudinal axis based on the adjusted camera to obtain a vertical axis component and a longitudinal axis component;

[0020] obtaining the pitch angle based on an angle relationship of the vertical axis component and the longitudinal axis component.

[0021] Optionally, the step of obtaining the mark point data based on the distance and the cotangent value of the pitch angle comprises:

[0022] taking the pitch angle as an independent variable and the distance as a dependent variable to obtain a vertical axis coordinate on the vertical axis and a horizontal axis coordinate on the horizontal axis;

[0023] obtaining the mark point data based on the vertical axis coordinate and the horizontal axis coordinate.

[0024] Optionally, after the step of fitting the mark point data based on the least square method to obtain the position information of the vehicle-mounted camera, the method further comprises:

[0025] detecting whether a longitudinal axis coordinate of the vehicle-mounted camera on the longitudinal axis is zero;

[0026] If the vehicle-mounted camera has a longitudinal axis coordinate of zero along the longitudinal axis, the spatial position information of the vehicle-mounted camera is obtained in combination with the longitudinal axis coordinate and the position information.

[0027] Optionally, the step of obtaining the attitude parameter of the vehicle-mounted camera comprises:

[0028] The attitude parameter of the vehicle-mounted camera is obtained by the acceleration sensor.

[0029] Embodiments of the present application further provide a device for calibrating an extrinsic parameter of a vehicle front-view camera, which comprises:

[0030] a parameter obtaining module, configured to obtain an attitude parameter of a vehicle-mounted camera;

[0031] a landmark aiming module, configured to aim the vehicle-mounted camera at preset landmarks at different positions respectively according to the attitude parameter, to obtain corresponding landmark data;

[0032] a data fitting module, configured to fit the landmark data based on a least square method, to obtain position information of the vehicle-mounted camera.

[0033] Embodiments of the present application further provide a vehicle, which comprises a memory, a processor, and a program for calibrating an extrinsic parameter of a vehicle front-view camera stored in the memory and capable of running on the processor. When the program for calibrating an extrinsic parameter of a vehicle front-view camera is executed by the processor, steps of a method for calibrating an extrinsic parameter of a vehicle front-view camera are implemented.

[0034] Embodiments of the present application further provide a computer readable storage medium, which stores a program for calibrating an extrinsic parameter of a vehicle front-view camera. When the program for calibrating an extrinsic parameter of a vehicle front-view camera is executed by a processor, steps of a method for calibrating an extrinsic parameter of a vehicle front-view camera are implemented.

[0035] This application proposes a method for calibrating the extrinsic parameters of a vehicle's forward-looking camera. The method involves acquiring the attitude parameters of the vehicle-mounted camera; aiming the camera at several marker points based on these attitude parameters to obtain corresponding marker point data; and fitting the marker point data using the least squares method to obtain the position information of the vehicle-mounted camera. This method moves from a specially designed calibration site to a general setting for camera parameter calibration. Specifically, it directly acquires the camera's attitude parameters, aims the camera at marker blocks at different locations, and calibrates the camera's position using the least squares method. This solves the technical problem of high cost in extrinsic parameter calibration of vehicle-mounted cameras, effectively reducing the environmental and site requirements for calibration work, and eliminating the need to create and arrange a checkerboard pattern during the calibration process. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the functional modules of the vehicle to which the device used in this application for calibrating the external parameters of the vehicle's forward-view camera belongs;

[0037] Figure 2 This is a flowchart illustrating a first exemplary embodiment of a method for calibrating extrinsic parameters of a vehicle's forward-looking camera according to this application.

[0038] Figure 3 This is a schematic diagram of a method for calibrating the extrinsic parameters of a vehicle's forward-looking camera, as described in this application, involving multiple measurements to fit a straight line.

[0039] Figure 4 This is a flowchart illustrating a second exemplary embodiment of a method for calibrating extrinsic parameters of a vehicle's forward-looking camera according to this application;

[0040] Figure 5 This is a schematic diagram of the vehicle coordinate system construction in a method for calibrating the extrinsic parameters of a vehicle's forward-looking camera according to this application;

[0041] Figure 6 This is a schematic diagram of multiple measurements for a method of calibrating the external parameters of a vehicle's forward-looking camera according to this application.

[0042] Figure 7 This is a flowchart illustrating a third exemplary embodiment of a method for calibrating extrinsic parameters of a vehicle's forward-looking camera according to this application.

[0043] Figure 8 This is a flowchart illustrating a fourth exemplary embodiment of a method for calibrating extrinsic parameters of a vehicle's forward-looking camera according to this application.

[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0045] It should be understood that the specific embodiments described herein are merely exemplary and do not limit the application.

[0046] The main solution of the embodiment of the application is: obtaining a pose parameter of a vehicle-mounted camera; aiming the vehicle-mounted camera at a plurality of marker points according to the pose parameter to obtain corresponding marker point data; and fitting the marker point data based on a least square method to obtain position information of the vehicle-mounted camera. From a special site specially set for camera parameter calibration to a general site to realize camera parameter calibration, that is, directly obtaining the pose parameter of the camera, then aiming the camera at the marker blocks at different positions, and calibrating the position of the camera based on the least square method, the technical problem of high cost of vehicle-mounted camera external parameter calibration can be solved, the requirements of the calibration work on the environment and the site are effectively reduced, and the vehicle-mounted camera external parameter calibration process does not need to make and place a checkerboard.

[0047] The technical terms involved in the embodiment of the application are:

[0048] The least square method, ordinary least squares, and least square method are a mathematical optimization technique. It finds the best function match of data by minimizing the sum of squares of errors. The unknown data can be easily obtained by using the least square method, and the sum of squares of errors between the obtained data and the actual data is minimized. The least square method can also be used for curve fitting, and other optimization problems can also be expressed by the least square method by minimizing the energy or maximizing the entropy.

[0049] The embodiment of the application considers that when the Zhang Zhengyou camera calibration method is used for external parameter calibration, a site specially used for camera calibration is needed, the checkerboard of a certain size and position is placed, and good lighting conditions are also needed for the identification of the checkerboard, resulting in high cost of vehicle-mounted camera external parameter calibration, which is not convenient to implement.

[0050] Therefore, the embodiment of the application scheme starts from the actual problem of vehicle-mounted camera external parameter calibration, combines the fitting capability of the least square method in data optimization, and designs a method for calibrating the external parameters of the vehicle front-view camera, solves the technical problem of high cost of vehicle-mounted camera external parameter calibration, effectively reduces the requirements of the calibration work on the environment and the site, and makes the vehicle-mounted camera external parameter calibration process not need to make and place a checkerboard.

[0051] Specifically, referring to Figure 1 , Figure 1The figure is a schematic diagram of a function module of a vehicle to which the device for calibrating the external parameters of a front-view camera of a vehicle belongs. The device for calibrating the external parameters of a front-view camera of a vehicle can be a device independent of the vehicle and capable of calibrating the external parameters of a vehicle-mounted camera, which can be carried on the vehicle in the form of hardware or software. The vehicle can be a smart mobile terminal such as a mobile phone or a tablet computer having a data processing function, or a fixed vehicle or a server having a data processing function.

[0052] In the embodiment, the vehicle to which the device for calibrating the external parameters of a front-view camera of a vehicle belongs at least includes an output module 110, a processor 120, a memory 130, and a communication module 140.

[0053] The memory 130 stores an operating system and a program for calibrating the external parameters of a front-view camera of a vehicle. The device for calibrating the external parameters of a front-view camera of a vehicle can store, in the memory 130, information such as the acquired attitude parameters of a vehicle-mounted camera, the obtained corresponding landmark data obtained by aiming the vehicle-mounted camera at preset landmark points at different positions according to the attitude parameters, and the position information of the vehicle-mounted camera obtained by fitting the landmark data based on the least square method. The output module 110 can be a display screen or the like. The communication module 140 can include a WIFI module, a mobile communication module, a Bluetooth module, and the like, and communicates with external devices or servers through the communication module 140.

[0054] When the program for calibrating the external parameters of a front-view camera of a vehicle in the memory 130 is executed by the processor, the following steps are implemented:

[0055] The attitude parameters of the vehicle-mounted camera are acquired.

[0056] The vehicle-mounted camera is aimed at preset landmark points at different positions according to the attitude parameters, and corresponding landmark data is obtained.

[0057] The landmark data is fitted based on the least square method, and the position information of the vehicle-mounted camera is obtained.

[0058] Further, when the program for calibrating the external parameters of a front-view camera of a vehicle in the memory 130 is executed by the processor, the following steps are implemented:

[0059] The distance between the target vehicle to which the vehicle-mounted camera belongs and the landmark points is determined.

[0060] The direction of the vehicle-mounted camera is adjusted to aim at the landmark points through a preset display screen, and the corresponding pitch angle is obtained.

[0061] The landmark data is obtained based on the cotangent value of the distance and the pitch angle.

[0062] Furthermore, when the program in memory 130 for calibrating the extrinsic parameters of the vehicle's forward-looking camera is executed by the processor, it also performs the following steps:

[0063] The front axle center, forward direction, vertical direction, and preset right-hand rule of the target vehicle are obtained respectively to determine the origin, horizontal axis, vertical axis, and longitudinal axis, thus obtaining the target vehicle coordinate system;

[0064] The distance is determined based on the position of the marker point in the target vehicle's coordinate system.

[0065] Furthermore, when the program in memory 130 for calibrating the extrinsic parameters of the vehicle's forward-looking camera is executed by the processor, it also performs the following steps:

[0066] Adjust the center point of the image to coincide with the marker point and adjust the component of the accelerometer on the horizontal axis to zero to determine the adjusted camera;

[0067] Based on the adjusted camera, the components of the accelerometer on the vertical axis and the longitudinal axis are read respectively to obtain the vertical axis component and the longitudinal axis component;

[0068] The pitch angle is obtained based on the angular relationship between the vertical axis component and the longitudinal axis component.

[0069] Furthermore, when the program in memory 130 for calibrating the extrinsic parameters of the vehicle's forward-looking camera is executed by the processor, it also performs the following steps:

[0070] Using the pitch angle as the independent variable and the distance as the dependent variable, we obtain the vertical axis coordinates on the vertical axis and the horizontal axis coordinates on the horizontal axis.

[0071] The marker point data is obtained based on the vertical axis coordinates and the horizontal axis coordinates.

[0072] Furthermore, when the program in memory 130 for calibrating the extrinsic parameters of the vehicle's forward-looking camera is executed by the processor, it also performs the following steps:

[0073] Detect whether the vertical coordinate of the vehicle-mounted camera on the vertical axis is zero;

[0074] If the vertical coordinate of the vehicle-mounted camera is zero, then the spatial position information of the vehicle-mounted camera can be obtained by combining the vertical coordinate with the position information.

[0075] Furthermore, when the program in memory 130 for calibrating the extrinsic parameters of the vehicle's forward-looking camera is executed by the processor, it also performs the following steps:

[0076] The attitude parameters of the vehicle-mounted camera are obtained through the accelerometer.

[0077] The embodiment obtains the attitude parameter of the vehicle-mounted camera, aims the vehicle-mounted camera at a plurality of marker points according to the attitude parameter, obtains corresponding marker point data, and fits the marker point data based on the least square method to obtain the position information of the vehicle-mounted camera. The camera parameter calibration is realized from a special site specially set for camera parameter calibration to a general site, that is, the attitude parameter of the camera is directly obtained, the camera is aimed at marker blocks at different positions, and the position of the camera is calibrated based on the least square method, which can solve the technical problem of high cost of vehicle-mounted camera external parameter calibration, effectively reduces the requirement of the calibration work on the environment and the site, and makes the vehicle-mounted camera external parameter calibration process not need to make and place a checkerboard.

[0078] The method embodiment of the present application is based on but not limited to the above vehicle architecture.

[0079] Reference Figure 2 , Figure 2 A flowchart of a first exemplary embodiment of a method for calibrating external parameters of a vehicle front-view camera is provided. The method for calibrating external parameters of a vehicle front-view camera comprises the following steps.

[0080] In step S210, the attitude parameter of the vehicle-mounted camera is obtained.

[0081] The execution subject of the method embodiment can be a device for calibrating external parameters of a vehicle front-view camera, or a vehicle or a server for vehicle-mounted camera external parameter calibration. The device for calibrating external parameters of a vehicle front-view camera can be integrated in a vehicle such as a smartphone, a tablet computer, etc. having a data processing function.

[0082] At present, the Zhang Zhengyou calibration method is mostly used for camera external parameter calibration, and a checkerboard needs to be accurately placed around the vehicle, so the requirement of the calibration site is relatively high.

[0083] For the case that the position of the camera is not convenient to directly measure, the embodiment of the present application provides a simple and easy-to-implement method to obtain the necessary camera position information, mainly realizes the calibration of the vehicle-mounted camera parameters, especially the calibration of the external parameters of the vehicle-mounted camera, and effectively reduces the requirement of the calibration work on the environment and the site.

[0084] Specifically, the acceleration sensor integrated on the vehicle-mounted camera is preferred in the embodiment of the present application, and in other embodiments, other sensors that can obtain the attitude parameter of the vehicle-mounted camera can also be used, which are not limited.

[0085] Step S220, aiming the vehicle-mounted camera at preset marker points at different positions according to the attitude parameter to obtain corresponding marker point data.

[0086] Specifically, the marker points are used for aiming the vehicle-mounted camera, and the marker points can be markers or marker blocks, or can be a marking point. By obtaining the attitude parameter, the vehicle-mounted camera is aimed at road markers at different positions according to the attitude parameter, and a plurality of measurements are obtained to obtain one or more sets of marker point data. The position information of the camera can be obtained based on the least square method. In this way, there is no need to arrange a checkerboard, and the requirements for the site are reduced.

[0087] The position of the marker point can be any position in front of the vehicle to which the vehicle-mounted camera belongs.

[0088] Step S230, fitting the marker point data based on the least square method to obtain the position information of the vehicle-mounted camera.

[0089] Specifically, referring to Figure 3 , Figure 3 A schematic diagram of a plurality of measurements of a method for calibrating the extrinsic parameters of a vehicle front-view camera is provided. A plurality of sets of marker point data are plotted in a plan view, and a plurality of sets of marker point data are fitted into a straight line based on the least square method to reduce measurement error, and the position information of the vehicle-mounted camera is obtained, i.e., the position of the vehicle-mounted camera is determined according to the slope of the straight line and the intercept of the straight line on the vertical axis, respectively.

[0090] The position of the vehicle-mounted camera includes x, y, and z coordinates in a spatial coordinate system.

[0091] The above scheme is used to obtain the attitude parameter of the vehicle-mounted camera. According to the attitude parameter, the vehicle-mounted camera is aimed at a plurality of marker points to obtain corresponding marker point data. The marker point data is fitted based on the least square method to obtain the position information of the vehicle-mounted camera. From a special site specially set for camera parameter calibration to a general site, the camera parameter calibration is realized, i.e., the attitude parameter of the camera is directly obtained, and then the camera is aimed at marker blocks at different positions. The position of the camera is calibrated based on the least square method, which can solve the technical problem of high cost of vehicle-mounted camera extrinsic parameter calibration, effectively reduce the requirements of the calibration work on the environment and the site, and make the vehicle-mounted camera extrinsic parameter calibration process not need to make and place a checkerboard.

[0092] Referring to Figure 4 , Figure 4 A flowchart of a second exemplary embodiment of a method for calibrating the extrinsic parameters of a vehicle front-view camera is provided. Based on the above Figure 2In the embodiment shown, in step S220, the vehicle-mounted camera is aimed at the preset mark point at different positions according to the attitude parameters, and corresponding mark point data is obtained, including:

[0093] In step S410, the distance between the target vehicle to which the vehicle-mounted camera belongs and the mark point is determined.

[0094] Specifically, the distance is the distance between the midpoint of the projection of the front axle of the vehicle on the ground and the mark point when the vehicle to which the vehicle-mounted camera belongs is stationary, and is used to calculate the position information of the vehicle-mounted camera.

[0095] Further, in step S410, the distance between the target vehicle to which the vehicle-mounted camera belongs and the mark point is determined, including:

[0096] The center of the front axle, the forward direction, the vertical direction and the preset right-hand rule of the target vehicle are acquired respectively to determine the origin, the horizontal axis, the vertical axis and the longitudinal axis, and the target vehicle coordinate system is obtained.

[0097] According to the position of the mark point in the target vehicle coordinate system, the distance is determined.

[0098] Reference Figure 5 , Figure 5 The application provides a vehicle coordinate system construction schematic diagram used in a method for calibrating an external parameter of a vehicle forward-looking camera. Specifically, according to the actual arrangement position of the vehicle forward-looking camera, the following coordinate system is established: as shown, the vehicle coordinate system o1 is established, the midpoint of the projection of the front axle of the vehicle on the ground is used as the origin, the positive direction of x is the forward direction, the direction of z is the vertical upward direction, and the direction of y is determined according to the right-hand rule; a mark point P is placed on the ground in front of the vehicle, and the distance between the mark point and the front axle of the vehicle is measured. The coordinates of the mark point in the vehicle coordinate system can be obtained as (d, 0, 0).

[0099] In step S420, the direction of the vehicle-mounted camera is adjusted to aim at the mark points through a preset display picture, and corresponding pitch angles are obtained.

[0100] Specifically, the direction of the vehicle-mounted camera is adjusted through an adjustment module integrated on the vehicle-mounted camera or a terminal device wirelessly connected with the vehicle-mounted camera, so that the vehicle-mounted camera aims at the corresponding mark point, and then the corresponding pitch angle is obtained.

[0101] Further, the vehicle-mounted camera is integrated with an acceleration sensor, the display picture includes a picture center point, and in step S420, the direction of the vehicle-mounted camera is adjusted to aim at the mark points through the preset display picture, and corresponding pitch angles are obtained, including:

[0102] Adjust the center point of the image to coincide with the marker point and adjust the component of the accelerometer on the horizontal axis to zero to determine the adjusted camera;

[0103] Based on the adjusted camera, the components of the accelerometer on the vertical axis and the longitudinal axis are read respectively to obtain the vertical axis component and the longitudinal axis component;

[0104] The pitch angle is obtained based on the angular relationship between the vertical axis component and the longitudinal axis component.

[0105] Specifically, the steps for adjusting the orientation of the vehicle-mounted camera include, but are not limited to, the user using a pre-set application on their terminal device to adjust the orientation of the vehicle-mounted camera via wireless communication; and / or automatically adjusting the camera orientation via an adjustment module integrated into the vehicle-mounted camera, so that the vehicle-mounted camera meets the following standards:

[0106] 1. The center point of the camera's display image is aligned with the marker point P on the ground so that the center point of the display image coincides with the marker point P; 2. The x-axis component of the accelerometer integrated into the vehicle camera is zero to ensure the camera is horizontal. The y and z components of the accelerometer are read, and the camera's pitch angle α is tanα = Gz / Gy.

[0107] Step S430: Obtain the marker point data based on the cotangent value of the distance and the pitch angle.

[0108] Specifically, such as Figure 6 As shown, Figure 6 This is a schematic diagram illustrating multiple measurements of a method for calibrating the extrinsic parameters of a vehicle's forward-looking camera, as described in this application. Without needing to draw a checkerboard pattern, several sets of marker point data are determined based on the distance between the vehicle and the marker point and the cotangent of the camera's pitch angle. The position information can then be obtained by fitting a straight line using the least squares method.

[0109] Further, in step S430, based on the cotangent value of the distance and the pitch angle, the marker point data is obtained, including:

[0110] Using the pitch angle as the independent variable and the distance as the dependent variable, we obtain the vertical axis coordinates on the vertical axis and the horizontal axis coordinates on the horizontal axis.

[0111] The marker point data is obtained based on the vertical axis coordinates and the horizontal axis coordinates.

[0112] Specifically, by repeating steps S410 to S430 multiple times, several sets of marker point data can be obtained. Based on the relationship between the distance between the vehicle and the marker point and the tilt angle of the vehicle camera, the following equation is constructed:

[0113]

[0114] In the formula, z1 is the z-direction coordinate of the camera in the vehicle coordinate system, x1 is the x-direction coordinate of the camera in the vehicle coordinate system, and d is the distance between the marker point P and the vehicle.

[0115] Taking d as the longitudinal coordinate and 1 / tan a as the transverse coordinate, a plurality of test results are plotted on a plane graph, and referring to Figure 3 , a plurality of data points are fitted into a straight line based on the least square method, and the slope of the straight line is the value of z1, and the intercept of the straight line on the longitudinal axis is the value of x1.

[0116] It should be noted that the front-view camera of the vehicle is generally installed on the symmetry plane of the vehicle and horizontally faces the front of the vehicle, so the pose of the camera can be determined by the angle a, that is, the value of the angle a can be determined by step S420. The y coordinate of the camera is 0 by default, and the position information is (x1, 0, z1), wherein the variables x1 and z1 are determined by step S430.

[0117] The embodiment determines the distance between the target vehicle to which the vehicle-mounted camera belongs and the marker point through the above scheme, adjusts the direction of the vehicle-mounted camera to aim at the plurality of marker points through the preset display screen to obtain the corresponding pitch angle, and obtains the marker point data based on the distance and the cotangent value of the pitch angle. Based on a plurality of experimental data, the position is obtained by fitting a straight line through the least square method. In this way, without drawing a checkerboard, the error is reduced through multiple measurement methods, which is low in cost and easy to implement.

[0118] Referring to Figure 7 , Figure 7 is a flowchart of a third exemplary embodiment of a method for calibrating the extrinsic parameters of a front-view camera of a vehicle. Based on the above Figure 4 embodiment shown in the embodiment, after step S230, the position information of the vehicle-mounted camera is obtained by fitting the marker point data based on the least square method, and the method further includes:

[0119] Step S710, detecting whether the longitudinal axis coordinate of the vehicle-mounted camera on the longitudinal axis is zero;

[0120] Step S720, if the longitudinal axis coordinate of the vehicle-mounted camera on the longitudinal axis is zero, combining the longitudinal axis coordinate and the position information to obtain the spatial position information of the vehicle-mounted camera.

[0121] Specifically, the coordinate of the longitudinal axis is zero, which is the symmetry plane of the vehicle. Since the front-view camera of the vehicle is generally installed on the symmetry plane of the vehicle and horizontally faces the front of the vehicle, after the least square method is used to fit a plurality of sets of marker point data, it is further detected whether the longitudinal axis coordinate of the vehicle-mounted camera is zero. If it is determined that the longitudinal axis coordinate of the vehicle-mounted camera is zero, that is, after it is determined that the vehicle-mounted camera is in the symmetry plane of the target vehicle, the position of the camera in the vehicle coordinate system can be accurately estimated in combination with the position information to adjust the camera to the corresponding position, so that the position information is applied to environmental perception, for example, to the ranging of road obstacles.

[0122] The embodiment can solve the technical problem of high cost of vehicle-mounted camera external parameter calibration, effectively reduce the requirements of the calibration work on the environment and the site, and make the vehicle-mounted camera external parameter calibration process not need to make and place a checkerboard, and further apply the position information to the ranging algorithm of road obstacles.

[0123] Reference Figure 8 , Figure 8 FIG. 4 is a flowchart of a fourth exemplary embodiment of a method for calibrating external parameters of a front-view camera of a vehicle according to the present application. Based on the above Figure 4 As shown in the embodiment shown in FIG. 4, in step S210, the attitude parameters of the vehicle-mounted camera are obtained, including:

[0124] In step S810, the attitude parameters of the vehicle-mounted camera are obtained by the acceleration sensor.

[0125] Specifically, with the development of camera and sensor technologies, the camera modules used for research can already integrate acceleration sensors together. Therefore, for the case that the camera position is not convenient to directly measure, the embodiments of the present application use the acceleration sensor integrated in the camera module to propose an innovative front-view camera calibration method, which does not need to recognize a checkerboard or a specially prepared site, and the acceleration sensor integrated in the camera can be used to assist in measuring the attitude information and position information of the camera.

[0126] The embodiment obtains the attitude parameter of the vehicle-mounted camera through the acceleration sensor, aims the vehicle-mounted camera at preset mark points at different positions according to the attitude parameter, obtains corresponding mark point data, and obtains the position information of the vehicle-mounted camera based on least square method fitting of the mark point data. The attitude parameter of the vehicle-mounted camera is obtained through the acceleration sensor integrated on the vehicle-mounted camera, and then the position information of the vehicle-mounted camera is calculated, thereby effectively reducing the requirement of the calibration work on the environment and the site.

[0127] In addition, the embodiment of the application further provides a device for calibrating the extrinsic parameter of the front-view camera of the vehicle, which comprises:

[0128] a parameter obtaining module, configured to obtain the attitude parameter of the vehicle-mounted camera;

[0129] a mark point aiming module, configured to aim the vehicle-mounted camera at preset mark points at different positions according to the attitude parameter, to obtain corresponding mark point data;

[0130] a data fitting module, configured to fit the mark point data based on least square method, to obtain the position information of the vehicle-mounted camera.

[0131] The principle and implementation process of the device for calibrating the extrinsic parameter of the front-view camera of the vehicle are realized, please refer to the above-mentioned embodiments, which will not be repeated here.

[0132] In addition, the embodiment of the application further provides a vehicle, which comprises a memory, a processor, and a program for calibrating the extrinsic parameter of the front-view camera of the vehicle stored on the memory and executable on the processor. The program for calibrating the extrinsic parameter of the front-view camera of the vehicle is executed by the processor to realize the steps of the method for calibrating the extrinsic parameter of the front-view camera of the vehicle.

[0133] Since the program for calibrating the extrinsic parameter of the front-view camera of the vehicle is executed by the processor, all the technical solutions of all the above-mentioned embodiments are adopted, and at least all the beneficial effects brought by all the technical solutions of all the above-mentioned embodiments are obtained, which will not be repeated here.

[0134] In addition, the embodiment of the application further provides a computer readable storage medium, which stores a program for calibrating the extrinsic parameter of the front-view camera of the vehicle. The program for calibrating the extrinsic parameter of the front-view camera of the vehicle is executed by the processor to realize the steps of the method for calibrating the extrinsic parameter of the front-view camera of the vehicle.

[0135] Since the program for calibrating the external parameters of the vehicle front-view camera is executed by the processor, all the technical solutions of all the foregoing embodiments are adopted, and all the beneficial effects brought by all the technical solutions of all the foregoing embodiments are at least achieved, which will not be repeated here.

[0136] Compared with the prior art, the method for calibrating the external parameters of the vehicle front-view camera provided in the embodiments of the application calibrates the external parameters of the vehicle front-view camera by obtaining the attitude parameters of the vehicle camera, aiming the vehicle camera at a plurality of marker points according to the attitude parameters to obtain corresponding marker point data, and fitting the marker point data based on the least square method to obtain the position information of the vehicle camera. The camera parameter calibration is realized from a special site specially set for camera parameter calibration to a general site, that is, the attitude parameters of the camera are directly obtained, the camera is aimed at marker blocks at different positions, and the position of the camera is calibrated based on the least square method, which can solve the technical problem of high cost of vehicle camera external parameter calibration, effectively reduces the requirements of the calibration work on the environment and the site, and makes the vehicle camera external parameter calibration process not need to make and place a checkerboard.

[0137] It should be noted that in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed, or inherent to such process, method, article, or system. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or system that includes the element.

[0138] The above sequence numbers of the embodiments of the application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0139] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) as described above, and includes a plurality of instructions for causing a vehicle to execute the method of each embodiment of the application.

[0140] The above is only the preferred embodiment of the application, and does not limit the patent scope of the application, and any equivalent structure or equivalent process transformation made by using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the application.

Claims

1. A method for calibrating extrinsic parameters of a vehicle's forward-looking camera, characterized in that, The method for calibrating the external parameters of a vehicle's forward-looking camera includes the following steps: Obtain the attitude parameters of the vehicle-mounted camera; Based on the attitude parameters, the vehicle-mounted camera is aimed at preset marker points at different locations to obtain the corresponding marker point data; The location information of the vehicle-mounted camera is obtained by fitting the marker point data using the least squares method. The step of aiming the vehicle-mounted camera at preset marker points at different locations based on the attitude parameters to obtain corresponding marker point data includes: Determine the distance between the target vehicle to which the vehicle-mounted camera belongs and the preset marker point; By adjusting the direction of the vehicle-mounted camera to aim at all the preset marker points through the preset display screen, the corresponding pitch angle is obtained; The marker point data is obtained based on the cotangent value of the distance and the pitch angle.

2. The method for calibrating extrinsic parameters of a vehicle forward-looking camera as described in claim 1, characterized in that, The step of determining the distance between the target vehicle to which the vehicle-mounted camera belongs and the preset marker point includes: The front axle center, forward direction, vertical direction, and preset right-hand rule of the target vehicle are obtained respectively to determine the origin, horizontal axis, vertical axis, and longitudinal axis, thus obtaining the target vehicle coordinate system; The distance is determined based on the position of the preset marker point in the target vehicle coordinate system.

3. The method for calibrating extrinsic parameters of a vehicle forward-looking camera as described in claim 2, characterized in that, The vehicle-mounted camera integrates an acceleration sensor, and the displayed image includes a center point. The step of adjusting the direction of the vehicle-mounted camera to aim at all preset marker points through a preset displayed image to obtain the corresponding pitch angle includes: Adjust the center point of the image to coincide with the preset marker point and adjust the component of the accelerometer on the horizontal axis to zero to determine the adjusted camera; Based on the adjusted camera, the components of the accelerometer on the vertical axis and the longitudinal axis are read respectively to obtain the vertical axis component and the longitudinal axis component; The pitch angle is obtained based on the angular relationship between the vertical axis component and the longitudinal axis component.

4. The method for calibrating extrinsic parameters of a vehicle forward-looking camera as described in claim 3, characterized in that, The step of obtaining the marker point data based on the cotangent value of the distance and the pitch angle includes: Using the pitch angle as the independent variable and the distance as the dependent variable, we obtain the vertical axis coordinates on the vertical axis and the horizontal axis coordinates on the horizontal axis. The marker point data is obtained based on the vertical axis coordinates and the horizontal axis coordinates.

5. The method for calibrating extrinsic parameters of a vehicle forward-looking camera as described in claim 3, characterized in that, The steps for obtaining the attitude parameters of the vehicle-mounted camera include: The attitude parameters of the vehicle-mounted camera are obtained through the accelerometer.

6. The method for calibrating extrinsic parameters of a vehicle forward-looking camera as described in claim 2, characterized in that, After the step of fitting the marker point data based on the least squares method to obtain the position information of the vehicle-mounted camera, the method further includes: Detect whether the vertical coordinate of the vehicle-mounted camera on the vertical axis is zero; If the vertical coordinate of the vehicle-mounted camera is zero, then the spatial position information of the vehicle-mounted camera can be obtained by combining the vertical coordinate with the position information.

7. A device for calibrating the external parameters of a vehicle's forward-looking camera, characterized in that, The device for calibrating the external parameters of the vehicle's forward-looking camera includes: The parameter acquisition module is used to acquire the attitude parameters of the vehicle-mounted camera; The marker aiming module is used to aim the vehicle-mounted camera at preset markers at different positions according to the attitude parameters, and obtain the corresponding marker data. The data fitting module is used to fit the marker point data based on the least squares method to obtain the position information of the vehicle-mounted camera; The step of aiming the vehicle-mounted camera at preset marker points at different locations based on the attitude parameters to obtain corresponding marker point data includes: Determine the distance between the target vehicle to which the vehicle-mounted camera belongs and the preset marker point; By adjusting the direction of the vehicle-mounted camera to aim at all the preset marker points through the preset display screen, the corresponding pitch angle is obtained; The marker point data is obtained based on the cotangent value of the distance and the pitch angle.

8. A vehicle, characterized in that, The vehicle includes a memory, a processor, and a program stored in the memory and executable on the processor for calibrating the extrinsic parameters of a vehicle's forward-view camera. When executed by the processor, the program for calibrating the extrinsic parameters of the vehicle's forward-view camera implements the steps of a method for calibrating the extrinsic parameters of a vehicle's forward-view camera as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for calibrating the extrinsic parameters of a vehicle's forward-view camera. When the program for calibrating the extrinsic parameters of a vehicle's forward-view camera is executed by a processor, it implements the steps of a method for calibrating the extrinsic parameters of a vehicle's forward-view camera as described in any one of claims 1-6.

Images