Calibration method and device for electronic rearview mirror parameters

Abstract

The application relates to a method and device for calibrating parameters of an electronic rearview mirror, wherein the method comprises: calculating a pose corresponding to a camera of the electronic rearview mirror of a vehicle based on a first coordinate system during the process that the vehicle drives into a site entrance and drives out of a site exit in a straight line at a uniform speed, wherein the pose represents the pose corresponding to the camera at different moments during driving; the first coordinate system is established based on a checkerboard, and the checkerboard is arranged on both sides of the site entrance; a straight line in which the pose of the camera during driving is located in space is determined, a unit vector of the straight line in the first coordinate system is determined, and a first rotation matrix is determined based on the unit vector; an attitude angle of the camera is determined based on the pose corresponding to the camera, and a second rotation matrix is determined based on the attitude angle and the first rotation matrix. Through the application, the problem that the calibration process of the camera external parameter is complicated in the prior art, thereby causing low calibration efficiency, is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of camera parameter calibration, in particular to a method and device for calibrating electronic rearview mirror parameters. BACKGROUND

[0002] Vehicle electronic rearview mirror camera is a basic tool for vehicle to perceive the distance of real world, such as accurate distance line and vehicle body and wheel position can be displayed in monitor through CMS, which is of great help to the driving and parking process of passenger car and commercial vehicle drivers. For example, after obtaining the image, various targets can be detected through visual perception model, so as to complete some active safety functions or auxiliary driving functions.

[0003] Generally, the initial angle of the electronic rearview mirror (Camera-Monitor System, CMS) is fixed, which is ensured by Y arm and installation process, but for the camera, the angle error has great influence on the display of the final monitor, so micro correction of camera external parameters is an indispensable step. Micro correction can be customized according to actual needs and conditions, but the current customized method has a cumbersome calibration process, and when the angle or focal length of the camera changes, it needs to be recalibrated, which is low in calibration efficiency. SUMMARY

[0004] The present application provides a method and device for calibrating electronic rearview mirror parameters to solve the problem of low calibration efficiency caused by the cumbersome calibration process of camera external parameters in the prior art.

[0005] In a first aspect, the present application provides a method for calibrating electronic rearview mirror parameters, comprising: calculating the pose of the camera corresponding to the electronic rearview mirror of the vehicle based on a first coordinate system during the process that the vehicle drives into the entrance of the site and drives out of the exit of the site in a straight line at a constant speed, wherein the pose represents the pose of the camera corresponding to the vehicle at different times during driving; the first coordinate system is established based on a checkerboard, and the checkerboard is arranged on both sides of the entrance of the site; determining the straight line in space where the camera pose is located during driving and determining the unit vector of the straight line in the first coordinate system, and determining the first rotation matrix based on the unit vector; wherein the first rotation matrix is the matrix from the second coordinate system to the first coordinate system, and the second coordinate system is established based on the starting position of the vehicle in the site; determining the attitude angle of the camera based on the corresponding pose of the camera, and determining the second rotation matrix based on the attitude angle and the first rotation matrix, wherein the second rotation matrix represents the external parameters of the camera.

[0006] Optionally, determining the attitude angle of the camera based on the camera corresponding pose, and determining a second rotation matrix based on the attitude angle and the first rotation matrix comprises: determining an average value of the attitude angle of the camera based on the camera corresponding pose; obtaining a ratio result of the average value of the attitude angle and the first rotation matrix, and determining the ratio result as the second rotation matrix.

[0007] Optionally, before determining the attitude angle of the camera based on the camera corresponding pose, the method further comprises: calculating distortion coefficients related to the camera and intrinsic parameters of the camera based on the first coordinate system; performing distortion correction on an image obtained by the camera based on the distortion coefficients, and determining a re-projection error of a feature point in the image after distortion correction; and substituting the re-projection error, the intrinsic parameters, the pose, the position of the feature point in the first coordinate system, and the unit vector into a first target formula of a nonlinear optimization model to optimize the pose, wherein the first target formula is used to obtain a first target residual based on the re-projection error, the intrinsic parameters, the pose, and the feature point, and the optimization of the pose is completed when the first target residual takes the minimum value.

[0008] Optionally, the first target formula is:

[0009]

[0010] wherein e is the first target residual, i is time, j is a feature point number, is a depth of the feature point to the image, K is a transformation matrix of the intrinsic parameters, exp(ξ i is a pose of the camera at i time, ξ is a pose Lie algebra expression symbol, is a re-projection residual of the feature point at i time, d i is a distance residual of the camera position to the straight line at i time, θ i is a pose residual of the camera at i time, m and n are positive integers.

[0011] Optionally, before determining the attitude angle of the camera based on the corresponding pose of the camera, the method further comprises: calculating a distortion coefficient related to the camera and an intrinsic parameter of the camera based on the first coordinate system; performing distortion correction on an image acquired by the camera based on the distortion coefficient, and determining a relative pose of the camera relative to the first coordinate system based on the image after distortion correction; and substituting the relative pose, the intrinsic parameter, the pose, the position of the feature point in the first coordinate system, and the unit vector into a second objective formula of a nonlinear optimization model to optimize the pose, wherein the second objective formula is used to obtain a second target residual based on the relative pose, the intrinsic parameter, the pose, and the feature point, and the optimization of the pose is completed when the second target residual takes the minimum value.

[0012] Optionally, calculating the corresponding pose of the camera of the electronic rearview mirror of the vehicle based on the first coordinate system comprises: acquiring images through the camera at every preset time interval, and deleting images in which all feature points are not detected from the acquired images, wherein each checkerboard is provided with a plurality of feature points; and calculating the corresponding pose of the camera based on the first coordinate system and the images acquired after the deletion of images.

[0013] Optionally, determining the straight line in space where the pose of the camera during driving is located comprises: acquiring the poses of the camera at different time instants during driving, and determining a target straight line based on the poses of the camera at different time instants, wherein the sum of distances from the poses of the camera at different time instants to the target straight line is less than the sum of target distances, and the sum of target distances is the sum of distances from the poses of the camera at different time instants to any straight line in space except the target straight line; and determining the target straight line as the straight line in space where the pose of the camera during driving is located.

[0014] In a second aspect, the present application provides a device for calibrating parameters of an electronic rearview mirror, comprising: a first processing module configured to calculate a pose corresponding to a camera of the electronic rearview mirror of a vehicle based on a first coordinate system during a process in which the vehicle enters a site entrance and exits a site exit in a straight line at a constant speed, wherein the pose represents the pose corresponding to the camera at different moments during the driving process; the first coordinate system is established based on a checkerboard, and the checkerboard is arranged on both sides of the site entrance; a second processing module configured to determine a straight line in space on which the pose of the camera during the driving process is located and determine a unit vector of the straight line in the first coordinate system, and determine a first rotation matrix based on the unit vector; wherein the first rotation matrix is a matrix for converting from a second coordinate system to the first coordinate system, and the second coordinate system is established based on a starting position of the vehicle in the site; a third processing module configured to determine an attitude angle of the camera based on the pose corresponding to the camera, and determine a second rotation matrix based on the attitude angle and the first rotation matrix, wherein the second rotation matrix represents the extrinsic parameters of the camera.

[0015] In a third aspect, the present application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus, wherein the processor is configured to execute the method for calibrating parameters of an electronic rearview mirror according to the first aspect of the present application.

[0016] In a fourth aspect, the present application further provides a computer storage medium storing computer executable instructions for executing the method for calibrating parameters of an electronic rearview mirror according to the first aspect of the present application.

[0017] The above technical solution provided by the embodiments of the present application has the following advantages compared with the prior art: the method provided by the embodiments of the present application only needs to arrange a checkerboard in a built site, and then the pose corresponding to the camera of the electronic rearview mirror of the vehicle during a process in which the vehicle enters a site entrance and exits a site exit in a straight line at a constant speed can be calculated based on the first coordinate system established based on the checkerboard, and then the straight line in space on which the pose of the camera during the driving process is located and the unit vector of the straight line in the first coordinate system can be determined, and the first rotation matrix can be determined based on the unit vector, and finally the extrinsic parameters of the camera can be determined based on the attitude angle corresponding to the pose and the first rotation matrix. It can be seen that in the embodiments of the present application, there is no need to fix a site, and the extrinsic parameters of the camera can be calculated without strict measurement after changing the site, which reduces the investment of calibration personnel and reduces the difficulty of calibration operation, and solves the problem of low calibration efficiency caused by the complicated calibration process of the extrinsic parameters of the camera in the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0020] One or more embodiments are illustrated by way of example with reference to the drawings, which are not limiting of the embodiments and which are merely meant to explain the principles of the embodiments. The reference numbers in the drawings indicate elements which functionally correspond to the other elements in the same group. The drawings in the accompanying drawings are not limiting of the proportions. The same reference numbers in the drawings indicate similar elements.

[0021] Figure 1 One of flowcharts of a parameter calibration method of an electronic rearview mirror according to an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a chessboard according to an embodiment of the present application;

[0023] Figure 3 A schematic diagram of vehicle driving according to an embodiment of the present application;

[0024] Figure 4 The second flowchart of the parameter calibration method of the electronic rearview mirror according to an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a nonlinear optimization model according to an embodiment of the present application;

[0026] Figure 6 The third flowchart of the parameter calibration method of the electronic rearview mirror according to an embodiment of the present application;

[0027] Figure 7 A schematic diagram of a parameter calibration device of an electronic rearview mirror according to an embodiment of the present application;

[0028] Figure 8 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0030] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For the purpose of simplicity, the elements and settings of particular examples in the following description are set forth by referring to the drawings. Of course, they are only examples and are not intended to limit the present application. In addition, reference numerals and / or letters can be repeated in different examples. Such repetition is for the purpose of simplification and clarity, and does not indicate a relationship between the various embodiments and / or settings being discussed.

[0031] The prior art micro-correcting camera external parameter method usually includes the following two kinds 1) a plurality of set distance vehicle distance calibration lines with a set length and a set height are arranged at a plurality of set distances parallel to the tail or the two sides of the vehicle body, a vehicle distance identification line is generated in the monitor according to the first and last pixel points, the vehicle distance identification line is compared with the vehicle distance calibration line one by one, and the error is corrected. This method can only realize one-to-one correspondence between the physical calibration line and the identification line in the image, and cannot realize full correction of the electronic rearview mirror CMS camera. When the angle or focal length of the camera changes, recalibration is required, and the calibration process is tedious. 2) a plurality of marker points are arranged in the field of view of the electronic rearview mirror to be calibrated, the corresponding relationship between the 3D points in the vehicle body world coordinate system and the 2D points in the non-distorted image and the corresponding relationship between the non-distorted points and the distorted points are established, the coordinates of each ground marker point are converted from the vehicle body world coordinate system to the coordinate system of the non-distorted image; the straight line equations between the corresponding non-distorted points of different ground marker points are calculated in the coordinate system of the non-distorted image; the distance constraints of the corresponding non-distorted points to the corresponding straight line equations of a plurality of sampling points on the straight line between the ground marker points are constructed respectively, and the sum of the distance constraints is taken as the objective function, and the external parameters of the electronic rearview mirror to be calibrated are solved. This method needs to know the specific positions of the vehicle and the 3D points in the unified coordinate system, and measurement is essential, or the vehicle needs to be parked accurately, which cannot meet the requirement of rapid calibration.

[0032] Based on this, the embodiments of the present application provide a calibration method for electronic rearview mirror parameters, as shown in Figure 1 The steps of the method include:

[0033] Step 101, during the process that the vehicle drives into the site entrance and drives out of the site exit in a straight line at a uniform speed, a pose corresponding to a camera of an electronic rearview mirror of the vehicle is calculated based on a first coordinate system, wherein the pose represents the pose of the camera at different moments during driving; the first coordinate system is established based on a checkerboard, and the checkerboard is arranged on both sides of the site entrance;

[0034] In the specific examples of the present application, the checkerboard can be a 4*5 checkerboard, and each small square in the checkerboard is 20cm*20cm, as shown in Figure 2 In other examples of the embodiments of the present application, the small squares in the checkerboard can be other shapes, such as circular squares, asymmetric circular squares, or circular ring squares. In addition, the size of the squares in the checkerboard can also be set according to actual needs. The above-mentioned 20cm*20cm is only an example.

[0035] In addition, the site in the embodiments of the present application is a site with a flat and straight road so that the vehicle can drive at a uniform speed during driving. In addition, in the embodiments of the present application, the checkerboard for calibration can be placed on both sides of the site entrance, and the distance between the checkerboard and the wheels is 30cm-100cm, and the two checkerboards are perpendicular to the driving direction of the vehicle, as shown in Figure 3 Further, one or more feature points are pre-set in the checkerboard. Taking the squares in the checkerboard as an example, the corner points (the common points of adjacent corners of black and white squares) in the checkerboard.

[0036] Step 102, determining a straight line in space where the pose of the camera during driving is located and determining a unit vector of the straight line in the first coordinate system, and determining a first rotation matrix based on the unit vector; wherein the first rotation matrix is a matrix for converting from a second coordinate system to the first coordinate system, and the second coordinate system is established based on the starting position of the vehicle in the site;

[0037] In the specific examples of the present application, the first coordinate system is established based on the checkerboard, that is, the plane of the checkerboard is taken as the plane where the x and y axes in the first coordinate system are located, specifically, the positive direction of the x axis is consistent with the driving direction of the vehicle, the y axis is perpendicular to the x axis, and the z axis is perpendicular to the plane where the x and y axes are located, and the feature points on the checkerboard are taken as the origin. Further, the first coordinate system can be used to calibrate based on Zhang Zhengyou's calibration method, different pose pictures of the calibration board are collected, the pixel coordinates of the corner points in the pictures are extracted, the initial values of the internal parameters of the camera (f x ,f y ,u0,v0) are calculated through a homography matrix, the distortion coefficients (k1, k2) are estimated by using a nonlinear least squares method, and then the pose (R i ,t i ) of the camera at time i can be obtained.

[0038] In step 103, the attitude angle of the camera is determined based on the pose corresponding to the camera, and a second rotation matrix is determined based on the attitude angle and the first rotation matrix, where the second rotation matrix represents the external parameter of the camera.

[0039] It can be seen that, in the embodiment of the present application, only the chessboard grid needs to be arranged in the built site, and then the pose corresponding to the camera of the electronic rearview mirror in the process that the vehicle enters the site entrance and exits the site exit in a straight line at a uniform speed can be calculated according to the first coordinate system established by the chessboard grid, and the straight line in which the camera pose is located in the space during driving and the unit vector of the straight line in the first coordinate system can be determined, the first rotation matrix can be determined based on the unit vector, and finally the external parameter of the camera can be determined according to the attitude angle corresponding to the pose and the first rotation matrix. It can be seen that, in the embodiment of the present application, there is no need to fix the site, and the external parameter of the camera can be calculated without strict measurement after changing the site, which reduces the investment of calibration personnel and the difficulty of calibration operation, and solves the problem of low calibration efficiency caused by the complicated calibration process of the external parameter of the camera in the prior art.

[0040] In an optional implementation of the embodiment of the present application, the method for determining the straight line in which the camera pose is located in the space during driving in step 102 can further include:

[0041] In step 11, the pose of the camera at different times during driving is obtained, and a target straight line is determined based on the pose of the camera at different times, where the sum of the distances from the pose of the camera at different times to the target straight line is less than the sum of the target distances, and the sum of the target distances is the sum of the distances from the pose of the camera at different times to any straight line in the space except the target straight line.

[0042] In step 12, the target straight line is determined as the straight line in which the camera pose is located in the space during driving.

[0043] It should be noted that, in the theoretical case, the pose of the camera at all times is a straight line l in the 3D space, and the attitude angle of the camera is the same at any time, that is, the z-axis of the vehicle coordinate system (second coordinate system) is parallel to the z-axis of the calibration plate coordinate system (first coordinate system), and the y-axis direction of the second coordinate system is parallel to the direction of the straight line l, but in the actual case, there will be errors due to environmental or other factors, and the pose of the camera is not on a straight line in the 3D space, and for the same reason, the attitude angle at different times may also be different. Therefore, in the embodiment of the present application, the straight line fitting needs to be performed according to the camera position, that is, a straight line is found such that the sum of the distances from all points of the straight line to the straight line is less than the sum of the distances from all points to other straight lines, so as to reduce the influence of external factors as much as possible in subsequent determination of the external parameter, so as to improve the accuracy of the determination of the external parameter. After the straight line is determined, the unit vector of the straight line in the first coordinate system (lx ,l y ,l z ), is parallel to the x-y axis plane, so l z = 0, the unit vector of the straight line l is (l x ,l y ,0), and the rotation matrix (the first rotation matrix) from the second coordinate system to the first coordinate system can be calculated as

[0044] For the above-mentioned manner of determining the attitude angle of the camera based on the corresponding pose of the camera in step 103, and determining the second rotation matrix based on the attitude angle and the first rotation matrix, further comprising:

[0045] Step 21, determining the average value of the attitude angle of the camera based on the corresponding pose of the camera;

[0046] Step 22, obtaining the ratio result of the average value of the attitude angle and the first rotation matrix, and determining the ratio result as the second rotation matrix.

[0047] It should be noted that the attitude angle of the camera is constant during driving in a theoretical environment, but in actual situations, there may be other factors affecting the driving of the vehicle or the camera, so the attitude angle of the camera may differ between different time points. Based on this, in the embodiments of the present application, the average value of the attitude angle of the camera is first determined, and the average value can make the influence of external factors on the attitude angle smaller, so that the second rotation matrix obtained based on the average value of the attitude angle and the first rotation matrix is more accurate. In a specific example, it is assumed that the center of the rear axle of the vehicle is O (the original point of the vehicle starting to drive) at time i, and the pose of the vehicle in the first coordinate system is wherein The transformation matrix includes the rotation matrix R and the displacement t, The rotation matrix is calculated based on the average value of the attitude angle at all time points, and the pose of the camera in the second coordinate system is wherein is the external parameter to be calibrated, obtained from the vehicle model. In addition, in the specific example, there is a formula that the pose of the camera in the first coordinate system is the rotation matrix of the camera in the first coordinate system Therefore wherein, is the first rotation matrix, the average value of the attitude angle R i is obtained, so it can be seen that the external parameter of the camera obtained by the method in the embodiments of the present application has a simple calculation process and low computing power requirement, and the efficiency of the calibration of the external parameter of the camera is improved.

[0048] In an optional implementation in the embodiments of the present application, for the manner of calculating the pose corresponding to the camera of the electronic rearview mirror of the vehicle based on the first coordinate system in step 101, further comprising:

[0049] Step 31, collect images through the camera every preset time interval, and delete the images in which all feature points are not detected from the collected images, wherein each checkerboard is provided with a plurality of feature points;

[0050] Step 32, calculate the pose corresponding to the camera based on the first coordinate system and the images collected after the image deletion.

[0051] For this, in a specific example, if the current vehicle travels at a speed of 2 m / s, the preset time interval can be 1 s, that is, image collection is performed every 1 s, and if all feature points are not detected in the collected image, the image needs to be deleted, and then the pose corresponding to the camera is calculated based on the image in which all feature points are collected and the first coordinate system.

[0052] In order to calculate more accurate external parameters in the embodiments of the present application, the pose can also be optimized, that is, before the attitude angle of the camera is determined based on the pose corresponding to the camera, as shown in Figure 4 The method of the embodiments of the present application can further comprise:

[0053] Step 401, calculate the distortion coefficient related to the camera and the internal parameter of the camera based on the first coordinate system;

[0054] Step 402, perform distortion removal processing on the image obtained by the camera based on the distortion coefficient, and determine the re-projection error of the feature points in the image after the distortion removal processing;

[0055] Step 403, substitute the re-projection error, the internal parameter, the pose, and the position and unit vector of the feature points in the first coordinate system into a first target formula of a nonlinear optimization model to optimize the pose, the first target formula being used to obtain a target residual error based on the re-projection error, the internal parameter, the pose, and the feature points, and the optimization of the pose being completed when the target residual error takes the minimum value.

[0056] The first target formula is:

[0057]

[0058] Wherein, e is the target residual error, i is the time, j is the feature point number, is the depth of the feature point to the image, K is the transformation matrix of the internal parameter, exp(ξ i is the pose of the camera at time i, and ξ is the pose Lie algebra expression symbol, is the reprojection error of the feature point at time i, d i is the distance error of the camera position to the line at time i, θ i is the pose error of the camera at time i, and m and n are positive integers.

[0059] In the embodiments of the present application, the acquired image is de-distorted by the distortion coefficient to make the image as close as possible to the image before distortion, so that more accurate pose can be obtained when the pose is optimized subsequently. In specific examples, the nonlinear optimization model can be a graph optimization model, and further, the graph optimization g2o library shown in FIG. 2 can be used to optimize the pose in the following manner. Figure 5 In the embodiments of the present application, the acquired image is de-distorted by the distortion coefficient to make the image as close as possible to the image before distortion, so that more accurate pose can be obtained when the pose is optimized subsequently. In specific examples, the nonlinear optimization model can be a graph optimization model, and further, the graph optimization g2o library shown in FIG. 2 can be used to optimize the pose in the following manner. Figure 5 In the embodiments of the present application, the acquired image is de-distorted by the distortion coefficient to make the image as close as possible to the image before distortion, so that more accurate pose can be obtained when the pose is optimized subsequently. In specific examples, the nonlinear optimization model can be a graph optimization model, and further, the graph optimization g2o library shown in FIG. 2 can be used to optimize the pose in the following manner.

[0060] In another optional embodiment of the embodiments of the present application, in order to calculate more accurate external parameters, other methods can also be used to optimize the pose, that is, before the pose angle of the camera is determined based on the corresponding pose of the camera, the method further includes the following steps. Figure 6

[0061] Step 601: calculating the distortion coefficient and the internal parameter of the camera based on the first coordinate system;

[0062] Step 602: de-distorting the image acquired by the camera based on the distortion coefficient, and determining the relative pose of the camera with respect to the first coordinate system based on the de-distorted image;

[0063] Step 603: substituting the relative pose, the internal parameter, the pose, and the position and unit vector of the feature point in the first coordinate system into a second target formula of a nonlinear optimization model to optimize the pose, wherein the second target formula is used to obtain a second target error based on the relative pose, the internal parameter, the pose, and the feature point, and the optimization of the pose is completed when the second target error takes the minimum value.

[0064] The second target formula is as follows:

[0065] wherein e is the second target error, is the pose of the camera, is the parameter to be optimized, that is, the relative pose of the camera and the checkerboard after optimization, ξ is the pose Lie algebra symbol, d i is the distance error of the camera position to the line at time i, θ i ​is the pose residual of the camera at time i, and m and n are positive integers.

[0066] After the pose in the embodiment of the application is optimized by the above two formulas, more accurate camera extrinsic parameters can be obtained.

[0067] In an optional implementation of the embodiment of the application, corresponding to the above Figure 1 The embodiment of the application also provides a device for calibrating electronic rearview mirror parameters, as shown in Figure 7 The device comprises:

[0068] The first processing module 701 is configured to calculate the pose of the camera corresponding to the electronic rearview mirror of the vehicle based on a first coordinate system during the process in which the vehicle drives into the entrance of the site and drives out of the exit of the site in a straight line at a constant speed, wherein the pose represents the pose of the camera corresponding to different moments during driving; and the first coordinate system is established based on a checkerboard, and the checkerboard is arranged on both sides of the entrance of the site.

[0069] In the specific example of the application, the checkerboard can be a 4*5 checkerboard, and each small square in the checkerboard is 20cm*20cm, as shown in Figure 2 In other examples of the embodiment of the application, the small squares in the checkerboard can be other shapes, such as circular squares, asymmetric circular squares, or circular ring squares. In addition, the size of the squares in the checkerboard can also be set according to actual needs. The above-mentioned 20cm*20cm is only an example.

[0070] In addition, the site in the embodiment of the application is a site with a straight road, so that the vehicle can drive at a constant speed during driving. In addition, the checkerboard for calibration can be placed on both sides of the entrance of the site in the embodiment of the application, and the distance between the checkerboard and the wheels is 30cm-100cm, and the two checkerboards are perpendicular to the driving direction of the vehicle, as shown in Figure 3 Further, one or more feature points are preset in the checkerboard. For example, the checkerboard is a small square, and the corner point (the common point of the adjacent corners of the black and white squares) in the checkerboard.

[0071] The second processing module 702 is configured to determine the straight line in which the pose of the camera during driving is located in the space and determine the unit vector of the straight line in the first coordinate system, and determine the first rotation matrix based on the unit vector; wherein the first rotation matrix is the matrix for converting from the second coordinate system to the first coordinate system, and the second coordinate system is established based on the starting position of the vehicle in the site.

[0072] In the specific examples of the present application, a first coordinate system is established based on the checkerboard, i.e., taking the plane of the checkerboard as the plane in which the x and y axes of the first coordinate system are located, specifically, the positive direction of the x axis is consistent with the driving direction of the vehicle, the y axis is perpendicular to the x axis, and the z axis is perpendicular to the plane in which the x and y axes are located, and taking a feature point on the checkerboard as the origin. Further, calibration can be performed based on the first coordinate system based on Zhang Zhengyou's calibration method, different pose pictures of the calibration board are collected, the pixel coordinates of the corner points in the pictures are extracted, the initial values of the internal parameters of the camera (f x ,f y ,u0,v0) are calculated through a homography matrix, the distortion coefficients (k1, k2) are estimated using a nonlinear least squares method, and then the pose (R i ,t i ) of the camera at time i can be obtained.

[0073] The third processing module 703 is configured to determine the attitude angle of the camera based on the corresponding pose of the camera, and determine a second rotation matrix based on the attitude angle and the first rotation matrix, wherein the second rotation matrix represents the external parameter of the camera.

[0074] With the device of the embodiments of the present application, only the checkerboard needs to be arranged in the built site, and then the corresponding pose of the camera of the electronic rearview mirror in the process of the vehicle driving into the site entrance and driving out of the site exit at a uniform speed in a straight line can be calculated according to the first coordinate system established based on the checkerboard, and then the straight line in which the pose of the camera in the space is located and the unit vector of the straight line in the first coordinate system can be determined, the first rotation matrix can be determined based on the unit vector, and finally the external parameter of the camera can be determined according to the attitude angle corresponding to the pose and the first rotation matrix. It can be seen that in the embodiments of the present application, there is no need to fix the site, and after changing the site, the external parameter of the camera can be calculated without strict measurement, which reduces the investment of calibration personnel and the difficulty of calibration operation, and solves the problem of low calibration efficiency caused by the complicated calibration process of the external parameter of the camera in the prior art.

[0075] In the optional implementation of the embodiments of the present application, the third processing module in the embodiments of the present application can further include: a first determination unit configured to determine the average value of the attitude angle of the camera based on the corresponding pose of the camera; and a first processing unit configured to obtain the ratio result of the average value of the attitude angle and the first rotation matrix, and determine the ratio result as the second rotation matrix.

[0076] In an optional implementation of the embodiments of the present application, before determining the attitude angle of the camera based on the corresponding pose of the camera, the device in the embodiments of the present application can further include: a fourth processing module configured to calculate the distortion coefficient related to the camera and the intrinsic parameter of the camera based on the first coordinate system; a fifth processing module configured to perform distortion removal processing on the image acquired by the camera based on the distortion coefficient, and determine the re-projection error of the feature points in the image after the distortion removal processing; and a sixth processing module configured to input the re-projection error, the intrinsic parameter, the pose, and the position and unit vector of the feature points in the first coordinate system into a first target formula of a non-linear optimization model to optimize the pose, wherein the first target formula is used to obtain a first target residual based on the re-projection error, the intrinsic parameter, the pose, and the feature points, and the optimization of the pose is completed when the first target residual takes the minimum value.

[0077] In an optional implementation of the embodiments of the present application, before determining the attitude angle of the camera based on the corresponding pose of the camera, the device in the embodiments of the present application can further include: a seventh processing module configured to calculate the distortion coefficient related to the camera and the intrinsic parameter of the camera based on the first coordinate system; an eighth processing module configured to perform distortion removal processing on the image acquired by the camera based on the distortion coefficient, and determine the relative pose of the camera relative to the first coordinate system based on the image after the distortion removal processing; and a ninth processing module configured to input the relative pose, the intrinsic parameter, the pose, and the position and unit vector of the feature points in the first coordinate system into a second target formula of a non-linear optimization model to optimize the pose, wherein the second target formula is used to obtain a second target residual based on the relative pose, the intrinsic parameter, the pose, and the feature points, and the optimization of the pose is completed when the second target residual takes the minimum value.

[0078] In an optional implementation of the embodiments of the present application, the first processing module in the embodiments of the present application can further include: a second processing unit configured to collect images through the camera every preset time interval, and delete the images in which all the feature points are not detected from the collected images, wherein each checkerboard is provided with a plurality of feature points; and a deletion unit configured to calculate the pose corresponding to the camera based on the first coordinate system and the images collected after the deletion.

[0079] In an optional implementation of the embodiments of the present application, the second processing module in the embodiments of the present application can further include: a third processing unit configured to acquire the poses of the camera at different time instants during the driving process, and determine a target straight line based on the poses of the camera at different time instants, wherein the sum of the distances from the poses of the camera at different time instants to the target straight line is less than the sum of the target distances, and the sum of the target distances is the sum of the distances from the poses of the camera at different time instants to any straight line in the space except the target straight line; and a second determination unit configured to determine the target straight line as the straight line in the space where the pose of the camera during the driving process is located.

[0080] AsFigure 8 As shown in the figure, the embodiment of the present application provides an electronic device, comprising a processor 811, a communication interface 812, a memory 813 and a communication bus 814, wherein the processor 811, the communication interface 812 and the memory 813 complete mutual communication through the communication bus 814,

[0081] The memory 813 is used for storing a computer program.

[0082] In an embodiment of the present application, the processor 811 is used for executing the program stored in the memory 813, and realizes the calibration method of the electronic rearview mirror parameter provided by any one of the foregoing method embodiments, and the role is similar, which will not be repeated here.

[0083] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the calibration method of the electronic rearview mirror parameter provided by any one of the foregoing method embodiments.

[0084] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0085] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus a general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0086] It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order

[0087] The above description is merely that of the specific embodiments of the application and as such is not to be taken in a limiting sense. Various modifications and alterations of the embodiments described herein will become apparent to those skilled in the art from the foregoing description, which does not limit the generality presented. It is the intention that all such modifications and alterations be considered equaliy by the spirit and scope of this application. It is therefore intended to cover in the appended claims all such changes and alterations that come within the scope of this application.

Claims

1. A method for calibrating parameters of an electronic rearview mirror, characterized in that, include: As the vehicle enters the venue from the entrance and exits the venue in a straight line at a constant speed, the pose of the camera corresponding to the electronic rearview mirror of the vehicle is calculated based on the first coordinate system. The pose represents the pose of the camera at different times during the driving process. The first coordinate system is established based on a chessboard grid, and the chessboard grid is set on both sides of the venue entrance. The system determines the straight line in space where the camera pose is located during the driving process and determines the unit vector of the straight line in the first coordinate system, and determines a first rotation matrix based on the unit vector; wherein, the first rotation matrix is ​​a matrix that transforms from the second coordinate system to the first coordinate system, and the second coordinate system is established based on the starting position of the vehicle in the field; The attitude angle of the camera is determined based on the corresponding pose of the camera, and a second rotation matrix is ​​determined based on the attitude angle and the first rotation matrix, wherein the second rotation matrix represents the extrinsic parameters of the camera.

2. The method according to claim 1, characterized in that, Determining the camera's attitude angle based on the camera's corresponding pose, and determining the second rotation matrix based on the attitude angle and the first rotation matrix, includes: The average value of the camera's attitude angle is determined based on the corresponding pose of the camera; Obtain the ratio of the average value of the attitude angle to the first rotation matrix, and determine the ratio as the second rotation matrix.

3. The method according to claim 1, characterized in that, Before determining the camera's attitude angle based on the camera's corresponding pose, the method further includes: The distortion coefficients and intrinsic parameters of the camera are calculated based on the first coordinate system. The distortion coefficients are used to perform distortion correction processing on the images acquired by the camera, and the reprojection error of feature points in the distortion-corrected images is determined. The reprojection error, the intrinsic parameters, the pose, the position of the feature point in the first coordinate system, and the unit vector are substituted into the first objective formula of the nonlinear optimization model to optimize the pose. The first objective formula is used to obtain a first objective residual based on the reprojection error, the intrinsic parameters, the pose, and the feature point, and the pose optimization is completed when the first objective residual is minimized.

4. The method according to claim 3, characterized in that, The first target formula is: Where e is the residual of the first target, i is time, and j is the feature point number. Let K be the depth from the feature point to the image, and K be the transformation matrix of the intrinsic parameters, exp(ξ). i ∧ Let ξ be the pose of the camera at time i, and let ξ be the Lie algebraic representation of the pose. Let d be the reprojection residual of the feature point at time i. i Let θ be the residual distance from the camera position to the line at time i. i Let m be the camera pose residual at time i, and m and n be positive integers.

5. The method according to claim 1, characterized in that, Before determining the camera's attitude angle based on the camera's corresponding pose, the method further includes: The distortion coefficients and intrinsic parameters of the camera are calculated based on the first coordinate system. The image acquired by the camera is subjected to distortion correction processing based on the distortion coefficient, and the relative pose of the camera with respect to the first coordinate system is determined based on the image after distortion correction processing. The relative pose, the intrinsic parameters, the pose and the position of the feature point in the first coordinate system, and the unit vector are substituted into the second objective formula of the nonlinear optimization model to optimize the pose. The second objective formula is used to obtain a second objective residual based on the relative pose, the intrinsic parameters, the pose and the feature point, and the pose optimization is completed when the value of the second objective residual is minimized.

6. The method according to claim 1, characterized in that, Calculating the pose of the camera corresponding to the electronic rearview mirror of the vehicle based on the first coordinate system includes: Images are captured by the camera at preset time intervals, and images from which not all feature points are detected are deleted. Each chessboard square contains multiple feature points. The pose of the camera is calculated based on the first coordinate system and the image acquired after image deletion.

7. The method according to claim 1, characterized in that, Determining the straight line in space where the camera pose lies during driving includes: The poses of the camera at different times during the driving process are acquired, and a target straight line is determined based on the poses of the camera at different times. The sum of the distances from the poses of the camera at different times to the target straight line is less than the sum of the target distances. The sum of the target distances is the sum of the distances from the poses of the camera at different times to any straight line in space other than the target straight line. The target straight line is defined as the straight line in space where the camera pose is located during the driving process.

8. A calibration device for electronic rearview mirror parameters, characterized in that, include: The first processing module is used to calculate the pose of the camera corresponding to the electronic rearview mirror of the vehicle based on a first coordinate system during the process of the vehicle entering the venue from the entrance and exiting the venue in a uniform straight line. The pose represents the pose of the camera at different times during the driving process. The first coordinate system is established based on a chessboard grid, and the chessboard grid is set on both sides of the venue entrance. The second processing module is used to determine the straight line in space where the camera pose is located during the driving process and to determine the unit vector of the straight line in the first coordinate system, and to determine a first rotation matrix based on the unit vector; wherein, the first rotation matrix is ​​a matrix that transforms from the second coordinate system to the first coordinate system, and the second coordinate system is established based on the starting position of the vehicle in the site; The third processing module is used to determine the attitude angle of the camera based on the corresponding pose of the camera, and to determine the second rotation matrix based on the attitude angle and the first rotation matrix, wherein the second rotation matrix represents the extrinsic parameters of the camera.

9. An electronic device, comprising: At least one communication interface; At least one bus connected to the at least one communication interface; At least one processor connected to the at least one bus; At least one memory connected to the at least one bus, wherein the processor is configured to perform a calibration method for the electronic rearview mirror parameters according to any one of claims 1 to 7.

10. A computer storage medium storing computer-executable instructions for performing a calibration method for the parameters of an electronic rearview mirror according to any one of claims 1 to 7.

Images