External parameter calibration method of vehicle-mounted around-view camera and storage medium

Abstract

The invention discloses an external parameter calibration method of a vehicle-mounted around-view camera and a storage medium, and the method comprises the steps: projecting an image point on an imaging plane of a camera to the ground through employing a reverse projection process, and calculating a reverse projection error between two adjacent cameras through employing a reverse projection point obtained in the reverse projection process; an objective function is constructed based on the inverse projection error, and finally, the objective function is utilized to adjust the external parameters of the vehicle-mounted around-view camera to complete calibration. The world coordinate of each image point does not need to be calculated in the constructed target function, so that the calculation amount in the calibration process is greatly reduced, and the camera calibration efficiency is improved; and when the inverse projection process is utilized, because the image points are uniformly projected to the ground, the influence of the camera height arrangement can be avoided, the problem that the calibration effect is poor due to the fact that the calculation of the re-projection error in the classical BA algorithm is influenced by the camera arrangement height is solved, and the calibration effect of the vehicle-mounted around-view camera can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an external parameter calibration method and storage medium for a vehicle-mounted surround view camera. Background Art

[0002] With the rapid development of road traffic and the increase of people's safety awareness, the Advanced Driver Assistance System (ADAS) that can help people reduce driving stress and improve safety factors has gained more and more attention. The Advanced Driver Assistance System uses various sensors installed on the car to sense the surrounding environment at any time during the driving process of the car, and combines the navigation map data to perform systematic calculations and analysis, so as to make the driver aware of possible dangers in advance. The on-board surround view camera plays an important role in the stable operation of the Advanced Driver Assistance System (ADAS). The use of on-board surround view cameras can effectively increase the safety of car driving.

[0003] There may be large differences in the installation height of the on-board surround view cameras. In this case, the classic bundle adjustment (BA) method is usually used to calibrate the camera extrinsics. The classic BA algorithm uses the minimum reprojection error as the objective function for calibration, but the reprojection error is affected by the camera layout height. In actual vehicles, the front and rear cameras may be much higher than the left and right cameras. In this case, the reprojection error of the front and rear cameras may be more than an order of magnitude lower than that of the left and right cameras. Continuing to use the BA algorithm for joint calibration will not produce good results. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an external parameter calibration method and storage medium for a vehicle-mounted surround view camera, which can effectively improve the calibration effect and improve the calibration efficiency of the vehicle-mounted surround view camera.

[0005] To achieve the above object, an embodiment of the present invention provides an external parameter calibration method for a vehicle-mounted surround view camera, comprising:

[0006] Determine the image points of all adjacent camera groups in the vehicle-mounted surround view camera on their imaging planes; wherein the adjacent camera groups include two adjacent cameras;

[0007] Calculating coordinates of inverse projection points on the ground of image points corresponding to two adjacent cameras in the adjacent camera group;

[0008] Calculating the inverse projection error between two adjacent cameras in each adjacent camera group according to the coordinates of the inverse projection point, and constructing the objective function of the vehicle-mounted surround view camera with the inverse projection error corresponding to each adjacent camera group;

[0009] The external parameters of the vehicle-mounted surround view camera are adjusted according to the objective function until the preset calibration completion condition is met and the adjustment of the external parameters is stopped.

[0010] As an improvement of the above solution, the step of calculating the coordinates of the inverse projection points on the ground of the image points corresponding to the two adjacent cameras in the adjacent camera group includes:

[0011] Obtaining external parameters of two adjacent cameras in the adjacent camera group;

[0012] The coordinates of the inverse projection points of the image points corresponding to the two adjacent cameras on the ground are calculated according to the external parameters.

[0013] As an improvement of the above solution, the external parameters include a rotation matrix and a translation vector; then, calculating the coordinates of the inverse projection points of the image points corresponding to the two adjacent cameras on the ground according to the external parameters includes:

[0014] For each image point on the camera, calculate the transposed matrix of the rotation matrix corresponding to the current camera;

[0015] Calculate the first coordinate of the current image point in the world coordinate system according to the transposed matrix, the translation vector and the coordinate of the current image point in the camera coordinate system, and obtain the second coordinate of the camera viewpoint corresponding to the current image point in the world coordinate system;

[0016] The coordinates of the reverse projection point of the current image point on the ground are calculated according to the first coordinates and the second coordinates.

[0017] As an improvement of the above solution, the step of calculating the coordinates of the reverse projection point of the current image point on the ground according to the first coordinates and the second coordinates includes:

[0018] Calculate the abscissa of the back-projection point according to the abscissa and the ordinate in the first coordinate and the abscissa and the ordinate in the second coordinate;

[0019] Calculate the ordinate of the back-projection point according to the ordinate and the ordinate in the first coordinate and the ordinate and the ordinate in the second coordinate;

[0020] The vertical coordinate of the back-projected point is set to 0.

[0021] As an improvement of the above solution, after constructing the objective function of the vehicle-mounted surround view camera, the method further includes:

[0022] Obtaining a first height of a left camera and a second height of a right camera in the vehicle-mounted surround view camera;

[0023] Calculating a first ratio of a back-projection error including the left camera to the first height, and calculating a second ratio of a back-projection error including the right camera to the second height;

[0024] The objective function is updated according to the first ratio and the second ratio.

[0025] As an improvement of the above solution, after stopping adjusting the external parameter, the method further includes:

[0026] Get the theoretical position of each camera;

[0027] Each camera is translated and rotated as a whole to the corresponding theoretical position.

[0028] As an improvement of the above solution, the calibration completion condition includes at least one of the following:

[0029] The number of adjustments of the external parameter reaches a preset threshold of the number of iterations;

[0030] Between two consecutive iterations, the change in the external parameter is less than the preset change threshold;

[0031] The ratio of the back projection error after adjusting the external parameter to the initial back projection error is less than a preset percentage threshold.

[0032] To achieve the above objectives, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the external parameter calibration method for the vehicle-mounted surround view camera as described in any of the above embodiments.

[0033] Compared with the prior art, the extrinsic parameter calibration method and storage medium of the vehicle-mounted surround view camera disclosed in the present invention projects the image points on the camera imaging plane to the ground through the back-projection process, and uses the back-projection points obtained through the back-projection process to calculate the back-projection error between two adjacent cameras, constructs an objective function based on the back-projection error, and finally uses the objective function to adjust the extrinsic parameters of the vehicle-mounted surround view camera to complete the calibration. Since the world coordinates of each image point do not need to be calculated in the constructed objective function, the amount of calculation in the calibration process is greatly reduced, thereby improving the efficiency of camera calibration; and when using the back-projection process, since the image points are uniformly projected to the ground, they are not affected by the camera height arrangement, which solves the problem that the calculation of the re-projection error in the classic BA algorithm is affected by the camera arrangement height, resulting in poor calibration effect, and can effectively improve the calibration effect of the vehicle-mounted surround view camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of imaging of a vehicle-mounted surround view camera;

[0035] Figure 2 This is the principle diagram of camera imaging;

[0036] Figure 3 It is a schematic diagram of the classic BA algorithm provided by the prior art;

[0037] Figure 4 This is a schematic diagram of the imaging difference between points on the ground and cameras at different heights;

[0038] Figure 5 This is a calibration effect diagram obtained by using the classic BA algorithm when the front and rear cameras provided by the prior art are much lower than the left and right cameras;

[0039] Figure 6 is a flow chart of an external parameter calibration method for a vehicle-mounted surround view camera provided by an embodiment of the present invention;

[0040] Figure 7 is a schematic diagram of the reverse projection joint calibration provided by an embodiment of the present invention;

[0041] Figure 8 It is a schematic diagram of camera position compensation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The in-vehicle surround view system requires four cameras installed on the front, back, left and right sides of the car to collect images, and then transmit the collected images to the system host. The host uses algorithms to synchronize, synthesize, and splice the four images, and finally sends the processed images to the LCD of the car's central control, so that the driver can observe the surrounding conditions of the vehicle without blind spots during driving or reversing to avoid accidents. The presence of the in-vehicle surround view camera allows the driver to clearly check whether there are obstacles around the vehicle and understand the relative position and distance of the obstacles, helping the driver to park the vehicle easily. It is not only very intuitive, but also has no blind spots. It can improve the driver's ability to calmly control the vehicle to park or pass through complex roads, and effectively reduce the occurrence of scratches, collisions, sinking and other accidents. See Figure 1 , Figure 1It is an imaging diagram of a vehicle-mounted surround-view camera. Vehicle-mounted surround-view cameras generally have a large field of view (FOV). After four cameras are arranged on the front, back, left and rear of the vehicle body at the same time, the imaging ranges of two adjacent cameras will have a large overlapping area (such as the left front, left rear, right front and right rear). In the overlapping area, each point will be imaged by two cameras respectively, which forms a constraint. Therefore, the BA algorithm is often used in the prior art to jointly optimize the posture of each camera, thereby improving the image stitching effect in the overlapping area.

[0044] Each camera has a viewpoint O, which can be simply understood as the location of the camera. Figure 2 , Figure 2 It is the principle diagram of camera imaging. The intersection point p of the line connecting a point P and O in the real world and the imaging plane is the image point of the world point P in the camera. Since the present invention only adjusts the external parameters, the imaging plane is replaced by the normalized plane. The distance from O to the normalized plane is 1, which eliminates the influence of the camera's internal parameters on the imaging. If the camera's external parameters are (R, t), the imaging process from P (X, Y, Z) in the world coordinate system to p (x, y) in the camera coordinate system can be expressed as:

[0045]

[0046] Where R is a 3x3 rotation matrix and t is a 3x1 translation vector. The formula implies the normalization process from the right side of the equation to the left side.

[0047] See also Figure 3 , Figure 3 This is the principle diagram of the classic BA algorithm provided by the existing technology. The same world point P is imaged as p1 and p2 in two cameras respectively. The images calculated by the current external parameters of the camera should be p1' and p2'. However, due to the error of the external parameters, p1 and p1', p2 and p2' do not coincide. Therefore, in the classic BA algorithm, it is necessary to adjust the external parameters of the two cameras to make p1 and p1', p2 and p2' coincide as much as possible. The error between the projection point p' calculated by the external parameters and the actual projection point p is called the reprojection error. The classic BA algorithm takes the minimum reprojection error as the objective function, and optimizes the external parameters (R, t) of each camera and the world point coordinates P (X, Y, Z). The objective function constructed in the classic BA algorithm is:

[0048]

[0049] Among them, R 1 and R 2 Represents the rotation matrix of camera 1 and camera 2 respectively; t 1 and t 2 Represent the translation vectors of camera 1 and camera 2 respectively; P i represents the i-th world point, i = 1, 2, ..., n; p 1iand p 2i They represent the image points of the i-th world point on camera 1 and camera 2 respectively.

[0050] For points on the ground, the reprojection error is affected by the camera placement height. For the same distance on the ground, the imaging distances of cameras at different heights vary greatly. For a schematic diagram of the imaging difference of points on the ground on cameras at different heights, please refer to Figure 4 In actual vehicles, the front and rear cameras may be much lower than the left and right cameras. In this case, the reprojection error of the front and rear cameras may be more than one order of magnitude lower than that of the left and right cameras. The results obtained by continuing to use the classic BA algorithm are as follows: Figure 5 As shown, Figure 5 The left picture in the figure shows the ideal stitching effect, and the right picture shows the stitching effect obtained after the camera is calibrated using the classic BA algorithm. It can be seen that after using the BA algorithm to calibrate the camera, the image stitching effect becomes worse (the image becomes narrower).

[0051] In view of the above defects, considering that the reprojection error will be significantly affected by the camera arrangement, an embodiment of the present invention provides an extrinsic parameter calibration method for a vehicle-mounted surround view camera based on back-projection error, see Figure 6 , Figure 6 : is a flow chart of an extrinsic parameter calibration method for a vehicle-mounted surround view camera provided by an embodiment of the present invention, and the extrinsic parameter calibration method for a vehicle-mounted surround view camera comprises:

[0052] S1, determining image points of all adjacent camera groups in the vehicle-mounted surround view camera on their imaging planes; wherein the adjacent camera groups include two adjacent cameras;

[0053] S2, calculating the coordinates of the inverse projection points on the ground of the image points corresponding to two adjacent cameras in the adjacent camera group;

[0054] S3, calculating the back projection error between two adjacent cameras in each adjacent camera group according to the coordinates of the back projection point, and constructing the objective function of the vehicle-mounted surround view camera with the back projection error corresponding to each adjacent camera group;

[0055] S4. Adjusting the external parameters of the vehicle-mounted surround view camera according to the objective function until the preset calibration completion condition is met and the adjustment of the external parameters is stopped.

[0056] Specifically, in step S1, since the vehicle-mounted surround view camera is composed of four cameras, namely, front, rear, left, and right, there will be overlapping areas in the images captured by two adjacent cameras. Therefore, when considering image stitching, it is necessary to consider the position and posture between the two adjacent cameras and calibrate the two adjacent cameras, thereby completing the calibration of the four cameras in the vehicle-mounted surround view camera. Based on this, first determine all adjacent camera groups in the vehicle-mounted surround view camera, each adjacent camera group includes two adjacent cameras, such as the adjacent camera groups: left camera + front camera, left camera + rear camera, right camera + front camera, right camera + rear camera. After obtaining the adjacent camera group, for each adjacent camera group, the image points of the two adjacent cameras on their imaging planes are determined respectively. There are multiple image points between the two adjacent cameras, and they have a one-to-one correspondence. For example, after determining the image point p1 of the left camera (camera 1) on its imaging plane, it is necessary to find the corresponding image point p2 on the imaging plane of the front camera (camera 2) based on this image point p1. At this time, the coordinates of p2 can be found according to the coordinates of p1, the coordinates of the left camera and the coordinates of the front camera.

[0057] Specifically, in step S2, the inverse projection transformation is a transformation method for inverse mapping the projection transformation existing in the process of the camera capturing the image of the application scene. The image of the application scene captured by the camera is obtained by mapping the reflection of each object in the world coordinate system to the camera plane, and the inverse projection transformation is to inversely map the image captured by the camera to obtain the actual physical coordinates of the object in the image in the world coordinate system. Different from the projection process of the classic BA algorithm of "first determining the world point, and then projecting the same world point on the imaging planes of the two cameras to obtain the image point", the inverse projection process of the embodiment of the present invention is to first determine the image point, and then project the image point to the ground, wherein the image point projected to the ground is called the inverse projection point.

[0058] The reverse projection process provided by the embodiment of the present invention can be referred to Figure 7 , the reverse projection points P1 and P2 are calculated using external parameters from the image points p1 and p2 of different cameras. Since the image loses one-dimensional information, P here is located on the ground, which is convenient for calculating the reverse projection. Adjust the camera external parameters to make P1 and P2 coincide as much as possible. If the reverse projection process from p to P is represented by f, that is, P = f(p).

[0059] Specifically, in step S2, the calculation of the coordinates of the inverse projection points of the image points corresponding to two adjacent cameras in the adjacent camera group on the ground includes: obtaining the external parameters of the two adjacent cameras in the adjacent camera group; and calculating the coordinates of the inverse projection points of the image points corresponding to the two adjacent cameras on the ground according to the external parameters.

[0060] Exemplarily, the extrinsic parameters include a rotation matrix R and a translation vector t, and the preset theoretical extrinsic parameters are used as initial values ​​to jointly optimize the camera extrinsic parameters based on the inverse projection error. In the embodiment of the present invention, the inverse projection process is performed by relying on extrinsic parameters. Then, the coordinates of the inverse projection points of the image points corresponding to the two adjacent cameras on the ground are calculated according to the extrinsic parameters, including: for each image point on the camera, the transposed matrix of the rotation matrix corresponding to the current camera is calculated; the first coordinates of the current image point in the world coordinate system are calculated according to the transposed matrix, the translation vector and the coordinates of the current image point in the camera coordinate system, and the second coordinates of the camera viewpoint corresponding to the current image point in the world coordinate system are obtained; the coordinates of the inverse projection point of the current image point on the ground are calculated according to the first coordinates and the second coordinates.

[0061] It is worth mentioning that the world coordinate system is also called the measurement coordinate system. It is a three-dimensional rectangular coordinate system. It can be used as a reference to describe the spatial position of the camera and the object to be measured. The position of the world coordinate system can be freely determined according to the actual situation. The camera coordinate system is also a three-dimensional rectangular coordinate system. The origin is located at the optical center of the camera lens. The x and y axes are parallel to the two sides of the image plane respectively. The z axis is the optical axis of the lens and is perpendicular to the image plane.

[0062] Specifically, for each image point on the camera, the transposed matrix of the rotation matrix corresponding to the current camera is calculated, and then the first coordinate of the current image point in the world coordinate system is calculated according to the transposed matrix, the translation vector and the coordinate of the current image point in the camera coordinate system, satisfying the following formula:

[0063]

[0064] Among them, x w is the horizontal coordinate in the first coordinate; y w is the ordinate in the first coordinate system; z w is the vertical coordinate in the first coordinate system; p is the coordinate of the current image point in the camera coordinate system; R′ is the transposed matrix; and t is the translation vector.

[0065] Specifically, the second coordinate of the camera viewpoint corresponding to the current image point in the world coordinate system is obtained, satisfying the following formula:

[0066]

[0067] Among them, O x is the horizontal coordinate in the second coordinate; O y is the ordinate in the second coordinate; O z is the vertical coordinate in the second coordinate.

[0068] Specifically, the coordinates of the reverse projection point of the current image point on the ground are calculated based on the first coordinates and the second coordinates, including: calculating the horizontal coordinate of the reverse projection point based on the horizontal coordinate and the vertical coordinate in the first coordinate and the horizontal coordinate and the vertical coordinate in the second coordinate; calculating the vertical coordinate of the reverse projection point based on the vertical coordinate and the vertical coordinate in the first coordinate and the vertical coordinate in the second coordinate; and setting the vertical coordinate of the reverse projection point to 0.

[0069] Exemplarily, the coordinates of the back-projection point satisfy the following formula:

[0070]

[0071] Wherein, X is the horizontal coordinate of the reverse projection point, Y is the vertical coordinate of the reverse projection point, and Z is the vertical coordinate of the reverse projection point.

[0072] Specifically, in step S3, after obtaining the coordinates of the back-projection points, the back-projection error between two adjacent cameras in each adjacent camera group is calculated according to the coordinates of the back-projection points, and the calibration of the two cameras satisfies the following formula:

[0073]

[0074] Among them, R 1 and R 2 Represents the rotation matrix of camera 1 and camera 2 respectively; t 1 and t 2 Represent the translation vectors of camera 1 and camera 2 respectively; p 1i and p 2i represents the i-th image point on camera 1 and camera 2 respectively; f(p 1i ) and f(p 2i ) represent the inverse projection point of the i-th image point in camera 1 and camera 2 to the ground; f(p 1i )-f(p 2i ) represents the back-projection error between camera 1 and camera 2.

[0075] After obtaining the back projection error between two adjacent cameras in all adjacent camera groups (4 groups), the objective function of the vehicle-mounted surround view camera is constructed using the back projection error corresponding to each adjacent camera group. The objective function satisfies the following formula:

[0076]

[0077] Among them, R f , R b , R l , R r Respectively represent the rotation matrices of the front camera, rear camera, left camera, and right camera; tf ,t b ,t l ,t r Respectively represent the translation vectors of the front camera, rear camera, left camera, and right camera; p fi 、p bi 、p li 、p ri Respectively represent the i-th image point on the front camera, rear camera, left camera, and right camera; f(p fi )、f(p bi )、f( li )、f( ri ) represent the inverse projection points of the i-th image point on the front camera, rear camera, left camera, and right camera respectively.

[0078] In the embodiment of the present invention, since the constructed objective function does not need to calculate the world coordinates P of each image point i , which greatly reduces the amount of calculation in the calibration process and makes the solution faster, thus improving the efficiency of camera calibration.

[0079] Specifically, in step S4, the external parameters of the vehicle-mounted surround view camera are adjusted according to the objective function. It can be understood that the external parameters of the four cameras need to be adjusted simultaneously each time the adjustment is made until the preset calibration completion conditions are met, and the adjustment of the external parameters is stopped, and the calibration is completed. The specific adjustment process of the external parameters can refer to the existing camera calibration technology, which will not be repeated here. Optionally, the calibration completion condition includes at least one of the following: the number of adjustments of the external parameters reaches a preset iteration number threshold; the change in the external parameters between two adjacent iterations is less than a preset change threshold; the ratio of the back projection error after adjusting the external parameters to the initial back projection error is less than a preset percentage threshold. It is worth noting that the iteration number threshold, the change threshold and the percentage threshold can be set by the user when calibrating the vehicle-mounted surround view camera, and are not specifically limited here.

[0080] Furthermore, the joint calibration process based on inverse projection still has the following problems: 1. The camera and the target can be arbitrarily scaled as a whole, and the residual changes proportionally with the scaling; 2. The camera and the target can be arbitrarily translated and rotated in the XY plane as a whole, that is, the solution is not unique.

[0081] Specifically, in order to solve the scaling problem, the objective function can be divided by the current camera height to normalize the residual. At this time, after constructing the objective function of the vehicle-mounted surround view camera, the method further includes: obtaining a first height of a left camera and a second height of a right camera in the vehicle-mounted surround view camera; calculating a first ratio of the back projection error containing the left camera to the first height, and calculating a second ratio of the back projection error containing the right camera to the second height; updating the objective function according to the first ratio and the second ratio.

[0082] For example, considering the on-board surround view camera, the height difference between the left and right cameras is generally not large, while the arrangement of the front and rear cameras may be significantly different from other cameras. In the embodiment of the present invention, the first height of the left camera is 0. zl and the second height O of the right camera zr As the denominator, the updated objective function is as follows:

[0083]

[0084] Specifically, for the overall translation and rotation problem in the XY plane, although the solved camera position may deviate significantly from the original position, this deviation is limited to the position, not the angle, and the relative position between the cameras is still accurate. On the other hand, the purpose of the extrinsic calibration of the vehicle-mounted camera is to solve the relative relationship between the camera and the vehicle body, and this relationship has a theoretical value in the design stage. Although there are errors in the vehicle body assembly stage, the vehicle body assembly error is still lower than the sum of a series of errors in the calibration process such as target processing, vehicle positioning, and feature point detection. Based on this, with the camera theoretical position as the goal, the overall camera position obtained by the joint calibration can be translated and rotated to the position closest to the theoretical position. This process is called camera position compensation. After stopping adjusting the external parameters, the method also includes: obtaining the theoretical position of each camera; translating and rotating each camera as a whole to the corresponding theoretical position.

[0085] For example, see Figure 8 , Figure 8 : is a schematic diagram of camera position compensation provided by an embodiment of the present invention. The left figure is a schematic diagram of the position of the vehicle-mounted surround view camera before position compensation, and the right figure is a schematic diagram of the position of the vehicle-mounted surround view camera after position compensation. The specific method of camera position compensation can be expressed as follows: two sets of points are known in the plane, and the pairing relationship between the two sets of points is known, and the two sets of points are made to coincide by rotation and translation. This process can be achieved through a variety of methods such as direct solution and iterative optimization. Here is a direct solution method for reference: Let the theoretical coordinates of the camera be (X i , Y i ), the optimized coordinates are (x i ,y i), calculate their mean coordinates (X avg ,Y avg ),(x avg ,y avg ), then the required translation vector is (X avg -x avg ,Y avg -y avg ). Subtract the corresponding mean coordinates from the two sets of coordinates to get the decentered coordinates (X i ', Y i '), (x i ',y i '), (X i ', Y i ') form a 2x4 matrix P M , P M ′ is P M The transposed matrix, (x i ',y i ') forms a 2x4 matrix p M , let matrix A = P M ′*p M , perform singular value decomposition on A to obtain A = UWV (U is the left singular matrix, W is the diagonal matrix, V is the right singular matrix, the specific meaning of UWV and the singular value decomposition process can be referred to the prior art, and will not be repeated here), then the required rotation matrix is ​​U*V. Finally, based on the obtained translation vector (X avg -x avg ,Y avg -y avg ) and the rotation matrix U*V are used to compensate the position of the vehicle-mounted surround view camera.

[0086] Compared with the prior art, the extrinsic parameter calibration method of the vehicle-mounted surround view camera disclosed in the present invention projects the image points on the camera imaging plane to the ground through the back-projection process, and uses the back-projection points obtained through the back-projection process to calculate the back-projection error between two adjacent cameras, constructs an objective function based on the back-projection error, and finally uses the objective function to adjust the extrinsic parameters of the vehicle-mounted surround view camera to complete the calibration. Since the world coordinates of each image point do not need to be calculated in the constructed objective function, the amount of calculation in the calibration process is greatly reduced, thereby improving the efficiency of camera calibration; and when using the back-projection process, since the image points are uniformly projected to the ground, they are not affected by the camera height arrangement, which solves the problem that the calculation of the re-projection error in the classic BA algorithm is affected by the camera arrangement height, resulting in poor calibration effect, and can effectively improve the calibration effect of the vehicle-mounted surround view camera.

[0087] Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the external parameter calibration method for the vehicle-mounted surround view camera as described in any of the above embodiments.

[0088] Exemplarily, the computer program may be divided into one or more modules / units, and the one or more modules / units are stored in a memory and executed by a processor to accomplish the present invention.

[0089] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the vehicle, and uses various interfaces and lines to connect various parts of the entire vehicle.

[0090] The memory can be used to store the computer program and / or module, and the processor implements the external parameter calibration method of the vehicle-mounted surround view camera described in the above embodiment by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0091] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for calibrating external parameters of a vehicle-mounted surround view camera, characterized in that: include: Determine the image points of all adjacent camera groups in the vehicle-mounted surround view camera on their imaging planes; wherein the adjacent camera groups include two adjacent cameras; Calculating coordinates of inverse projection points on the ground of image points corresponding to two adjacent cameras in the adjacent camera group; Calculating the inverse projection error between two adjacent cameras in each adjacent camera group according to the coordinates of the inverse projection point, and constructing the objective function of the vehicle-mounted surround view camera with the inverse projection error corresponding to each adjacent camera group; The external parameters of the vehicle-mounted surround view camera are adjusted according to the objective function until the preset calibration completion condition is met and the adjustment of the external parameters is stopped.

2. The extrinsic parameter calibration method of the vehicle-mounted surround view camera according to claim 1, characterized in that: The calculating the coordinates of the inverse projection points on the ground of the image points corresponding to the two adjacent cameras in the adjacent camera group includes: Obtaining external parameters of two adjacent cameras in the adjacent camera group; The coordinates of the inverse projection points of the image points corresponding to the two adjacent cameras on the ground are calculated according to the external parameters.

3. The extrinsic parameter calibration method of the vehicle-mounted surround view camera as claimed in claim 2, characterized in that: The external parameters include a rotation matrix and a translation vector; then, calculating the coordinates of the inverse projection points of the image points corresponding to the two adjacent cameras on the ground according to the external parameters includes: For each image point on the camera, calculate the transposed matrix of the rotation matrix corresponding to the current camera; Calculate the first coordinate of the current image point in the world coordinate system according to the transposed matrix, the translation vector and the coordinate of the current image point in the camera coordinate system, and obtain the second coordinate of the camera viewpoint corresponding to the current image point in the world coordinate system; The coordinates of the reverse projection point of the current image point on the ground are calculated according to the first coordinates and the second coordinates.

4. The extrinsic parameter calibration method of the vehicle-mounted surround view camera as claimed in claim 3, characterized in that: The step of calculating the coordinates of the reverse projection point of the current image point on the ground according to the first coordinates and the second coordinates includes: Calculate the abscissa of the back-projection point according to the abscissa and the ordinate in the first coordinate and the abscissa and the ordinate in the second coordinate; Calculate the ordinate of the back-projection point according to the ordinate and the ordinate in the first coordinate and the ordinate and the ordinate in the second coordinate; The vertical coordinate of the back-projection point is set to 0.

5. The extrinsic parameter calibration method of the vehicle-mounted surround view camera according to claim 1, characterized in that: After constructing the objective function of the vehicle-mounted surround view camera, the method further includes: Obtaining a first height of a left camera and a second height of a right camera in the vehicle-mounted surround view camera; Calculating a first ratio of a back-projection error including the left camera to the first height, and calculating a second ratio of a back-projection error including the right camera to the second height; The objective function is updated according to the first ratio and the second ratio.

6. The extrinsic parameter calibration method of the vehicle-mounted surround view camera according to claim 1, characterized in that: After stopping adjusting the external parameter, the method further includes: Get the theoretical position of each camera; Each camera is translated and rotated as a whole to the corresponding theoretical position.

7. The extrinsic parameter calibration method of a vehicle-mounted surround view camera according to claim 1, characterized in that: The calibration completion condition includes at least one of the following: The number of adjustments of the external parameter reaches a preset threshold of the number of iterations; Between two consecutive iterations, the change in the external parameter is less than the preset change threshold; The ratio of the back projection error after adjusting the external parameter to the initial back projection error is less than a preset percentage threshold.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the external parameter calibration method for the vehicle-mounted surround view camera according to any one of claims 1 to 7.

Images