Vehicle camera calibration correction method and device, equipment and storage medium

Abstract

The invention provides a vehicle camera calibration correction method, device and equipment and a storage medium, and the method comprises the steps: obtaining the current position and the current yaw angle of each camera in a calibration cloth coordinate system obtained through the calibration, and carrying out the calibration of each camera based on the standard calibration position and the current position of the camera in a vehicle body coordinate system; and correcting the current yaw angle to obtain a corrected yaw angle of the camera in the vehicle body coordinate system, and determining a predicted position of the camera in the calibration coordinate system after the yaw angle is corrected, so that the predicted position of the camera is corrected according to the standard calibration position of the camera to obtain a corrected position of the camera in the vehicle body coordinate system. According to the method, the yaw angle and the position in each path of camera calibration result are corrected, the camera calibration precision is improved, the problem of abnormal panoramic image splicing effect is solved, the splicing quality is improved, a more real scene is displayed by the panoramic image, and thus higher precision is shown in the use process of automatic parking.

Description

Technical Field

[0001] This application relates to the technical field of vehicle cameras, and particularly relates to a calibration and correction method, device, equipment and storage medium for vehicle cameras. Background Art

[0002] The panoramic imaging system plays a crucial role in modern vehicles. During reverse parking, parking, and slow driving, the driver can observe the panoramic view, greatly improving driving safety and convenience. To achieve this function, four surround-view cameras around the vehicle, front, rear, left, and right, are relied on, and a panoramic view is formed through image stitching technology and displayed on the in-vehicle display screen. Therefore, the calibration of the surround-view cameras becomes a key factor for image stitching accuracy and viewing angle accuracy.

[0003] However, there are certain problems with current after-sales calibration: it is difficult for the vehicle to align with the calibration cloth as precisely as in factory calibration. At the same time, it is also difficult to keep the ground where the calibration cloth is located flat. It entirely depends on the driver to park the vehicle to ensure alignment with the calibration cloth, resulting in an increase in the error between the calibrated camera position and the actual camera position, and further increasing the error in the panoramic image and the mosaic, affecting the driver's perception and the accuracy of the automatic parking function. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, this application aims to provide a calibration and correction method, device, equipment and storage medium for vehicle cameras to solve the problem of high errors in panoramic images and mosaics caused by large calibration errors of after-sales cameras in the prior art.

[0005] An embodiment of this application provides a calibration and correction method for vehicle cameras, and the method includes:

[0006] Obtain the current position and the current yaw angle of each camera obtained from the current calibration in the calibration cloth coordinate system;

[0007] For each camera, based on the standard calibration position of the camera in the vehicle body coordinate system and the current position, correct the current yaw angle to obtain the corrected yaw angle of the camera in the vehicle body coordinate system;

[0008] Determine the predicted position of the camera in the calibration cloth coordinate system after the yaw angle is corrected, and correct the predicted position of the camera based on the standard calibration position of the camera to obtain the corrected position of the camera in the vehicle body coordinate system.

[0009] Optionally, for each camera, based on the standard calibration position of the camera in the vehicle body coordinate system and the current position, correcting the current yaw angle includes:

[0010] Based on the standard calibration positions of each camera in the vehicle body coordinate system and the current positions of each camera, determine the inclination angle of the vehicle relative to the calibration cloth during this calibration process;

[0011] For each camera, rotate the camera based on the inclination angle to correct the current yaw angle of the camera.

[0012] Optionally, determining the inclination angle of the vehicle relative to the calibration cloth during this calibration process based on the standard calibration positions of each camera in the vehicle body coordinate system and the current positions of each camera includes:

[0013] Based on the standard calibration positions of the front camera and the rear camera in the vehicle body coordinate system, determine the standard vehicle body vector;

[0014] Based on the current positions of the front camera and the rear camera in the calibration cloth coordinate system, determine the current vehicle body vector of the vehicle;

[0015] According to the standard vehicle body vector and the current vehicle body vector, determine the inclination angle of the vehicle relative to the calibration cloth during this calibration process.

[0016] Optionally, determining the inclination angle of the vehicle relative to the calibration cloth during this calibration process according to the standard vehicle body vector and the current vehicle body vector includes:

[0017] Determine the cross product between the standard vehicle body vector and the current vehicle body vector, and determine the vector length of the standard vehicle body vector and the vector length of the current vehicle body vector;

[0018] Based on the cross product, the vector length of the standard vehicle body vector, and the vector length of the current vehicle body vector, determine the inclination angle of the vehicle relative to the calibration cloth during this calibration process.

[0019] Optionally, determining the predicted position of the camera in the calibration cloth coordinate system after yaw angle correction includes:

[0020] Based on the inclination angle and the current position of the camera in the calibration cloth coordinate system, determine the predicted position of the camera in the calibration cloth coordinate system.

[0021] Optionally, correcting the predicted position of the camera based on the standard calibration position of the camera includes:

[0022] Based on the standard calibration position and the predicted position of the camera, determine the translation distance to be applied to the camera;

[0023] Starting from the predicted position, translate the camera based on the translation distance to be applied to the camera to correct the predicted position of the camera.

[0024] Optionally, obtain the current positions and current yaw angles of each camera obtained in this calibration in the calibration cloth coordinate system, including:

[0025] Obtain the result of this calibration, where the result of this calibration includes the offsets and postures of each camera relative to the center of the vehicle's rear axle;

[0026] Determine the offsets of each camera relative to the center of the vehicle's rear axle as the current positions of each camera in the calibration cloth coordinate system;

[0027] Determine the postures of each camera relative to the center of the vehicle's rear axle as the current yaw angles of each camera in the calibration cloth coordinate system.

[0028] The embodiment of the present application further provides a calibration and correction device for vehicle cameras, and the device includes:

[0029] A calibration result acquisition module, configured to obtain the current positions and current yaw angles of each camera obtained in this calibration in the calibration cloth coordinate system;

[0030] An attitude adjustment module, configured to, for each camera, correct the current yaw angle based on the standard calibration position of the camera in the vehicle body coordinate system and the current position, to obtain the corrected yaw angle of the camera in the vehicle body coordinate system;

[0031] An offset adjustment module, configured to determine the predicted position of the camera in the calibration cloth coordinate system after the yaw angle is corrected, and correct the predicted position of the camera based on the standard calibration position of the camera, to obtain the corrected position of the camera in the vehicle body coordinate system.

[0032] The embodiment of the present application further provides an electronic device, and the electronic device includes:

[0033] A processor and a memory;

[0034] The processor is configured to execute the steps of the calibration and correction method for vehicle cameras provided in any embodiment of the present application by calling the program or instruction stored in the memory.

[0035] The embodiment of the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the calibration and correction method for vehicle cameras provided in any embodiment of the present application.

[0036] In summary, the present application proposes a calibration and correction method for vehicle cameras. This method obtains the current positions and current yaw angles of each camera obtained in the current calibration in the calibration cloth coordinate system. Then, for each camera, based on the standard calibration position and the current position of the camera in the vehicle body coordinate system, the current yaw angle is corrected to obtain the corrected yaw angle of the camera in the vehicle body coordinate system, and the predicted position of the camera in the calibration cloth coordinate system after the yaw angle correction is determined. Thus, the predicted position of the camera is corrected according to the standard calibration position of the camera to obtain the corrected position of the camera in the vehicle body coordinate system, realizing the correction of the yaw angle and position in the calibration results of each camera, improving the camera calibration accuracy, solving the problem of abnormal panoramic image stitching effect, avoiding ghosting at the stitching position of the panoramic image display image, being able to more truly express the installation position and attitude of the camera on the vehicle, improving the stitching quality, making the panoramic image display a more real scene, and thus showing higher accuracy during the use of automatic parking. Moreover, this method, through the obtained position and heading angle in the calibration and combining with the standard calibration position of the camera, first corrects the yaw angle and then corrects the position, can take into account the influence of the yaw angle adjustment on the camera position, and further corrects the camera position on this basis, further improving the camera calibration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 is a flowchart of a calibration and correction method for vehicle cameras provided by an embodiment of the present application;

[0039] Figure 2 is a schematic diagram of a calibration result provided by an embodiment of the present application;

[0040] Figure 3 is an overall framework diagram provided by an embodiment of the present application;

[0041] Figure 4 is a correction flowchart provided by an embodiment of the present application;

[0042] Figure 5 is a schematic structural diagram of a calibration and correction device for vehicle cameras provided by an embodiment of the present application;

[0043] Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, rather than limiting the invention. In addition, it should be noted that for the convenience of description, only the parts related to the invention are shown in the drawings.

[0045] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and embodiments.

[0046] As mentioned in the background art, in view of the problems in the prior art, the present application proposes a calibration and correction method for a vehicle camera. Figure 1 is a flowchart of a calibration and correction method for a vehicle camera provided by an embodiment of the present application. Refer to Figure 1 , the calibration and correction method for the vehicle camera specifically includes:

[0047] S110. Obtain the current position and current yaw angle of each camera obtained in this calibration in the calibration cloth coordinate system.

[0048] Among them, each camera in the vehicle can be a front camera, a rear camera, a left camera, and a right camera respectively. This calibration can be a camera calibration performed using an after-sales calibration process.

[0049] Exemplarily, a calibration environment can be prepared first. Find a flat ground, and the ground is in an environment with uniform illumination. The size of the ground is 7m×10m, and the ground is convenient for the vehicle to enter and exit. Furthermore, the calibration cloth can be paved in this calibration environment, and the vehicle can be parked on the calibration cloth, and the vehicle can be parked according to the parking requirements (such as the rear axle of the vehicle needs to be aligned with the edge of the calibration cloth, etc.). After the vehicle is parked on the calibration cloth, the after-sales calibration process can be started. For example, the calibration operation can be performed by executing a calibration script, and then the result of this calibration can be obtained.

[0050] Specifically, the result of this calibration can be obtained, so as to obtain the current position and current yaw angle of each camera in the calibration cloth coordinate system. It should be noted that the purpose of camera calibration is to obtain the position and attitude of the camera relative to the center of the rear axle of the vehicle. However, due to reasons such as uneven ground in the actual calibration environment and the vehicle not being aligned with the calibration cloth during parking, it is difficult to align the vehicle body coordinate system with the calibration cloth coordinate system, and there is an offset between the two coordinate systems. Therefore, considering the deviation between the vehicle body coordinate system and the calibration cloth coordinate system, the result of this calibration can be regarded as data in the calibration cloth coordinate system. By correcting it, data in the true vehicle body coordinate system can be obtained, so as to obtain the true position and true attitude of the camera relative to the center of the rear axle of the vehicle.

[0051] In a specific embodiment, obtaining the current positions and current yaw angles of each camera obtained in this calibration in the calibration cloth coordinate system includes:

[0052] Obtaining the result of this calibration, where the result of this calibration includes the offsets and postures of each camera relative to the center of the vehicle's rear axle;

[0053] Determine the offset of each camera relative to the center of the vehicle's rear axle as the current position of each camera in the calibration cloth coordinate system; determine the posture of each camera relative to the center of the vehicle's rear axle as the current yaw angle of each camera in the calibration cloth coordinate system.

[0054] Specifically, the result of this calibration may include the offsets and postures of each camera relative to the center of the vehicle's rear axle. Among them, the offset can describe the distance of each camera relative to the center of the vehicle's rear axle in each direction, and the posture can describe the yaw angle of each camera relative to the center of the vehicle's rear axle.

[0055] Furthermore, considering the deviation between the vehicle body coordinate system and the calibration cloth coordinate system during this calibration process, the obtained calibration result is actually data in the calibration cloth coordinate system and cannot reflect the situation in the real vehicle body coordinate system. Therefore, the offset of each camera relative to the center of the vehicle's rear axle can be used as the current position of each camera in the calibration cloth coordinate system, and the posture of each camera relative to the center of the vehicle's rear axle can be used as the current yaw angle of each camera in the calibration cloth coordinate system.

[0056] Through the above embodiments, the current positions and current yaw angles of each camera in the calibration cloth coordinate system can be obtained from the result of this calibration.

[0057] Exemplarily, Figure 2 is a schematic diagram of a calibration result provided by an embodiment of the present application. As Figure 2 shown in (a) therein, in a standard calibration environment, the center of the vehicle's rear axle is aligned with the origin O of the calibration cloth coordinate system, and the rear axle of the vehicle is parallel to the side of the calibration cloth, so that the vehicle body coordinate system is aligned with the calibration cloth coordinate system (xOy). As Figure 2 shown in (b), in the actual calibration environment, there is a deviation between the center O1 of the vehicle's rear axle and the origin O of the calibration cloth coordinate system, resulting in the misalignment of the vehicle body coordinate system (xO1y) and the calibration cloth coordinate system (xOy).

[0058] S120. For each camera, based on the standard calibration position of the camera in the vehicle body coordinate system and the current position, correct the current yaw angle to obtain the corrected yaw angle of the camera in the vehicle body coordinate system.

[0059] Among them, the standard calibration position of the camera in the vehicle body coordinate system can be the position information obtained through the standard calibration process in the factory in advance, that is, the standard calibration position is obtained by collecting when the vehicle is accurately aligned with the calibration cloth.

[0060] As Figure 2 shown, when the vehicle is accurately aligned with the calibration cloth, the vehicle body coordinate system is aligned with the calibration cloth coordinate system. Among them, Figure (a) respectively shows the standard calibration position P0 of the front camera in the vehicle body coordinate system, the standard calibration position P1 of the left camera in the vehicle body coordinate system, the standard calibration position P2 of the rear camera in the vehicle body coordinate system, and the standard calibration position P3 of the rear side camera in the vehicle body coordinate system. For this calibration, as Figure 2 shown in Figure (b), it respectively shows the current position T0 of the front camera in the calibration cloth coordinate system, the current position T1 of the left camera in the calibration cloth coordinate system, the current position T2 of the rear camera in the calibration cloth coordinate system, and the current position T3 of the rear side camera in the calibration cloth coordinate system.

[0061] Specifically, for each camera, the position deviation between the vehicle and the calibration cloth can be calculated through the standard calibration position (in the vehicle body coordinate system) and the current position (in the calibration cloth coordinate system) of the camera, so as to correct this calibration through this position deviation to align the vehicle body coordinate system with the calibration cloth coordinate system.

[0062] In the embodiment of the present application, the correction can be divided into two steps. The first step is to first adjust the current yaw angle of the camera, and the second step is to adjust the current position of the camera.

[0063] In a specific implementation manner, for each camera, based on the standard calibration position of the camera in the vehicle body coordinate system and the current position, the current yaw angle is corrected, including the following steps:

[0064] Step 11: Based on the standard calibration positions of each camera in the vehicle body coordinate system and the current positions of each camera, determine the tilt angle of the vehicle relative to the calibration cloth during this calibration process;

[0065] Step 12: For each camera, rotate the camera based on the tilt angle to correct the current yaw angle of the camera.

[0066] Among them, in Step 11, the standard orientation of the vehicle in the calibration cloth can be measured according to the standard calibration positions of each camera in the vehicle body coordinate system, and then the actual orientation of the vehicle in the calibration cloth during this calibration can be measured according to the current positions of each camera. Based on the angle difference between the standard orientation and the actual orientation, the tilt angle of the vehicle relative to the calibration cloth during this calibration process is determined.

[0067] Optionally, for step 11 above, based on the standard calibration positions of each camera in the vehicle body coordinate system and the current positions of each camera, determine the tilt angle of the vehicle relative to the calibration cloth during this calibration process, including the following steps:

[0068] Step 111: Based on the standard calibration positions of the front camera and the rear camera in the vehicle body coordinate system, determine the standard vehicle body vector;

[0069] Step 112: Based on the current positions of the front camera and the rear camera in the calibration cloth coordinate system, determine the current vehicle body vector;

[0070] Step 113: According to the standard vehicle body vector and the current vehicle body vector, determine the tilt angle of the vehicle relative to the calibration cloth during this calibration process.

[0071] Among them, in step 111, the standard vehicle body vector can be calculated according to the standard calibration position of the rear camera in the vehicle body coordinate system and the standard calibration position of the front camera in the vehicle body coordinate system. This standard vehicle body vector can reflect the standard orientation of the vehicle in the calibration cloth. As Figure 2 shown, the standard vehicle body vector can be

[0072] Furthermore, in step 112, the current vehicle body vector can be calculated according to the current position of the rear camera in the calibration cloth coordinate system and the current position of the front camera in the calibration cloth coordinate system. This current vehicle body vector can reflect the actual orientation of the vehicle being calibrated in the calibration cloth. As Figure 2 shown, the current vehicle body vector can be

[0073] Furthermore, in step 113, the tilt angle of the vehicle relative to the calibration cloth can be calculated according to the standard vehicle body vector and the current vehicle body vector.

[0074] For step 113 above, in one example, according to the standard vehicle body vector and the current vehicle body vector, determine the tilt angle of the vehicle relative to the calibration cloth during this calibration process, including the following steps:

[0075] Step 1131: Determine the cross product between the standard vehicle body vector and the current vehicle body vector, and determine the vector length of the standard vehicle body vector and the vector length of the current vehicle body vector;

[0076] Step 1132: Based on the cross product, the vector length of the standard vehicle body vector, and the vector length of the current vehicle body vector, determine the tilt angle of the vehicle relative to the calibration cloth during this calibration process.

[0077] Among them, in step 1131, the cross product between the standard vehicle body vector and the current vehicle body vector can be calculated, that is And calculate the vector length of the standard body vector, that is Calculate the vector length of the current body vector, that is

[0078] Considering that the vehicle is an objective object, the position of the camera on the vehicle is fixed, and the vehicle as a whole has the characteristics of a rigid body. Therefore, the inclination direction of the vehicle relative to the calibration cloth can also be determined by the sign of the vector product between the standard body vector and the current body vector, that is, through The sign of to obtain the inclination direction. If the vector product is greater than 0, it means that the vehicle is biased towards the left front relative to the calibration cloth. If the vector product is less than 0, it means that the vehicle is biased towards the right front relative to the calibration cloth.

[0079] Furthermore, in step 1132, the inclination angle of the vehicle can be calculated according to the vector product, the vector length of the standard body vector, and the vector length of the current body vector, as shown in the following formula:

[0080]

[0081] In the formula, yaw is the inclination angle of the vehicle relative to the calibration cloth.

[0082] Through the above steps, the inclination angle of the vehicle relative to the calibration cloth in the current calibration process can be calculated based on the vector product, vector length, etc. between the standard body vector and the current body vector, ensuring the accuracy of the inclination angle.

[0083] After obtaining the inclination angle of the vehicle relative to the calibration cloth in the current calibration process, further, for each camera, the inclination angle can be used to rotate the camera to correct the current yaw angle of the camera. Exemplarily, after obtaining the inclination angle of the vehicle, in order to achieve correction, rotation needs to be performed in the reverse direction. Therefore, a negative sign can be added to the inclination angle, and then it is accumulated with the current yaw angle to obtain the corrected yaw angle of the camera.

[0084] For example, add a negative sign to the inclination angle, And then correct the yaw angle Y i ' = Y i + yaw.

[0085] S130. Determine the predicted position of the camera in the calibration cloth coordinate system after yaw angle correction, and correct the predicted position of the camera based on the standard calibration position of the camera to obtain the corrected position of the camera in the vehicle body coordinate system.

[0086] In the embodiment of the present application, considering that the camera rotates to a certain extent during the yaw angle correction process, and the rotation will change the position of the camera in the calibration cloth coordinate system. Therefore, after the yaw angle correction, the latest position of the camera after rotation from the current position in the calibration cloth coordinate system can be re-determined, and this position is used as the predicted position.

[0087] In a specific implementation, determining the predicted position of the camera in the calibration cloth coordinate system after yaw angle correction includes:

[0088] Based on the tilt angle and the current position of the camera in the calibration cloth coordinate system, determine the predicted position of the camera in the calibration cloth coordinate system.

[0089] Exemplarily, based on the tilt angle and the current position of the camera in the calibration cloth coordinate system, calculating the predicted position of the camera in the calibration cloth coordinate system can satisfy the following formula:

[0090]

[0091] In the formula, M i .x and M i .y are respectively the predicted position on the x-axis and the predicted position on the y-axis of the i-th camera in the calibration cloth coordinate system, that is, (M i .x, M i .y) is the predicted position of the i-th camera in the calibration cloth coordinate system, T i .x and T i .y are respectively the current position on the x-axis and the current position on the y-axis of the i-th camera in the calibration cloth coordinate system, that is, (T i .x, T i .y) is the current position of the i-th camera in the calibration cloth coordinate system;

[0092] After calculating the predicted position of the camera in the calibration cloth coordinate system, further, the predicted position of the rotated camera can be corrected according to the standard calibration position of the camera.

[0093] In a specific implementation, correcting the predicted position of the camera based on the standard calibration position of the camera includes the following steps:

[0094] Step 21: Based on the standard calibration position and the predicted position of the camera, determine the distance to be translated of the camera;

[0095] Step 22: Starting from the predicted position, translate the camera based on the distance to be translated of the camera to correct the predicted position of the camera.

[0096] Among them, in step 21, the distance to be translated of the camera can be calculated through the standard calibration position and the predicted position of the camera, as shown in the following formula:

[0097]

[0098] In the formula, dx and dy are respectively the distances to be translated in the x-axis direction and the y-axis direction, P i .x, Pi .y are the standard calibration positions on the x-axis and y-axis of the i-th camera in the vehicle body coordinate system, that is, (P i .x, P i .y) is the standard calibration position of the i-th camera in the vehicle body coordinate system.

[0099] After calculating the distance to be translated, further, the camera can be translated starting from the predicted position with the distance to be translated as the translation amount, and the translated position is used as the corrected position of the camera in the vehicle body coordinate system.

[0100] Through the above implementation manner, on the basis of correcting the attitude, the influence of attitude adjustment on the position can be further considered, and the offset can be corrected in combination with this influence to ensure the accuracy of the correction.

[0101] The corrected yaw angles and corrected positions of each camera in the vehicle body coordinate system obtained through the above method are the actual poses and offsets of each camera in the vehicle relative to the center of the rear axle of the vehicle. The corrected yaw angles and corrected positions can be used to update the calibration results and stored on the disk as the corrected calibration results.

[0102] Figure 3 is an overall framework diagram provided by an embodiment of the present application. As Figure 3 shown, in the input layer, through the static calibration in the factory calibration room, the positions and yaw angles of each camera relative to the center of the rear axle of the vehicle can be obtained. The input data of the input layer can also include the internal parameters of the four cameras and the data streams of the four cameras.

[0103] The calibration layer can include feature detection and attitude calculation. Among them, feature detection can include steps such as image enhancement, image denoising, and corner detection to obtain distorted corner information and undistorted corner information. Attitude calculation can include steps such as calculating physical space coordinates, calculating pixel coordinates, and calculating reprojection errors to obtain the camera offset and attitude, that is, the current position and current yaw angle of the camera in the calibration cloth coordinate system.

[0104] The correction layer can include attitude estimation and attitude correction. Attitude estimation includes estimating the yaw angle of the vehicle body during calibration, estimating the offset of the vehicle body during calibration, and estimating the attitude of the vehicle body on the calibration cloth through the position of the camera on the vehicle body; attitude correction can include adjusting the attitude of the vehicle body, adjusting the offset of the vehicle body, and correcting the calibration results through the conversion of Euler angles and rotation matrices.

[0105] Figure 4 is a correction flow chart provided by an embodiment of the present application. As Figure 4As shown, first, four sets of extrinsic parameters of the cameras can be output through this calibration, that is, the current positions and current yaw angles of the four cameras in the calibration cloth coordinate system. Then, the front and rear cameras and the left and right cameras are each divided into a group, and the yaw angles of the two groups are calculated. The tilt angle is calculated based on the rigid body information of the known positions of the cameras on the vehicle. Based on the rotation characteristics of the rigid body, the vehicle body coordinate system is rotated towards the calibration cloth coordinate system. After rotation, the translation distance of the camera in the calibration cloth coordinate system is calculated for translation processing, so as to obtain the corrected calibration yaw angle and corrected position (in the vehicle body coordinate system). Based on this, the extrinsic parameters of the four cameras are modified, and the modified information is stored in the disk. Thus, the calibration correction process is completed.

[0106] The above correction process is mainly to correct the results of this calibration through the prior information of static calibration after one calibration is completed. The correction process includes attitude correction (i.e., correction of the yaw angle) and offset correction (i.e., correction of the position). The calibration cloth coordinate system takes the origin of the calibration cloth as the origin of the coordinate system. The attitude and offset of the camera in the calibration cloth coordinate system are calculated through the physical coordinates and pixel coordinates of the corner points. Using the invariant characteristics of the rigid body, the attitude and offset of the vehicle relative to the calibration cloth are calculated, and then the attitude information in the extrinsic parameters of the camera is corrected using the rigid body rotation transformation. Furthermore, through translation, the vehicle body coordinate system of this calibration is aligned with the calibration cloth coordinate system to obtain the corrected calibration result.

[0107] The calibration correction method for vehicle cameras provided in the embodiments of the present application obtains the current positions and current yaw angles of each camera obtained through this calibration in the calibration cloth coordinate system. Then, for each camera, based on the standard calibration position and the current position of the camera in the vehicle body coordinate system, the current yaw angle is corrected to obtain the corrected yaw angle of the camera in the vehicle body coordinate system, and the predicted position of the camera in the calibration cloth coordinate system after the yaw angle correction is determined. Thus, the predicted position of the camera is corrected according to the standard calibration position of the camera to obtain the corrected position of the camera in the vehicle body coordinate system, realizing the correction of the yaw angle and position in the calibration results of each camera, improving the camera calibration accuracy, solving the problem of abnormal panoramic image stitching effect, more truly expressing the installation position and attitude of the camera on the vehicle, improving the stitching quality, making the panoramic image display a more real scene, and thus showing higher accuracy during the use of automatic parking. Moreover, this method combines the position and heading angle obtained through calibration with the standard calibration position of the camera, first corrects the yaw angle and then corrects the position, which can take into account the influence of the adjustment of the yaw angle on the camera position, and further corrects the camera position on this basis, further improving the camera calibration accuracy.

[0108] Figure 5It is a schematic structural diagram of a calibration and correction device for a vehicle camera provided by an embodiment of the present application. The device includes a calibration result acquisition module 510, an attitude adjustment module 520, and an offset adjustment module 530, where:

[0109] The calibration result acquisition module 510 is configured to acquire the current position and the current yaw angle of each camera obtained in the current calibration in the calibration cloth coordinate system.

[0110] The attitude adjustment module 520 is configured to, for each camera, correct the current yaw angle based on the standard calibration position of the camera in the vehicle body coordinate system and the current position, to obtain the corrected yaw angle of the camera in the vehicle body coordinate system.

[0111] The offset adjustment module 530 is configured to determine the predicted position of the camera in the calibration cloth coordinate system after the yaw angle is corrected, and correct the predicted position of the camera based on the standard calibration position of the camera, to obtain the corrected position of the camera in the vehicle body coordinate system..

[0112] Based on the above embodiments, optionally, the attitude adjustment module 520 includes an inclination angle determination unit and a rotation unit, where:

[0113] The inclination angle determination unit is configured to determine the inclination angle of the vehicle relative to the calibration cloth during the current calibration based on the standard calibration positions of each camera in the vehicle body coordinate system and the current positions of each camera.

[0114] The rotation unit is configured to, for each camera, rotate the camera based on the inclination angle to correct the current yaw angle of the camera.

[0115] Based on the above embodiments, optionally, the inclination angle determination unit is specifically configured to:

[0116] Determine the standard vehicle body vector based on the standard calibration positions of the front camera and the rear camera in the vehicle body coordinate system; determine the current vehicle body vector based on the current positions of the front camera and the rear camera in the calibration cloth coordinate system; determine the inclination angle of the vehicle relative to the calibration cloth during the current calibration according to the standard vehicle body vector and the current vehicle body vector.

[0117] Based on the above embodiments, optionally, the inclination angle determination unit is further configured to determine the cross product between the standard vehicle body vector and the current vehicle body vector, and determine the vector length of the standard vehicle body vector and the vector length of the current vehicle body vector; determine the inclination angle of the vehicle relative to the calibration cloth during the current calibration based on the cross product, the vector length of the standard vehicle body vector, and the vector length of the current vehicle body vector.

[0118] Based on the above embodiments, optionally, the offset adjustment module 530 is further configured to determine a predicted position of the camera in the calibration cloth coordinate system based on the tilt angle and the current position of the camera in the calibration cloth coordinate system.

[0119] Based on the above embodiments, optionally, the offset adjustment module 530 is further configured to determine a distance to be translated for the camera based on the standard calibration position and the predicted position of the camera; starting from the predicted position, translate the camera based on the distance to be translated for the camera to correct the predicted position of the camera.

[0120] Based on the above embodiments, optionally, the calibration result acquisition module 510 is specifically configured to:

[0121] Obtain the current calibration result, where the current calibration result includes the offset and attitude of each camera relative to the center of the vehicle's rear axle; determine the offset of each camera relative to the center of the vehicle's rear axle as the current position of each camera in the calibration cloth coordinate system; and determine the attitude of each camera relative to the center of the vehicle's rear axle as the current yaw angle of each camera in the calibration cloth coordinate system.

[0122] The vehicle camera calibration and correction device provided by the embodiments of the present application can execute the steps in the vehicle camera calibration and correction method provided by the method embodiments of the present application, and the implementation steps and beneficial effects are not described herein again.

[0123] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 6 shown, the electronic device 400 includes one or more processors 401 and a memory 402.

[0124] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0125] The memory 402 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 401 may run the program instructions to implement the calibration and correction method of the vehicle camera according to any embodiment of the present application described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage media.

[0126] In one example, the electronic device 400 may further include: an input device 403 and an output device 404, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). The input device 403 may include, for example, a keyboard, a mouse, etc. The output device 404 may output various information to the outside, including warning prompt information, braking force, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0127] Of course, for simplicity, Figure 6 only some of the components related to the present application in the electronic device 400 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 400 may further include any other appropriate components.

[0128] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps of the calibration and correction method of the vehicle camera provided by any embodiment of the present application.

[0129] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the embodiments of the present application. The programming languages include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0130] In addition, an embodiment of the present application may also be a computer-readable storage medium storing computer program instructions, which, when run by a processor, cause the processor to execute the steps of the calibration and correction method of the vehicle camera provided in any embodiment of the present application.

[0131] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0132] It should be noted that the terms used in the present application are only for describing specific embodiments and do not limit the scope of the present application. As shown in the specification and claims of the present application, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. The term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, or device including the element.

[0133] It should also be noted that the orientation or positional relationship indicated by terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application. Unless otherwise clearly specified and limited, terms such as "installed", "connected", "connected to" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific situations.

[0134] In this article, specific examples are used to elaborate on the principles and implementation modes of the present application. The description of the above embodiments is only for helping to understand the method and its core idea of the present application. The above is only the preferred implementation mode of the present application. It should be noted that due to the limitation of literal expression and objectively there are infinite specific structures, for those of ordinary skill in the art, without departing from the principle of the present application, several improvements, retouches or changes can be made, or the above technical features can be combined in an appropriate way; these improvements, retouches, changes or combinations, or directly applying the concept and technical solution of the invention to other occasions without improvement, shall all be regarded as the protection scope of the present application.

Claims

1. A calibration and correction method for a vehicle camera, characterized in that: include: Get the current position and current yaw angle of each camera obtained in this calibration in the calibration coordinate system; For each camera, based on the standard calibration position of the camera in the vehicle body coordinate system and the current position, the current yaw angle is corrected to obtain the corrected yaw angle of the camera in the vehicle body coordinate system; The predicted position of the camera in the calibration coordinate system after the yaw angle correction is determined, and the predicted position of the camera is corrected based on the standard calibration position of the camera to obtain the corrected position of the camera in the vehicle body coordinate system.

2. The method according to claim 1, characterized in that For each camera, based on the standard calibration position of the camera in the vehicle coordinate system and the current position, the current yaw angle is corrected, including: Based on the standard calibration position of each camera in the vehicle coordinate system and the current position of each camera, determine the inclination angle of the vehicle relative to the calibration cloth during this calibration process; For each camera, the camera is rotated based on the tilt angle to correct a current yaw angle of the camera.

3. The method according to claim 2, characterized in that Based on the standard calibration position of each camera in the vehicle coordinate system and the current position of each camera, the inclination angle of the vehicle relative to the calibration cloth during this calibration process is determined, including: Determine the standard body vector of the vehicle based on the standard calibration positions of the front camera and the rear camera in the vehicle body coordinate system; Determining a current body vector of the vehicle based on current positions of the front camera and the rear camera in the calibration coordinate system; According to the standard vehicle body vector and the current vehicle body vector, the inclination angle of the vehicle relative to the calibration cloth during this calibration process is determined.

4. The method according to claim 3, characterized in that Determining the inclination angle of the vehicle relative to the calibration cloth during this calibration process according to the standard vehicle body vector and the current vehicle body vector, including: Determine a vector product between the standard vehicle body vector and the current vehicle body vector, and determine a vector length of the standard vehicle body vector and a vector length of the current vehicle body vector; Based on the vector product, the vector length of the standard vehicle body vector, and the vector length of the current vehicle body vector, the inclination angle of the vehicle relative to the calibration cloth during this calibration process is determined.

5. The method according to claim 2, characterized in that: Determining the predicted position of the camera in the calibration coordinate system after yaw angle correction includes: Based on the tilt angle and the current position of the camera in the calibration cloth coordinate system, a predicted position of the camera in the calibration cloth coordinate system is determined.

6. The method according to claim 1, characterized in that Correcting the predicted position of the camera based on a standard calibrated position of the camera includes: Determining a distance to be translated of the camera based on a standard calibrated position and a predicted position of the camera; Starting from the predicted position, the camera is translated based on the distance to be translated of the camera to correct the predicted position of the camera.

7. The method according to claim 1, characterized in that Get the current position and yaw angle of each camera in the calibration coordinate system, including: Obtaining the calibration result, wherein the calibration result includes the offset and posture of each camera relative to the center of the rear axle of the vehicle; The offset of each camera relative to the center of the rear axle of the vehicle is determined as the current position of each camera in the calibration coordinate system; The posture of each camera relative to the center of the rear axle of the vehicle is determined as the current yaw angle of each camera in the calibration coordinate system.

8. A calibration and correction device for a vehicle camera, characterized in that: include: The calibration result acquisition module is used to obtain the current position and current yaw angle of each camera obtained in this calibration in the calibration coordinate system; A posture adjustment module is used to correct the current yaw angle of each camera based on the standard calibration position of the camera in the vehicle body coordinate system and the current position to obtain the corrected yaw angle of the camera in the vehicle body coordinate system; The offset adjustment module is used to determine the predicted position of the camera in the calibration coordinate system after the yaw angle correction, and correct the predicted position of the camera based on the standard calibration position of the camera to obtain the corrected position of the camera in the vehicle body coordinate system.

9. An electronic device, characterized in that: The electronic device comprises: Processor and memory; The processor is used to execute the steps of the vehicle camera calibration and correction method as described in any one of claims 1 to 7 by calling the program or instruction stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, which enables a computer to execute the steps of the calibration and correction method for a vehicle camera as claimed in any one of claims 1 to 7.

Images