Camera calibration method, system and vehicle

Abstract

The application discloses a camera calibration method and system and a vehicle, wherein the method comprises: acquiring a first image collected by a camera when an outside rearview mirror is in an unfolded state, selecting feature points from the first image, and estimating theoretical motion tracks of the feature points in a preset calibration time period in a folding process of the outside rearview mirror; in the folding process of the outside rearview mirror, controlling the camera to collect a second image in the preset calibration time period in the folding process, identifying the feature points in the second image, and determining actual motion tracks of the feature points in the calibration time period; and based on the theoretical motion tracks and the actual motion tracks of the feature points, calibrating a position and posture of the camera in a world coordinate system. The camera calibration method provided by the application can complete dynamic calibration of the camera in an environment without relying on manual arrangement, thereby saving calibration sites and calibration time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a camera calibration method, system and vehicle. BACKGROUND

[0002] The streaming rearview mirror is a new automobile technology, which uses a camera installed on the automobile rearview mirror to collect images, and then displays the images on the screen in the vehicle to replace the traditional rearview mirror. Through this technology, the field of view of the rearview mirror can be expanded, the wind resistance can be reduced, and the visibility in bad environments such as rain, snow and night can be enhanced.

[0003] At present, the control of the field of view range of the streaming rearview mirror mainly depends on the installation position of the camera and the pre-checking parameters of the field of view. However, during the installation of the electronic rearview mirror, due to the accumulation of size chain tolerances and assembly errors, the streaming camera on the electronic rearview mirror often deviates from the theoretical position, resulting in that the field of view cannot meet the regulatory requirements. In the prior art, the calibration method of the streaming camera on the rearview mirror often uses static calibration. After the vehicle is parked, a calibration cloth is laid manually, the vehicle is connected with a diagnostic instrument, the diagnostic instrument runs a calibration program, the feature points of the checkboard on the calibration cloth are extracted, and the calibration is calculated. However, the manual laying of the calibration cloth has a large error, which affects the calibration result, and a large site is occupied, and the working hours for calibration are long.

[0004] Therefore, it is necessary to provide an improved technical solution to overcome the above technical problems in the prior art. SUMMARY

[0005] In view of the above problems, the purpose of the present application is to provide a camera calibration method, system and vehicle, which can complete the dynamic calibration of the camera in an environment without relying on manual arrangement, and save the calibration site and time.

[0006] The present application provides a camera calibration method for calibrating the camera of the rearview mirror of a vehicle, which comprises:

[0007] When the rearview mirror is in an unfolded state, a first image collected by the camera is acquired, feature points are selected from the first image, and a theoretical motion trajectory of the feature points in a preset calibration time period in the folding process of the rearview mirror is estimated;

[0008] During the folding process of the rearview mirror, the camera is controlled to collect a second image in the preset calibration time period in the folding process, the feature points are identified in the second image, and the actual motion trajectory of the feature points in the calibration time period is determined;

[0009] According to the theoretical motion track and the actual motion track of the feature point, a position and posture of the camera in a world coordinate system is calibrated.

[0010] Further, the step of estimating the theoretical motion track of the feature point in a preset calibration time period during the folding process of the outside rearview mirror comprises:

[0011] During the folding process of the outside rearview mirror, a folding angle of the outside rearview mirror and a folding motion time corresponding to each folding angle are obtained.

[0012] According to the folding angle and the folding motion time corresponding to each folding angle, the theoretical motion track of the feature point in the preset calibration time period during the folding process of the outside rearview mirror is estimated.

[0013] Further, the step of calibrating the position and posture of the camera in the world coordinate system based on the theoretical motion track and the actual motion track of the feature point comprises:

[0014] According to the theoretical motion track of the feature point, a position change relationship of the feature point in the world coordinate system is obtained.

[0015] According to the actual motion track of the feature point, a position change relationship of the feature point in the camera coordinate system is obtained.

[0016] According to the position change relationship of the feature point in the world coordinate system and the camera coordinate system, a calibration equation is constructed.

[0017] The position and posture of the camera in the world coordinate system are calibrated through the calibration equation.

[0018] Further, the step of constructing the calibration equation according to the position change relationship of the feature point in the world coordinate system and the camera coordinate system comprises:

[0019] According to the world coordinate and the camera coordinate at the mth position, a calibration equation is constructed: A m X = XB m .

[0020] According to the world coordinate and the camera coordinate at the nth position, a calibration equation is constructed: A n X = XB n .

[0021] Wherein, A m and A n are the world coordinates in the world coordinate system at the mth position and the nth position respectively, B m and B n are the camera coordinates in the camera coordinate system at the mth position and the nth position respectively.

[0022] obtaining a position change equation of the feature point from the mth position to the nth position according to the calibration equation at the mth position and the calibration equation at the nth position: AX=XB, wherein,

[0023]

[0024] R A and T A are a rotation matrix and a translation matrix of the feature point from the mth position to the nth position under the world coordinate system, R B and T B are a rotation matrix and a translation matrix of the feature point from the mth position to the nth position under the camera coordinate system;

[0025] obtaining a conversion relationship X from the world coordinate system to the camera coordinate system by solving the calibration equation, the X including a rotation matrix R X indicating the camera coordinate system relative to the world coordinate system, and a translation matrix T X indicating the camera coordinate system origin relative to the world coordinate system origin.

[0026] Further, the method further includes determining position coordinates of points in the environment under the camera coordinate system according to the rotation matrix R X and the translation matrix T X obtained by solving, and the conversion relationship is:

[0027]

[0028] wherein (x c , y c , z c , 1) T is a homogeneous coordinate under the camera coordinate system, and (x w , y w , z w , 1) T is a homogeneous coordinate under the world coordinate system.

[0029] Further, the selecting the feature point from the first image includes: selecting a point on the vehicle body in the first image as the feature point, or selecting a point that is stationary relative to the vehicle body in the first image as the feature point.

[0030] The application also provides a camera calibration system, including a camera, a processing module and a display module.

[0031] The camera is connected with the processing module, and is used for collecting a first image when an outside rearview mirror is in an unfolded state, and collecting a second image in a preset calibration time period when the outside rearview mirror is in a folding process.

[0032] The display module is connected with the processing module, and is used for displaying the first image and the second image collected by the camera and selecting a feature point used for camera calibration;

[0033] The processing module is used for estimating a theoretical motion trajectory of the feature point in a preset calibration time period in the folding process of the outside rearview mirror, controlling the camera to collect a second image in the preset calibration time period in the folding process of the outside rearview mirror, identifying the feature point in the second image, and determining an actual motion trajectory of the feature point in the calibration time period, and calibrating a position and posture of the camera in a world coordinate system based on the theoretical motion trajectory and the actual motion trajectory of the feature point.

[0034] Further, the processing module is further used for obtaining a folding angle of the outside rearview mirror and a folding motion time corresponding to each folding angle in the folding process of the outside rearview mirror, and estimating the theoretical motion trajectory of the feature point in the preset calibration time period in the folding process of the outside rearview mirror according to the folding angle and the folding motion time corresponding to each folding angle.

[0035] Further, the processing module is further used for obtaining a position change relationship of the feature point in the world coordinate system according to the theoretical motion trajectory of the feature point, obtaining a position change relationship of the feature point in a camera coordinate system according to the actual motion trajectory of the feature point, constructing a calibration equation according to the position change relationships of the feature point in the world coordinate system and in the camera coordinate system, and calibrating the position and posture of the camera in the world coordinate system through the calibration equation.

[0036] The application further provides a vehicle comprising the camera calibration system.

[0037] The camera calibration method provided by the application changes the relative position of the feature point and the camera through the folding motion of the outside rearview mirror, constructs a calibration equation by using the position change relationship of the same feature point in the world coordinate system and in the camera coordinate system, obtains the conversion relationship from the world coordinate system to the camera coordinate system, completes the calibration of the camera, does not need to rely on the artificially arranged environment, saves the calibration site and the calibration time, and improves the calibration precision. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the application, and therefore should not be regarded as a limitation to the scope. Other related drawings can also be obtained by those skilled in the art without creative labor.

[0039] Figure 1 A flowchart of a camera calibration method provided by an embodiment of the present application is shown in FIG. 1.

[0040] Figure 2 A schematic diagram of a theoretical motion trajectory and an actual motion trajectory of a feature point provided by an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0041] The foregoing and other technical contents, features and effects of the present application will be apparent from the following detailed description of the preferred embodiments of the present application, which are given by way of example only, with reference to the accompanying drawings. The technical means and effects of the present application for achieving the intended purposes will be more fully understood from the detailed description, however, the accompanying drawings are provided for reference and illustration only, and are not intended to limit the present application, and some well-known parts can not be shown. In the various drawings, like reference numerals are used to represent like elements. For the sake of clarity, the various parts in the drawings are not drawn to scale.

[0042] An embodiment of the present application provides a camera calibration method for calibrating a camera of an outside rearview mirror of a vehicle, as shown in FIG. 1, the method comprises the following steps: Figure 1

[0043] Step S110: When the outside rearview mirror is in an unfolded state, a first image captured by the camera is acquired, a feature point is selected from the first image, and a theoretical motion trajectory of the feature point in a preset calibration time period in the folding process of the outside rearview mirror is estimated.

[0044] Specifically, before starting the calibration, the outside rearview mirror of the vehicle needs to be adjusted to the default regulatory field of view, i.e., the outside rearview mirror is in an unfolded state, and a background with a large color difference from the vehicle body is selected, so that the selected feature point can be accurately identified in the subsequent calibration process. During the folding process of the outside rearview mirror, the folding angle of the outside rearview mirror and the folding motion time corresponding to each folding angle are acquired; and according to the folding angle and the folding motion time corresponding to each folding angle, the theoretical motion trajectory of the feature point in the preset calibration time period in the folding process of the outside rearview mirror is estimated. The folding parameters of the outside rearview mirror can be obtained from the vehicle's debut information, or can be recorded in real time by the electronic control unit of the vehicle.

[0045] Optionally, a point on the vehicle body in the first image can be selected as the feature point, or a point in the first image that is stationary relative to the vehicle body can be selected as the feature point, which should be as easily identifiable as possible from the surrounding pixel points, and which should be located within the field of view of the camera at all times during the folding process of the outside rearview mirror. In other embodiments, an artificial marking method can also be used to more easily identify the feature point in the image captured by the camera.​

[0046] Step S120: In the folding process of the outside rearview mirror, the camera is controlled to collect a second image in a preset calibration time period in the folding process, feature points are identified in the second image, and an actual motion trajectory of the feature points in the calibration time period is determined.

[0047] In an embodiment, when the vehicle adopts a 60-frame camera, the preset calibration time period is 0.5 s from the start of folding of the outside rearview mirror in the unfolded state, and the 60-frame camera can collect 30 frames of images in 0.5 s. Feature point identification is performed on the 30 frames of images respectively, 30 pieces of position information of the feature points in the 30 frames of images can be obtained, and the actual motion trajectory of the feature points can be obtained according to the 30 pieces of position information.

[0048] Step S130: Calibrating the position and posture of the camera in the world coordinate system based on the theoretical motion trajectory and the actual motion trajectory of the feature points.

[0049] Specifically, the position change relationship of the feature points in the world coordinate system is obtained according to the theoretical motion trajectory of the feature points; the position change relationship of the feature points in the camera coordinate system is obtained according to the actual motion trajectory of the feature points; the calibration equation is constructed according to the position change relationship of the feature points in the world coordinate system and the camera coordinate system; and the position and posture of the camera in the world coordinate system are calibrated through the calibration equation.

[0050] Exemplarily, when the camera collects 30 frames of images in the folding process, the theoretical motion trajectory and the actual motion trajectory of the feature points are matched according to the time stamp corresponding to each frame of image, and the matching result is as shown in FIG. 6. Figure 2 Figure 2 In FIG. 6, trajectory A is the theoretical motion trajectory of the feature points, and trajectory B is the actual motion trajectory of the feature points. The world coordinates of each position of the feature points in the world coordinate system can be obtained according to the theoretical motion trajectory of the feature points, and the camera coordinates of each position of the feature points in the camera coordinate system can be obtained according to the actual motion trajectory of the feature points.

[0051] Specifically, taking the feature points from the mth position to the nth position as an example, a calibration equation is constructed according to the world coordinates and the camera coordinates at the mth position: A m X = XB m ; and a calibration equation is constructed according to the world coordinates and the camera coordinates at the nth position: A n X = XB n ; wherein A m and A n are the world coordinates in the world coordinate system at the mth position and the nth position respectively, B m and B n ​Camera coordinates of the camera at the mth position and the nth position, respectively;

[0052] Through the above two calibration equations, A m XB m = A n XB n ;

[0053] The above formula can be converted to:

[0054] Therefore, according to the calibration equations at the mth position and the nth position, the position change equation of the feature point from the mth position to the nth position is obtained: AX = XB, wherein,

[0055]

[0056] R A and T A are the rotation matrix and the translation matrix of the feature point from the mth position to the nth position in the world coordinate system, R B and T B are the rotation matrix and the translation matrix of the feature point from the mth position to the nth position in the camera coordinate system;

[0057] According to the above position change equation, we can obtain:

[0058]

[0059] By solving the above equation, the conversion relationship X from the world coordinate system to the camera coordinate system can be obtained, X includes a rotation matrix R X representing the rotation of the camera coordinate system relative to the world coordinate system, and a translation matrix T X representing the translation of the origin of the camera coordinate system relative to the origin of the world coordinate system; the specific solving process of the rotation matrix R X and the translation matrix T X is not described in detail.

[0060] The rotation matrix R X and the translation matrix T X obtained by solving can determine the position and angle of the camera installed in the world coordinate system, and can also map the points in the vehicle driving environment from the world coordinate system to the camera coordinate system:

[0061]

[0062] Wherein, (x c ,y c ,z c ,1) T is the homogeneous coordinate in the camera coordinate system, (xw y w z w 1) T is the homogeneous coordinate in the world coordinate system.

[0063] The application also provides a camera calibration system, comprising a camera, a processing module and a display module.

[0064] The camera is connected with the processing module, and is used to collect a first image when the outside rearview mirror is in an unfolded state, and collect a second image in a preset calibration time period during folding of the outside rearview mirror; the display module is connected with the processing module, and is used to display the first image and the second image collected by the camera, and select a feature point for camera calibration; the display module has a man-machine interaction function, such as a touch display screen, and a user can select a suitable feature point for calibration through the man-machine interaction function of the display module; the processing module is used to estimate a theoretical motion track of the feature point in the preset calibration time period during folding of the outside rearview mirror; during folding of the outside rearview mirror, the camera is controlled to collect the second image in the preset calibration time period during folding, the feature point is identified in the second image, and an actual motion track of the feature point in the calibration time period is determined; based on the theoretical motion track and the actual motion track of the feature point, a position and posture of the camera in the world coordinate system are calibrated.

[0065] Specifically, the processing module is further used to acquire a folding angle of the outside rearview mirror and a folding motion time corresponding to each folding angle during folding of the outside rearview mirror; and estimate the theoretical motion track of the feature point in the preset calibration time period during folding of the outside rearview mirror according to the folding angle and the folding motion time corresponding to each folding angle.

[0066] Specifically, the processing module is further used to obtain a position change relationship of the feature point in the world coordinate system according to the theoretical motion track of the feature point; obtain a position change relationship of the feature point in a camera coordinate system according to the actual motion track of the feature point; construct a calibration equation according to the position change relationship of the feature point in the world coordinate system and the position change relationship of the feature point in the camera coordinate system; and calibrate the position and posture of the camera in the world coordinate system through the calibration equation.

[0067] The application also provides a vehicle, comprising the camera calibration system as described above.

[0068] In conclusion, the camera calibration method provided by the application integrates the parameter information of the folding of the outside rearview mirror into the camera calibration, changes the relative position of the feature point and the camera through the folding movement of the outside rearview mirror, constructs a calibration equation by using the position change relationship of the same feature point in the world coordinate system and the position change relationship in the camera coordinate system, obtains the conversion relationship from the world coordinate system to the camera coordinate system, completes the calibration of the camera, and does not need to rely on the artificially arranged environment, thereby saving the calibration site and the calibration time and improving the calibration precision.

[0069] In the description of the present application, unless explicitly defined and limited, the terms "arrange", "mount", "connect" and the like should be understood in a broad sense, for example, can be fixedly connected, can be detachably connected, or integrally connected, can be mechanically connected, or electrically connected, can be directly connected, or indirectly connected through an intermediate medium. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances. The terms "first", "second", "third" and the like are only used to distinguish similar attributes of elements, and do not indicate or imply relative importance or a specific order. The terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, in addition to containing the listed elements, other elements not explicitly listed can also be contained.

[0070] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A camera calibration method for calibrating a camera of an exterior mirror of a vehicle, the method comprising: The method comprises: acquiring a first image captured by the camera when the outside rearview mirror is in an unfolded state, and selecting feature points from the first image; acquiring a folding angle of the outside rearview mirror and a folding motion time corresponding to each folding angle during a folding process of the outside rearview mirror; estimating a theoretical motion track of the feature points in a preset calibration time period during the folding process of the outside rearview mirror according to the folding angle and the folding motion time corresponding to each folding angle; controlling the camera to capture a second image in the preset calibration time period during the folding process of the outside rearview mirror, identifying the feature points in the second image, and determining an actual motion track of the feature points in the calibration time period; obtaining a position change relationship of the feature points in a world coordinate system according to the theoretical motion track of the feature points; obtaining a position change relationship of the feature points in a camera coordinate system according to the actual motion track of the feature points; constructing a calibration equation according to the position change relationship of the feature points in the world coordinate system and the camera coordinate system; calibrating a position and posture of the camera in the world coordinate system through the calibration equation.

2. The camera calibration method of claim 1, wherein, The method of constructing a calibration equation according to the position change relationship of the feature points in the world coordinate system and the camera coordinate system comprises: Construct a calibration equation according to the world coordinates and camera coordinates at the mth position: A m X = XB m ; Construct a calibration equation according to the world coordinates and camera coordinates at the nth position: A n X = XB n ; wherein A m and A n are world coordinates in a world coordinate system at the mth position and the nth position, respectively, B m and B n are camera coordinates in a camera coordinate system at the mth position and the nth position, respectively. obtaining a position change equation of the feature points from an mth position to an nth position according to the calibration equation at the mth position and the nth position: AX = XB, wherein, ， R A and T A are the rotation matrix and translation matrix of the feature point from the mth position to the nth position in the world coordinate system, R B and T B are the rotation matrix and translation matrix of the feature point from the mth position to the nth position in the camera coordinate system; A conversion relationship X from the world coordinate system to the camera coordinate system is obtained by solving the calibration equation, the X including a rotation matrix R representing a rotation of the camera coordinate system relative to the world coordinate system X , and a translation matrix T representing a translation of a camera coordinate system origin relative to a world coordinate system origin X .

3. The camera calibration method of claim 2, wherein, The method also includes determining the position coordinates of the points in the environment in the camera coordinate system according to the rotation matrix R X and the translation matrix T X obtained by solving, and converting the relationship into: wherein, is the homogeneous coordinate in the camera coordinate system, is the homogeneous coordinate in the world coordinate system.

4. The camera calibration method of claim 1, wherein, The method of selecting feature points from the first image comprises: selecting a point on a vehicle body in the first image as a feature point, or selecting a point that is stationary relative to the vehicle body in the first image as a feature point.

5. A camera calibration system, characterized by, The method comprises: The camera is connected with the processing module, and is configured to capture a first image when the outside rearview mirror is in an unfolded state, and capture a second image in a preset calibration time period during a folding process of the outside rearview mirror; The display module is connected with the processing module, and is configured to display the first image and the second image captured by the camera, and select feature points for camera calibration; The processing module is configured to acquire a folding angle of the outside rearview mirror and a folding motion time corresponding to each folding angle during a folding process of the outside rearview mirror, and estimate a theoretical motion track of the feature points in a preset calibration time period during the folding process of the outside rearview mirror according to the folding angle and the folding motion time corresponding to each folding angle; In the folding process of the outside rearview mirror, the camera is controlled to collect a second image in the preset calibration time period in the folding process, identify the feature points in the second image, and determine an actual motion trajectory of the feature points in the calibration time period; a position change relationship of the feature points in a world coordinate system is obtained according to a theoretical motion trajectory of the feature points, a position change relationship of the feature points in a camera coordinate system is obtained according to an actual motion trajectory of the feature points, a calibration equation is constructed according to the position change relationships of the feature points in the world coordinate system and the camera coordinate system, and a position and posture of the camera in the world coordinate system is calibrated through the calibration equation.

6. A vehicle characterized by comprising: The vehicle comprises the camera calibration system according to claim 5.

Images