A vehicle-mounted camera external parameter calibration method based on control points and positioning information

Abstract

The application discloses a kind of vehicle camera extrinsic calibration methods based on control point and positioning information, comprising steps S1: with the center of rear axle of vehicle as origin establishes vehicle three-dimensional coordinate system, records the coordinate information of all vehicle cameras in vehicle three-dimensional coordinate system in vehicle;Step S2: select calibration area on ground, layout control point with different number in calibration area, record the position information of each control point on ground;Step S3: vehicle advances to calibration area, records the vehicle position information in vehicle advancing process by positioning system;Step S4: screen out all the images containing control point, based on number, obtain the position information of control point on ground in image, based on the position of vehicle when shooting image, convert the position information of control point on ground into coordinate information under vehicle three-dimensional coordinate system, according to the coordinate information of vehicle camera under vehicle three-dimensional coordinate system and the coordinate information of control point, calculate the extrinsic calibration of vehicle camera.

Description

Technical Field

[0001] The present invention relates to the field of visual positioning technology, and in particular to a method for calibrating extrinsic parameters of a vehicle-mounted camera based on control points and positioning information. Background Art

[0002] In the field of computer vision, to achieve ideal three-dimensional effects or large-scale, multi-view image information, it is necessary to calibrate the position and posture of the image acquisition camera, also known as the camera's extrinsic parameters. Existing camera extrinsic calibration methods typically place feature points on the ground, measure the world coordinates of the feature points and the world coordinates of the on-board camera through engineering surveying methods, and then use the solvePNP algorithm to calculate the extrinsic parameter transformation matrix of the calibration camera based on these two coordinates. However, this calibration method is relatively cumbersome and requires a lot of work for vehicles equipped with multiple cameras, which affects calibration efficiency. Summary of the Invention

[0003] To solve the above problems, the present invention provides a vehicle-mounted camera extrinsic parameter calibration method based on control points and positioning information to solve the problems in the prior art.

[0004] To achieve the above purpose, the present invention provides a method for calibrating the external parameters of a vehicle-mounted camera based on control points and positioning information, comprising:

[0005] Step S1: establishing a vehicle-mounted three-dimensional coordinate system with the center of the vehicle's rear axle as the origin, and recording coordinate information of all vehicle-mounted cameras in the vehicle in the vehicle-mounted three-dimensional coordinate system;

[0006] Step S2: selecting a calibration area on the ground, placing control points with different numbers in the calibration area, and recording the position information of each control point on the ground;

[0007] Step S3: The vehicle moves into the calibration area, and the positioning system records the vehicle position information during the vehicle's movement. Based on the vehicle position information, it is determined whether the positioning system accurately positions the vehicle. If so, step S4 is executed; if not, the vehicle returns to the starting position and the positioning system is calibrated.

[0008] Step S4: Start the vehicle-mounted camera to continuously capture external images, record the position of the vehicle when each frame of the image is captured through the positioning device, filter out all images containing the control point, obtain the position information of the control point on the ground in the image based on the number, and based on the position of the vehicle when the image is captured, convert the position information of the control point on the ground into coordinate information in the vehicle-mounted three-dimensional coordinate system, and calculate the external parameters of the vehicle-mounted camera based on the coordinate information of the vehicle-mounted camera and the coordinate information of the control point in the vehicle-mounted three-dimensional coordinate system.

[0009] Furthermore, in step S4, calculating the extrinsic parameters of the vehicle-mounted camera includes the following steps:

[0010] Step S41: acquiring the first frame of the image in which the control point appears in the captured image in the order captured by the vehicle-mounted camera, recording the number of the control point in the first frame of the image, and calculating the coordinates of the control point in the vehicle-mounted three-dimensional coordinate system at this time;

[0011] Step S42: Continuing to acquire a second frame of image containing the control points, obtaining the numbers of the control points in the second frame of image, removing the control points that have already appeared in the first frame of image, and calculating the coordinates of the newly appeared control points in the vehicle-borne three-dimensional coordinate system. Repeating this step until the coordinate calculation of all the control points in the vehicle-borne three-dimensional coordinate system is completed;

[0012] Step S43: using the coordinates of each control point and the corresponding vehicle position coordinates as a set of input parameters, calculating the vehicle camera extrinsic parameters corresponding to each control point;

[0013] Step S44: Select the vehicle-mounted camera external parameters corresponding to each of the control points in turn, reversely calculate the virtual coordinates of all the control points, and select the external parameters that make the errors between the virtual coordinates and the actual coordinates of each control point within a preset accuracy range and with the smallest error value as the external parameters of the vehicle-mounted camera.

[0014] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0015] 1. The present invention establishes a vehicle-mounted three-dimensional coordinate system with the rear axle of the vehicle as the origin, thereby determining the coordinates of each vehicle-mounted camera on the vehicle in the coordinate system; a positioning device is used to constantly locate the position of the vehicle. Since the vehicle-mounted three-dimensional coordinate system has been established with the rear axle of the vehicle as the center, the origin of the vehicle-mounted three-dimensional coordinate system can be determined based on the vehicle position, and then the position information of the control point on the ground is converted into coordinates in the vehicle-mounted coordinate system based on the position of the control point on the ground and the current position of the origin of the vehicle-mounted three-dimensional coordinate system. Finally, the external parameters of the vehicle-mounted camera can be calculated using the coordinates of the vehicle-mounted camera and the coordinates of the control point; the present invention sets a positioning device in the vehicle and establishes a vehicle-mounted three-dimensional coordinate system, so that during calibration, the external parameters of the vehicle-mounted camera can be calibrated only according to the position of the vehicle, thereby eliminating the need to measure the relative distance between the control point and the vehicle-mounted camera in the traditional calibration method, thereby greatly improving the calibration efficiency of the vehicle-mounted camera.

[0016] 2. Since each control point is numbered, the position of the control point can be quickly obtained through the number of the control point; the extrinsic parameters of the vehicle-mounted camera are calculated by the coordinates of the vehicle-mounted camera, the coordinates of the control point and the actual distance between the vehicle-mounted camera and the control point, so each control point can be used to calculate the extrinsic parameters of the vehicle-mounted camera; by moving the vehicle to different positions and using different control points to calculate the extrinsic parameters of the vehicle-mounted camera, multiple calculation results of the extrinsic parameters are obtained, and since the points in the image are converted into actual coordinates through the extrinsic parameter matrix of the vehicle-mounted camera, the coordinates of the control points are reversely calculated through step S44, the accuracy of each extrinsic parameter can be verified one by one, and the extrinsic parameter with the highest accuracy is selected as the extrinsic parameter of the vehicle-mounted camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information according to the present invention;

[0018] Figure 2 A top view of the vehicle of the present invention when traveling;

[0019] Figure 3 are the coordinates of the control point of the present invention in the vehicle-mounted three-dimensional coordinate system.

[0020] In the figure: 1. Vehicle; 2. Control point; 3. Vehicle-mounted camera. DETAILED DESCRIPTION

[0021] The following is a further explanation of the embodiments of the present invention with reference to the accompanying drawings. It should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the modules or components referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the present invention.

[0022] like Figure 1 and Figure 2 As shown, a method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information includes:

[0023] Step S1: establishing a vehicle-mounted three-dimensional coordinate system with the center of the rear axle of the vehicle 1 as the origin, and recording the coordinate information of all vehicle-mounted cameras 3 in the vehicle 1 in the vehicle-mounted three-dimensional coordinate system;

[0024] Step S2: Select a calibration area on the ground, arrange control points 2 with different numbers in the calibration area, and record the position information of each control point 2 on the ground;

[0025] Step S3: Vehicle 1 moves into the calibration area, and the positioning system records the position information of multiple vehicles 1 during the moving process. Based on the position information, it is determined whether the positioning system accurately locates the vehicle 1. If so, step S4 is executed. If not, the vehicle 1 returns to the starting position and the positioning system is calibrated.

[0026] Step S4: Start the vehicle-mounted camera 3 to continuously capture external images, record the position of the vehicle 1 when each frame of the image is captured through the positioning device, filter out images containing all control points 2, obtain the position information of the control points 2 on the ground in the image based on the number, and convert the position information of the control points 2 on the ground into coordinate information in the vehicle-mounted three-dimensional coordinate system based on the position of the vehicle 1 when the image is captured. Calculate the external parameters of the vehicle-mounted camera 3 based on the coordinate information of the vehicle-mounted camera 3 and the coordinate information of the control points 2 in the vehicle-mounted three-dimensional coordinate system.

[0027] The present invention establishes a vehicle-borne three-dimensional coordinate system with the rear axle of the vehicle 1 as the origin, thereby determining the coordinates of each vehicle-borne camera 3 on the vehicle 1 in the coordinate system; a positioning device is used to constantly locate the position of the vehicle 1. Since the vehicle-borne three-dimensional coordinate system has been established with the rear axle of the vehicle 1 as the center, the origin of the vehicle-borne three-dimensional coordinate system can be determined according to the position of the vehicle 1, and then the position information of the control point 2 on the ground is converted into coordinates in the vehicle-borne coordinate system according to the position of the control point 2 on the ground and the current position of the origin of the vehicle-borne three-dimensional coordinate system. Finally, the external parameters of the vehicle-borne camera 3 can be calculated by the coordinates of the vehicle-borne camera 3 and the coordinates of the control point 2. The present invention sets a positioning device in the vehicle 1 and establishes a vehicle-borne three-dimensional coordinate system, so that during calibration, the external parameters of the vehicle-borne camera 3 can be calibrated only according to the position of the vehicle 1, thereby eliminating the need to measure the relative distance between the control point 2 and the vehicle-borne camera 3 in the traditional calibration method, thereby greatly improving the calibration efficiency of the vehicle-borne camera 3.

[0028] In step S1, the vehicle-mounted three-dimensional coordinate system takes the horizontal plane where the rear axle center is located as the XOY plane, and the direction of the line between the rear axle center of vehicle 1 and the front axle center of vehicle 1 as the Y axis. Figure 2 The direction pointing to the right side of the vehicle 1 in the XOY plane is the X-axis, and the direction perpendicular to the XOY plane is the Z-axis.

[0029] In step S2, multiple control points 2 are set in the calibration area. Each control point 2 can be used to calculate the extrinsic parameters of the vehicle-mounted camera 3. Therefore, by setting multiple control points 2, different calculation results of the extrinsic parameters of the same vehicle-mounted camera 3 can be obtained. By selecting the most accurate extrinsic parameter calculation result from the multiple calculation results, the accuracy of the extrinsic parameter calculation of the vehicle-mounted camera 3 can be improved.

[0030] In step S3, the positioning system includes a first positioning module and a second positioning module. The first positioning module and the second positioning module locate the position of vehicle 1 at preset time intervals and generate first positioning coordinates and second positioning coordinates respectively. The position information of vehicle 1 is the average value of the first positioning coordinates and the second positioning coordinates. After vehicle 1 arrives at the calibration area, the standard deviation of the difference between the first positioning coordinates and the second positioning coordinates of each position is calculated, and it is determined whether the standard deviation is within the preset range. If yes, vehicle 1 continues to move forward. If not, vehicle 1 returns to the starting position and calibrates the first positioning module and the second positioning module.

[0031] When the vehicle 1 advances to the calibration area, the first positioning module and the second positioning module can detect whether the positioning of the vehicle 1 from the starting position to the arrival at the calibration area is accurate. If the standard deviation of the error of the first positioning coordinate and the second positioning coordinate is large, it proves that the error fluctuation between the two is large. The main reason is that during the forward movement of the vehicle 1, the position of the vehicle 1 positioned by the first positioning module and the second positioning module has a large offset at a certain moment or within a certain time period, that is, the positioning information of the first positioning module or the second positioning module is inaccurate. If it is not adjusted, it will seriously affect the accuracy of the external parameters of the vehicle-mounted camera 3.

[0032] In this embodiment, the first positioning module and the second positioning module are both UBW positioning modules. The positioning accuracy of the UBW positioning module can reach centimeter level, which can effectively ensure the accuracy of the position positioning of the vehicle 1.

[0033] In this embodiment, the ground position information of control point 2 and the position information of vehicle 1 both include longitude and latitude, with the position information of vehicle 1 representing the longitude and latitude of the origin of the vehicle's 3D coordinate system. If the position information of vehicle 1 acquired by the positioning device is not at the origin of the vehicle's 3D coordinate system, the position information of the origin of the vehicle's 3D coordinate system will need to be calculated based on this position information, which increases the complexity of the calibration process.

[0034] In step S4, calculating the external parameters of the vehicle-mounted camera 3 includes the following steps:

[0035] Step S41: obtaining the first frame of image in which the control point 2 appears in the captured image in the order captured by the vehicle-mounted camera 3, recording the number of the control point 2 in the first frame of image, and calculating the coordinates of the control point 2 in the vehicle-mounted three-dimensional coordinate system at this time;

[0036] Step S42: Continue to acquire a second frame image containing control point 2, obtain the number of control point 2 in the second frame image, eliminate the control point 2 that has already appeared in the first frame image, calculate the coordinates of the newly appeared control point 2 in the vehicle-borne three-dimensional coordinate system, and repeat this step until the coordinates of all control points 2 in the vehicle-borne three-dimensional coordinate system are calculated;

[0037] Step S43: using the coordinates of each control point 2 and the corresponding position coordinates of the vehicle 1 as a set of input parameters, calculating the external parameters of the vehicle-mounted camera 3 corresponding to each control point 2;

[0038] Step S44: Select the external parameters of the vehicle-mounted camera 3 corresponding to each control point 2 in turn, reversely calculate the virtual coordinates of all control points 2, and select the external parameters that make the error between the virtual coordinates and the actual coordinates of each control point 2 within the preset accuracy range and the smallest error value as the external parameters of the vehicle-mounted camera 3.

[0039] Since each control point 2 is numbered, the position of the control point 2 can be quickly obtained through the number of the control point 2; the external parameters of the vehicle-mounted camera 3 are calculated by the coordinates of the vehicle-mounted camera 3, the coordinates of the control point 2 and the actual distance between the vehicle-mounted camera 3 and the control point 2, so each control point 2 can be used to calculate the external parameters of the vehicle-mounted camera 3; by moving the vehicle 1 to different positions and using different control points 2 to calculate the external parameters of the vehicle-mounted camera 3, multiple calculation results of the external parameters are obtained, and since the points in the image are converted into actual coordinates through the external parameter matrix of the vehicle-mounted camera 3, the coordinates of the control point 2 are reversely calculated through step S44, the accuracy of each external parameter can be verified one by one, and the external parameter with the highest accuracy is selected as the external parameter of the vehicle-mounted camera 3.

[0040] The coordinates of control point 2 in the vehicle-mounted three-dimensional coordinate system are:

[0041] Among them, s i is the distance between control point 2 and the origin of the vehicle-mounted three-dimensional coordinate system, h is the height of the origin of the vehicle-mounted three-dimensional coordinate system from the ground, θ i is the angle between the line connecting the control point 2 and the origin of the vehicle-mounted three-dimensional coordinate system and the X-axis; based on the position information of the vehicle 1 located by the positioning device and the latitude and longitude information of the control point 2 itself, the distance s between the control point 2 and the origin of the vehicle-mounted three-dimensional coordinate system can be directly obtained. i , then based on the distance s i The projection on the XOY plane can conveniently calculate the X-axis and Y-axis coordinates of the control point 2 in the vehicle-mounted three-dimensional coordinate system, and then complete the transformation of the control point 2 to the vehicle-mounted three-dimensional coordinate system.

[0042] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0043] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0044] The above embodiments merely represent several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be based on the appended claims.

[0045] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information, characterized in that: include: Step S1. Establishing a vehicle-mounted three-dimensional coordinate system with the center of the vehicle's rear axle as the origin, and recording the coordinate information of all vehicle-mounted cameras in the vehicle in the vehicle-mounted three-dimensional coordinate system; Step S2. Select a calibration area on the ground, arrange control points with different numbers in the calibration area, and record the position information of each control point on the ground; Step S3. The vehicle advances into the calibration area, and the positioning system records the vehicle's position information during the vehicle's advance. Based on the vehicle's position information, the positioning system determines whether the vehicle's positioning is accurate. If so, step S4 is executed. If not, the vehicle returns to its starting position and the positioning system is calibrated. Step S4. Starting the vehicle-mounted camera to continuously capture external images, recording the position of the vehicle at the time of capturing each frame of the image using a positioning device, selecting all images containing the control point, obtaining the position information of the control point on the ground within the image based on the number, converting the position information of the control point on the ground into coordinate information in the vehicle-mounted three-dimensional coordinate system based on the position of the vehicle at the time of capturing the image, and calculating the external parameters of the vehicle-mounted camera based on the coordinate information of the vehicle-mounted camera and the coordinate information of the control point in the vehicle-mounted three-dimensional coordinate system; In step S4, calculating the extrinsic parameters of the vehicle-mounted camera includes the following steps: Step S41. Acquire the first frame of the image in which the control point appears in the captured image in the order captured by the vehicle-mounted camera, record the number of the control point in the first frame of the image, and calculate the coordinates of the control point in the vehicle-mounted three-dimensional coordinate system at this time; Step S42: Continue acquiring a second frame of image containing the control points, obtain the numbers of the control points in the second frame, remove the control points that already appear in the first frame of image, and calculate the coordinates of the newly appearing control points in the vehicle-borne three-dimensional coordinate system. Repeat this step until the coordinates of all control points in the vehicle-borne three-dimensional coordinate system are calculated. Step S43. Using the coordinates of each control point and the corresponding vehicle position coordinates as a set of input parameters, calculate the vehicle camera extrinsic parameters corresponding to each control point; Step S44. Select the vehicle-mounted camera external parameters corresponding to each of the control points in turn, reversely calculate the virtual coordinates of all the control points, and select the external parameters that make the errors between the virtual coordinates and the actual coordinates of each control point within the preset accuracy range and with the smallest error value as the external parameters of the vehicle-mounted camera.

2. The method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information according to claim 1, characterized in that: In step S1, the vehicle-mounted three-dimensional coordinate system takes the horizontal plane where the center of the rear axle of the vehicle is located as the XOY plane, the direction of the line between the center of the rear axle of the vehicle and the center of the front axle of the vehicle as the Y axis, the direction perpendicular to the Y axis in the XOY plane as the X axis, and the direction perpendicular to the XOY plane as the Z axis.

3. The method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information according to claim 1, characterized in that: In step S2, a plurality of control points are set in the calibration area.

4. The method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information according to claim 1, wherein: In step S3, the positioning system includes a first positioning module and a second positioning module. The first positioning module and the second positioning module locate the position of the vehicle at preset time intervals and generate first positioning coordinates and second positioning coordinates respectively. The vehicle position information is the average value of the first positioning coordinates and the second positioning coordinates. After the vehicle arrives at the calibration area, the standard deviation of the difference between the first positioning coordinates and the second positioning coordinates of each position is calculated, and it is determined whether the standard deviation is within a preset range. If yes, the vehicle continues to move forward. If not, the vehicle returns to the starting position and calibrates the first positioning module and the second positioning module.

5. The method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information according to claim 1, characterized in that: The position information of the control point on the ground and the vehicle position information both include longitude and latitude, and the vehicle position information is the longitude and latitude of the origin of the vehicle-mounted three-dimensional coordinate system.

6. The method for extrinsic calibration of a vehicle-mounted camera based on control points and positioning information according to claim 1, characterized in that: The coordinates of the control point in the vehicle-mounted three-dimensional coordinate system are: , where s i is the distance between the control point and the origin of the vehicle-borne three-dimensional coordinate system, h is the height of the origin of the vehicle-borne three-dimensional coordinate system from the ground, θ i The angle between the line connecting the control point and the origin of the vehicle-borne three-dimensional coordinate system and the X-axis.

Images