Camera calibration mode determination method, device and system and storage medium

Abstract

The invention discloses a method, a device and a system for determining a camera calibration mode, and a storage medium. The method, the device and the system are used for selecting the camera calibration mode with the highest accuracy for different intersections. The method comprises the following steps: acquiring a plurality of candidate driving tracks respectively corresponding to a vehicle in a target area in a plurality of camera calibration modes; comparing the plurality of candidate driving tracks with a real driving track; selecting a target driving track with the highest similarity degree with the real driving track from the plurality of candidate driving tracks according to a comparison result; and determining a camera calibration mode corresponding to the target driving track as an optimal calibration mode. The method is used for automatically selecting an optimal camera calibration mode according to a scene of a target area and improving the accuracy of road information identification.

Description

Technical Field

[0001] The present application relates to the field of camera calibration technology, and in particular to a method, device, system and storage medium for determining a camera calibration method. Background Art

[0002] Roadside perception is the use of sensors such as cameras, millimeter-wave radars, and lidars, combined with roadside edge computing, with the ultimate goal of achieving instantaneous intelligent perception of traffic participants, road conditions, etc. on the road section. In the roadside perception system, cameras are used to capture various information on the road, such as vehicles and pedestrians. In order to accurately extract information from these images, such as speed and position, camera calibration must be performed. The purpose of camera calibration is to eliminate or correct errors in the imaging process and ensure that the data obtained from the image is consistent with the size and position of the actual object. Since there are many ways to calibrate a camera, and different camera calibration methods differ in applicable scenarios, the optimal calibration method needs to be determined for different intersections, different areas, and different cameras. At present, manual testing and selection of appropriate camera calibration methods are often required, which is labor-intensive and inefficient.

[0003] Therefore, how to provide a method for determining a camera calibration method to automatically select the optimal camera calibration method according to the scene in the target area and improve the accuracy of road information recognition has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a method, device, system and storage medium for determining a camera calibration method, which are used to automatically select the optimal camera calibration method according to the scene of the target area to improve the accuracy of road information recognition.

[0005] The present application provides a method for determining a camera calibration mode, comprising:

[0006] Obtain multiple candidate driving trajectories corresponding to vehicles in the target area under multiple camera calibration methods;

[0007] Comparing the plurality of candidate driving trajectories with the actual driving trajectories;

[0008] Selecting a target driving trajectory with the highest degree of similarity to the actual driving trajectory from the plurality of candidate driving trajectories according to the comparison result;

[0009] Determine that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method.

[0010] The beneficial effect of the present application is that: by automatically acquiring multiple candidate driving trajectories corresponding to vehicles in the target area under multiple camera calibration methods, and comparing the multiple candidate driving trajectories with the actual driving trajectories, and then selecting the target driving trajectory with the highest degree of similarity to the actual driving trajectory from the multiple candidate driving trajectories according to the comparison results, and finally determining that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method. Thus, the optimal camera calibration method is automatically selected according to the scene in the target area, and the accuracy of road information recognition is improved.

[0011] In one embodiment, the multiple candidate driving trajectories corresponding to the vehicles in the target area under multiple camera calibration modes are obtained, including:

[0012] The candidate driving trajectories of the vehicles in the target area under the checkerboard calibration method and the candidate driving trajectories of the vehicles in the target area under the feature point calibration method are obtained.

[0013] In one embodiment, comparing the plurality of candidate driving trajectories with the actual driving trajectories includes:

[0014] Display the map corresponding to the target area on the bird's-eye view;

[0015] Projecting the plurality of candidate driving trajectories and the actual driving trajectory onto a map corresponding to the target area;

[0016] The plurality of candidate driving trajectories are compared with the actual driving trajectories on a map corresponding to the target area.

[0017] In one embodiment, when the target area is an intersection, displaying a map corresponding to the target area on a bird's-eye view includes:

[0018] The central longitude and latitude of the intersection are used as the origin of the local coordinates, and the geodetic coordinate system corresponding to the map is converted into the local coordinate system of the target area;

[0019] Then, the local coordinate system of the target area is converted into the pixel coordinates of the target area on the bird's-eye view;

[0020] The map corresponding to the target area is displayed on the bird's-eye view according to the pixel coordinates of the target area on the bird's-eye view.

[0021] In one embodiment, projecting the plurality of candidate driving trajectories and the actual driving trajectory onto a map corresponding to the target area includes:

[0022] Converting the longitude and latitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system;

[0023] Converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories;

[0024] The plurality of candidate driving trajectories and the actual driving trajectories are projected onto a map corresponding to the target area according to pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories.

[0025] In one embodiment, converting the latitude and longitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system includes:

[0026] The latitude and longitude coordinates corresponding to the multiple candidate driving trajectories and the actual driving trajectories are converted into the local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system by the following formula:

[0027] x=(lon-lon base )*Π / 180*R a *cos(lat*Π / 180);

[0028] y=(lat-lat base )*Π / 180*R b ;

[0029] Where (x, y) is the coordinate of the local coordinate system; (lon base ,lat base ) is the longitude and latitude of the center of the intersection; (lon, lat) is the longitude and latitude corresponding to the map; R a is the radius of the Earth, the distance from the center of the Earth to the equator; R b is the polar radius, the distance from the center of the Earth to the South Pole or the North Pole.

[0030] In one embodiment, converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories comprises:

[0031] The local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the multiple candidate driving trajectories and the actual driving trajectories by the following formula:

[0032] pix x =floor(x-limit x [0])*pixel2dist;

[0033] pix y =height-floor(y-limity [0])*pixel2dist;

[0034] Among them, (pix x ,pix y ) is the pixel coordinate of the bird's-eye view; floor represents the floor rounding function; height is the height of the bird's-eye view; (x, y) is the coordinate of the local coordinate system; limit x and limity represent the size of the region of interest, both are lists; pixel2dist represents the number of pixels in the bird's-eye view corresponding to the target area within 1m.

[0035] The present application also provides a device for determining a camera calibration method, comprising:

[0036] An acquisition module, used to acquire multiple candidate driving trajectories corresponding to vehicles in a target area under multiple camera calibration methods;

[0037] A comparison module, used for comparing the plurality of candidate driving trajectories with the actual driving trajectories;

[0038] A selection module, configured to select a target driving trajectory having the highest degree of similarity to the actual driving trajectory from the plurality of candidate driving trajectories according to the comparison result;

[0039] The determination module is used to determine that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method.

[0040] In one embodiment, the acquisition module is further used to:

[0041] The candidate driving trajectories of the vehicles in the target area under the checkerboard calibration method and the candidate driving trajectories of the vehicles in the target area under the feature point calibration method are obtained.

[0042] In one embodiment, the comparison module includes:

[0043] A display submodule, used to display a map corresponding to the target area on the bird's-eye view;

[0044] A projection submodule, used for projecting the plurality of candidate driving trajectories and the actual driving trajectory onto a map corresponding to the target area;

[0045] The comparison submodule is used to compare the multiple candidate driving trajectories with the actual driving trajectories on the map corresponding to the target area.

[0046] In one embodiment, when the target area is an intersection, the display submodule is further used to:

[0047] The central longitude and latitude of the intersection are used as the origin of the local coordinates, and the geodetic coordinate system corresponding to the map is converted into the local coordinate system of the target area;

[0048] Then, the local coordinate system of the target area is converted into the pixel coordinates of the target area on the bird's-eye view;

[0049] The map corresponding to the target area is displayed on the bird's-eye view according to the pixel coordinates of the target area on the bird's-eye view.

[0050] In one embodiment, the projection submodule is further used to:

[0051] Converting the longitude and latitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system;

[0052] Converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories;

[0053] The plurality of candidate driving trajectories and the actual driving trajectories are projected onto a map corresponding to the target area according to pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories.

[0054] In one embodiment, converting the latitude and longitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system includes:

[0055] The latitude and longitude coordinates corresponding to the multiple candidate driving trajectories and the actual driving trajectories are converted into the local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system by the following formula:

[0056] x=(lon-lon base )*Π / 180*R a *cos(lat*Π / 180);

[0057] y=(lat-lat base )*Π / 180*R b ;

[0058] Where (x, y) is the coordinate of the local coordinate system; (lon base ,lat base ) is the longitude and latitude of the center of the intersection; (lon, lat) is the longitude and latitude corresponding to the map; R a is the radius of the Earth, the distance from the center of the Earth to the equator; R b is the polar radius, the distance from the center of the Earth to the South Pole or the North Pole.

[0059] In one embodiment, converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories comprises:

[0060] The local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the multiple candidate driving trajectories and the actual driving trajectories by the following formula:

[0061] pix x =floor(x-limit x [0])*pixel2dist;

[0062] pix y =height-floor(y-limit y [0])*pixel2dist;

[0063] Among them, (pix x ,pix y ) is the pixel coordinate of the bird's-eye view; floor represents the floor rounding function; height is the height of the bird's-eye view; (x, y) is the coordinate of the local coordinate system; limit x and limit y Indicates the size of the region of interest, all of which are lists; pixel2dist indicates the number of pixels in the bird's-eye view corresponding to the target area within 1m.

[0064] The present application also provides a system for determining a camera calibration method, including:

[0065] at least one processor; and,

[0066] a memory communicatively connected to the at least one processor; wherein,

[0067] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the method for determining the camera calibration method recorded in any of the above embodiments.

[0068] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the camera calibration method determination system, the camera calibration method determination system can implement the camera calibration method determination method recorded in any of the above embodiments.

[0069] Other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0070] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings:

[0072] Figure 1 This is a flow chart of a method for determining a camera calibration method in an embodiment of the present application;

[0073] Figure 2 This is a display diagram of a chessboard in the camera field of view in one embodiment of the present application;

[0074] Figure 3 A distribution diagram of intersection feature points within the camera field of view in an embodiment of the present application;

[0075] Figure 4 This is a projection effect diagram of a ground point at an intersection in an embodiment of the present application;

[0076] Figure 5 A bird's-eye view of a map corresponding to a target area in an embodiment of the present application

[0077] Figure 6 It is a structural schematic diagram of a device for determining a camera calibration method in one embodiment of the present application;

[0078] Figure 7 Schematic diagram of the hardware structure of a system for determining a camera calibration method in one embodiment of the present application. DETAILED DESCRIPTION

[0079] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application.

[0080] RTK (Real time kinematic) is a technology used for high-precision positioning. It uses the differential method of real-time processing of the carrier phase observations of two measuring stations to send the carrier phase collected by the base station to the user receiver for coordinate difference calculation. The RTK receiver can receive GPS satellite information and calculate high-precision position information. Install the RTK receiver on the vehicle, and then connect the RTK receiver to the vehicle to obtain the data generated by the RTK receiver, such as the vehicle's position, speed, direction, etc. RTK technology can be used to obtain data such as the real coordinate points and driving trajectory of the road surface.

[0081] The calibration of the camera is mainly to determine the parameters of the camera, including internal parameters and external parameters. Among them, the internal parameters mainly describe the internal structure of the camera, including the focal length, distortion parameters, etc. of the camera; the external parameters describe the position and posture of the camera in the world coordinate system, including the rotation matrix and translation vector. Through camera calibration, we can obtain the internal and external parameters of the camera, thereby improving the processing accuracy of the image, and use it to obtain the position and posture of the vehicle relative to the environment in autonomous driving. The method provided in this application can integrate multiple camera calibration technologies, and can select different calibration methods for different intersections, different areas of the same intersection, and multiple cameras at the intersection, so as to optimize the recognition effect of the camera.

[0082] Figure 1 Flow chart of a method for determining a camera calibration method in an embodiment of the present application. Figure 1 As shown, the method can be implemented as the following steps S101-S104:

[0083] In step S101, a plurality of candidate driving trajectories corresponding to vehicles in a target area under a plurality of camera calibration modes are obtained;

[0084] In step S102, the plurality of candidate driving trajectories are compared with the actual driving trajectory;

[0085] In step S103, a target driving trajectory having the highest degree of similarity to the actual driving trajectory is selected from the plurality of candidate driving trajectories according to the comparison result;

[0086] In step S104, it is determined that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method.

[0087] In this application, multiple candidate driving trajectories corresponding to vehicles in the target area under multiple camera calibration methods are obtained. Since there are multiple camera calibration methods, in one embodiment of this application, two camera calibration methods, chessboard calibration and feature point calibration, are used as examples for explanation, that is, candidate driving trajectories of vehicles in the target area under the chessboard calibration method and candidate driving trajectories of vehicles in the target area under the feature point calibration method are obtained. Among them: Figure 2 This is a diagram showing a chessboard in the camera field of view in one embodiment of the present application. Figure 2 As shown in the figure, the chessboard calibration is to first place chessboards at different positions in the camera's field of view, and use Zhang Zhengyou's calibration method to obtain the camera's intrinsic parameters and distortion coefficients. Figure 3 is a distribution diagram of the intersection feature points within the camera field of view in one embodiment of the present application, such as Figure 3 As shown in the figure, RTK recorded positions are used as feature points at different places in the intersection; then, the pixel coordinates of the feature points in the camera field of view are determined; the pixel coordinates of the feature points and the corresponding RTK coordinates are input into the preset algorithm, and the calculated camera intrinsic parameters and distortion coefficients are input to obtain the external parameters; according to the camera intrinsic parameters, distortion coefficients and external parameters, the 3D ground points in the scene can be projected into the picture to obtain the corresponding relationship between the ground point pixels and the 3D positions in the intersection area. The projection effect is as follows: Figure 4 As shown, the checkerboard calibration method is completed. Calibration based on feature points is also a reference Figure 3 The artificially set feature points in the image are calculated according to the pixel coordinates of the feature points and the corresponding RTK longitude and latitude, and the conversion relationship between the two is obtained, which is the calibration based on the feature points. Since both the checkerboard calibration and the calibration method based on feature points are prior arts, this application will not be elaborated in detail. Through two camera calibration methods respectively, the candidate driving trajectories under the corresponding camera calibration method are obtained. Specifically, the camera recognition goal is to first capture the vehicle image, the image contains the vehicle's location information, and then use image recognition technology to identify the vehicle, such as various target detection algorithms to identify the vehicle, and calculate the vehicle's location pixel points. The longitude and latitude of the vehicle are then obtained based on the internal and external parameters. The main process is to convert the pixel points into points in the camera coordinate system, then into points in the world coordinate system, and finally into points in the geographic coordinate system.

[0088] Compare the multiple candidate driving trajectories with the actual driving trajectory. In the present application, the RTK driving trajectory of the vehicle is obtained as the actual driving trajectory of the vehicle. In order to compare the multiple candidate driving trajectories with the actual driving trajectory, first display the map corresponding to the target area on the bird's-eye view; then, project the multiple candidate driving trajectories and the actual driving trajectory onto the map corresponding to the target area; and then compare the multiple candidate driving trajectories with the actual driving trajectory on the map corresponding to the target area. Specifically, the process of displaying the map corresponding to the target area on the bird's-eye view is as follows: take the central longitude and latitude of the intersection as the origin of the local coordinates, set the area of ​​interest to determine the display area of ​​the map, such as selecting a preset area within the intersection. Since the coordinates of the map are the geodetic coordinate system, that is, the WGS84 coordinate system, the geodetic coordinate system corresponding to the map is converted into the local coordinate system corresponding to the target area. For example, the geodetic coordinate system can be converted into the local coordinate system corresponding to the target area by the following formula:

[0089] x=(lon-lon base )*Π / 180*R a *cos(lat*Π / 180);

[0090] y=(lat-lat base )*Π / 180*R b ;

[0091] Where (x, y) is the coordinate of the local coordinate system; (lon base ,lat base ) is the longitude and latitude of the center of the intersection; (lon, lat) is the longitude and latitude corresponding to the map; R a is the radius of the Earth, the distance from the center of the Earth to the equator; R b is the polar radius, the distance from the center of the Earth to the South Pole or the North Pole.

[0092] Then, the local coordinate system corresponding to the target area is converted into the pixel coordinates of the target area on the bird's-eye view. For example, the local coordinate system corresponding to the target area can be converted into the pixel coordinates of the target area on the bird's-eye view by the following formula:

[0093] pix x =floor(x-limit x [0])*pixel2dist;

[0094] pix y =height-floor(y-limit y [0])*pixel2dlst;

[0095] Among them, (pixx,pix y) is the pixel coordinate of the bird's-eye view; floor represents the floor rounding function; height is the height of the bird's-eye view; (x, y) is the coordinate of the local coordinate system; limit x and limity represent the size of the region of interest, both are lists; pixel2 dist represents the number of pixels in the bird's-eye view corresponding to the target area within 1m.

[0096] Specifically, use limit x and limit y Indicates the display area range after the target area is converted to the local coordinate system coordinates, for example, the center point is (lon, lat), and then limit x For the list [-100,100], limit y For the list [-50,50], it means that the map range is 100 meters to the left, 100 meters to the right, 50 meters down, and 50 meters up, with (lon, lat) as the origin. x [0] is -100.

[0097] Furthermore, the map corresponding to the target area can be displayed on the bird's-eye view according to the pixel coordinates of the target area on the bird's-eye view. Figure 5 is a bird's-eye view of a map corresponding to a target area in an embodiment of the present application, such as Figure 5 Shown is the converted map display area.

[0098] After obtaining the bird's-eye view of the map corresponding to the target area, the multiple candidate driving trajectories and the actual driving trajectory are projected onto the map corresponding to the target area. That is to say, after the map is displayed, the RTK driving trajectory and the driving trajectory obtained by camera calibration will be projected into the map display area together. Similarly, the longitude and latitude coordinates corresponding to the multiple candidate driving trajectories and the actual driving trajectory are first converted into the local coordinates of the multiple candidate driving trajectories and the actual driving trajectory in the local coordinate system. For example, the longitude and latitude coordinates corresponding to the multiple candidate driving trajectories and the actual driving trajectory can be converted into the local coordinates of the multiple candidate driving trajectories and the actual driving trajectory in the local coordinate system by the following formula:

[0099] x=(lon-lon base )*Π / 180*R a *cos(lat*Π / 180);

[0100] y=(lat-lat base )*Π / 180*R b ;

[0101] Where (x, y) is the coordinate of the local coordinate system; (lon base,lat base ) is the longitude and latitude of the center of the intersection; (lon, lat) is the longitude and latitude corresponding to the map; R a is the radius of the Earth, the distance from the center of the Earth to the equator; R b is the polar radius, the distance from the center of the Earth to the South Pole or the North Pole.

[0102] Then, the local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the multiple candidate driving trajectories and the actual driving trajectories. For example, the local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the multiple candidate driving trajectories and the actual driving trajectories by the following formula:

[0103] pix x =floor(x-limit x [0])*pixel2dist;

[0104] pix y =height-floor(y-limit y [0])*pixel2dist;

[0105] Among them, (pix x ,pix y ) is the pixel coordinate of the bird's-eye view; floor represents the floor rounding function; height is the height of the bird's-eye view; (x, y) is the coordinate of the local coordinate system; limit x and limity represent the size of the region of interest, both are lists; pixel2 dist represents the number of pixels in the bird's-eye view corresponding to the target area within 1m.

[0106] Finally, the plurality of candidate driving trajectories and the actual driving trajectories are projected onto a map corresponding to the target area according to the pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories.

[0107] The plurality of candidate driving trajectories are compared with the actual driving trajectories on the map corresponding to the target area. For the case of two camera calibration methods, there will be three trajectories of the same vehicle target in the map display area, and then the actual driving trajectory of RTK is used as a reference to see which of the candidate driving trajectories of the other two camera calibration methods is closest to the RTK driving trajectory. There are many specific determination methods, for example, the similarity between the candidate driving trajectory and the RTK driving trajectory can be calculated as the corresponding comparison result.

[0108] According to the comparison result, a target driving trajectory with the highest similarity to the actual driving trajectory is selected from the plurality of candidate driving trajectories, and the candidate driving trajectory with the highest similarity is determined as the optimal driving trajectory. Furthermore, the camera calibration method corresponding to the target driving trajectory is determined to be the optimal calibration method.

[0109] In one embodiment of the present application, after determining that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method, the camera calibration method of the target area is adjusted to the optimal calibration method.

[0110] In one embodiment of the present application, the target area can also be evenly divided into multiple small areas, for example, it can be set to be evenly divided into 9 parts, 6 parts, etc., or the segmentation setting can be customized. In these different small areas, the candidate driving trajectories that are closer to the actual driving trajectories are respectively selected as the corresponding target driving trajectories, and the camera calibration method corresponding to the target driving trajectory is determined to be the optimal calibration method.

[0111] In the application process of this method, for different intersections, different cameras, and different small areas, you can choose to select the corresponding camera calibration method and determine the corresponding camera parameters through the method provided by this application. Use a dictionary to store the calibration configuration, the keyword is the pre-set hash value, and the value is the 3D position corresponding to the camera pixel. Traverse each pixel on the image i , using two methods, chessboard calibration and feature-based calibration, to obtain the 3D spatial position p1 and p2 corresponding to the pixel point, and convert these two points p1 and p2 into pixel coordinates b1 and b2 of the bird's-eye view, and determine which area b1 and b2 belong to. Since the calibration method of each area is determined, the corresponding relationship is determined according to the calibration method. First, according to the pixel point pixel i Calculate the hash value. Specifically, calculate the hash value using the following formula;

[0112] haxi=pix x +width*pix y ;

[0113] Where haxi is the hash value corresponding to the pixel; (pix x ,pix y ) is the pixel coordinate of the camera image; width is the width of the camera image.

[0114] If the area is calibrated by a checkerboard, p1 is used as the 3D position corresponding to the hash value; if the area is calibrated by a feature-based method, p2 is used as the 3D position corresponding to the hash value.

[0115] The obtained configuration parameters can then be used for subsequent intersection camera calibration. Subsequent camera recognition will have a corresponding recognition frame, and then a positioning pixel point will be calculated, and then the corresponding 3D position can be directly found from the configuration file. This can make it clearer, more intuitive, and more efficient to select the optimal camera fusion calibration method, saving a lot of human resource costs.

[0116] The beneficial effect of the present application is that: by automatically acquiring multiple candidate driving trajectories corresponding to vehicles in the target area under multiple camera calibration methods, and comparing the multiple candidate driving trajectories with the actual driving trajectories, and then selecting the target driving trajectory with the highest degree of similarity to the actual driving trajectory from the multiple candidate driving trajectories according to the comparison results, and finally determining that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method. Thus, the optimal camera calibration method is automatically selected according to the scene in the target area, and the accuracy of road information recognition is improved.

[0117] In one embodiment, the above step S101 may be implemented as follows:

[0118] The candidate driving trajectories of the vehicles in the target area under the checkerboard calibration method and the candidate driving trajectories of the vehicles in the target area under the feature point calibration method are obtained.

[0119] In one embodiment, the above step S102 may be implemented as the following steps A1-A3:

[0120] In step A1, a map corresponding to the target area is displayed on the bird's-eye view;

[0121] In step A2, the plurality of candidate driving trajectories and the actual driving trajectory are projected onto a map corresponding to the target area;

[0122] In step A3, the plurality of candidate driving trajectories are compared with the actual driving trajectories on a map corresponding to the target area.

[0123] In one embodiment, when the target area is an intersection, the above step A1 may be implemented as the following steps A11-A13:

[0124] In step A11, the central longitude and latitude of the intersection are used as the origin of the local coordinates, and the geodetic coordinate system corresponding to the map is converted into the local coordinate system of the target area;

[0125] In step A12, the local coordinate system of the target area is converted into the pixel coordinates of the target area on the bird's-eye view;

[0126] In step A13, a map corresponding to the target area is displayed on the bird's-eye view according to the pixel coordinates of the target area on the bird's-eye view.

[0127] In one embodiment, the above step A2 may be implemented as the following steps A21-A23:

[0128] In step A21, the longitude and latitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory are converted into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system;

[0129] In step A22, the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories;

[0130] In step A23, the plurality of candidate driving trajectories and the actual driving trajectory are projected onto a map corresponding to the target area according to the pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectory.

[0131] In one embodiment, the above step A21 may be implemented as follows:

[0132] The latitude and longitude coordinates corresponding to the multiple candidate driving trajectories and the actual driving trajectories are converted into the local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system by the following formula:

[0133] x=(lon-lon base )*Π / 180*R a *cos(lat*Π / 180);

[0134] y=(lat-lat base )*Π / 180*R b ;

[0135] Where (x, y) is the coordinate of the local coordinate system; (lon base ,lat base ) is the longitude and latitude of the center of the intersection; (lon, lat) is the longitude and latitude corresponding to the map; R a is the radius of the Earth, the distance from the center of the Earth to the equator; R b is the polar radius, the distance from the center of the Earth to the South Pole or the North Pole.

[0136] In one embodiment, the above step A22 may be implemented as follows:

[0137] The local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the multiple candidate driving trajectories and the actual driving trajectories by the following formula:

[0138] pix x =floor(x-limit x[0])*pixel2dist;

[0139] pix y =height-floor(y-limit y [0])*pixel2dist;

[0140] Among them, (pix x ,pix y ) is the pixel coordinate of the bird's-eye view; floor represents the floor rounding function; height is the height of the bird's-eye view; (x, y) is the coordinate of the local coordinate system; limit x and limit y Indicates the size of the region of interest, all of which are lists; pixel2dist indicates the number of pixels in the bird's-eye view corresponding to the target area within 1m.

[0141] In one embodiment, the method can also be implemented as follows:

[0142] The camera calibration method of the target area is adjusted to the optimal calibration method.

[0143] Figure 6 FIG. 1 is a schematic diagram of a structure of a device for determining a camera calibration method in an embodiment of the present application. Figure 6 As shown, the device comprises:

[0144] An acquisition module 601 is used to acquire multiple candidate driving trajectories corresponding to vehicles in a target area under multiple camera calibration modes;

[0145] A comparison module 602, used for comparing the plurality of candidate driving trajectories with the actual driving trajectory;

[0146] A selection module 603 is used to select a target driving trajectory with the highest similarity to the actual driving trajectory from the plurality of candidate driving trajectories according to the comparison result;

[0147] The determination module 604 is used to determine that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method.

[0148] In one embodiment, the acquisition module is further used to:

[0149] The candidate driving trajectories of the vehicles in the target area under the checkerboard calibration method and the candidate driving trajectories of the vehicles in the target area under the feature point calibration method are obtained.

[0150] In one embodiment, the comparison module includes:

[0151] A display submodule, used to display a map corresponding to the target area on the bird's-eye view;

[0152] A projection submodule, used for projecting the plurality of candidate driving trajectories and the actual driving trajectory onto a map corresponding to the target area;

[0153] The comparison submodule is used to compare the multiple candidate driving trajectories with the actual driving trajectories on the map corresponding to the target area.

[0154] In one embodiment, when the target area is an intersection, the display submodule is further used to:

[0155] The central longitude and latitude of the intersection are used as the origin of the local coordinates, and the geodetic coordinate system corresponding to the map is converted into the local coordinate system of the target area;

[0156] Then, the local coordinate system of the target area is converted into the pixel coordinates of the target area on the bird's-eye view;

[0157] The map corresponding to the target area is displayed on the bird's-eye view according to the pixel coordinates of the target area on the bird's-eye view.

[0158] In one embodiment, the projection submodule is further used to:

[0159] Converting the longitude and latitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system;

[0160] Converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories;

[0161] The plurality of candidate driving trajectories and the actual driving trajectories are projected onto a map corresponding to the target area according to pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories.

[0162] In one embodiment, converting the latitude and longitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system includes:

[0163] The latitude and longitude coordinates corresponding to the multiple candidate driving trajectories and the actual driving trajectories are converted into the local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system by the following formula:

[0164] x=(lon-lon base )*Π / 180*R a *cos(lat*Π / 180);

[0165] y=(lat-lat base)*Π / 180*R b ;

[0166] Where (x, y) is the coordinate of the local coordinate system; (lon base ,lat base ) is the longitude and latitude of the center of the intersection; (lon, lat) is the longitude and latitude corresponding to the map; R a is the radius of the Earth, the distance from the center of the Earth to the equator; R b is the polar radius, the distance from the center of the Earth to the South Pole or the North Pole.

[0167] In one embodiment, converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories comprises:

[0168] The local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the multiple candidate driving trajectories and the actual driving trajectories by the following formula:

[0169] pix x =floor(x-limit x [0])*pixel2dist;

[0170] pix y =height-floor(y-limit y [0])*pixel2dist;

[0171] Among them, (pix x ,pix y ) is the pixel coordinate of the bird's-eye view; floor represents the floor rounding function; height is the height of the bird's-eye view; (x, y) is the coordinate of the local coordinate system; limit x and limity represent the size of the region of interest, both are lists; pixel2dist represents the number of pixels in the bird's-eye view corresponding to the target area within 1m.

[0172] Figure 7 FIG. 1 is a schematic diagram of the hardware structure of a system for determining a camera calibration method in an embodiment of the present application. Figure 7 As shown, the system for determining the camera calibration method includes:

[0173] at least one processor 720; and,

[0174] A memory 704 in communication with the at least one processor 720; wherein,

[0175] The memory 704 stores instructions that can be executed by the at least one processor 720, and the instructions are executed by the at least one processor 720 to implement the method for determining the camera calibration mode recorded in any of the above embodiments.

[0176] Reference Figure 7 The camera calibration determination system 700 may include one or more of the following components: a processing component 702, a memory 704, a power component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714, and a communication component 716.

[0177] The processing component 702 generally controls the overall operation of the camera calibration determination system 700. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0178] The memory 704 is configured to store various types of data to support the operation of the camera calibration determination system 700. Examples of such data include instructions for any application or method operating on the camera calibration determination system 700, such as text, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0179] The power supply component 706 provides power to various components of the camera calibration determination system 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the camera calibration determination system 700.

[0180] The multimedia component 708 includes a screen that provides an output interface between the camera calibration mode determination system 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 may also include a front camera and / or a rear camera. When the camera calibration mode determination system 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0181] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the camera calibration determination system 700 is in an operation mode, such as an alarm mode, a recording mode, a voice recognition mode, and a voice output mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0182] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0183] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the camera calibration mode determination system 700. For example, the sensor assembly 714 may include a sound sensor. In addition, the sensor assembly 714 may detect the open / closed state of the camera calibration mode determination system 700, the relative positioning of components, such as the display and keypad of the camera calibration mode determination system 700, the sensor assembly 714 may also detect the operating state of the camera calibration mode determination system 700 or a component of the camera calibration mode determination system 700, the camera calibration mode determination system 700 orientation or acceleration / deceleration and the temperature change of the camera calibration mode determination system 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0184] The communication component 716 is configured to enable the camera calibration mode determination system 700 to provide the ability to communicate with other devices and cloud platforms in a wired or wireless manner. The camera calibration mode determination system 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0185] In an exemplary embodiment, the camera calibration method determination system 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the camera calibration method determination method described in any of the above embodiments.

[0186] The present application also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor corresponding to the camera calibration method determination system, the camera calibration method determination system can implement the camera calibration method determination method recorded in any of the above embodiments.

[0187] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0188] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0191] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining a camera calibration method, characterized in that: include: Obtain multiple candidate driving trajectories corresponding to vehicles in the target area under multiple camera calibration methods; Comparing the plurality of candidate driving trajectories with the actual driving trajectories; Selecting a target driving trajectory with the highest degree of similarity to the actual driving trajectory from the plurality of candidate driving trajectories according to the comparison result; Determine that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method.

2. The method according to claim 1, characterized in that The method of obtaining multiple candidate driving trajectories corresponding to the vehicles in the target area under multiple camera calibration methods includes: The candidate driving trajectories of the vehicles in the target area under the checkerboard calibration method and the candidate driving trajectories of the vehicles in the target area under the feature point calibration method are obtained.

3. The method according to claim 1, characterized in that The comparing the plurality of candidate driving trajectories with the actual driving trajectories includes: Display the map corresponding to the target area on the bird's-eye view; Projecting the plurality of candidate driving trajectories and the actual driving trajectory onto a map corresponding to the target area; The plurality of candidate driving trajectories are compared with the actual driving trajectories on a map corresponding to the target area.

4. The method according to claim 1, characterized in that When the target area is an intersection, displaying a map corresponding to the target area on the bird's-eye view includes: The central longitude and latitude of the intersection are used as the origin of the local coordinates, and the geodetic coordinate system corresponding to the map is converted into the local coordinate system of the target area; Then, the local coordinate system of the target area is converted into the pixel coordinates of the target area on the bird's-eye view; The map corresponding to the target area is displayed on the bird's-eye view according to the pixel coordinates of the target area on the bird's-eye view.

5. The method according to claim 4, characterized in that The projecting the plurality of candidate driving trajectories and the actual driving trajectory onto a map corresponding to the target area includes: Converting the longitude and latitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectory into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectory in the local coordinate system; Converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories; The plurality of candidate driving trajectories and the actual driving trajectories are projected onto a map corresponding to the target area according to pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories.

6. The method according to claim 5, characterized in that The step of converting the latitude and longitude coordinates corresponding to the plurality of candidate driving trajectories and the actual driving trajectories into local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system includes: The latitude and longitude coordinates corresponding to the multiple candidate driving trajectories and the actual driving trajectories are converted into the local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system by the following formula: x=(long-long base )*Π / 180*R a *cos(lat*Π / 180); y=(lat-lat base )*Π / 180*R b ; Where (x, y) is the coordinate of the local coordinate system; (lon base ,lat base ) is the longitude and latitude of the center of the intersection; (lon, lat) is the longitude and latitude corresponding to the map; R a is the radius of the Earth, the distance from the center of the Earth to the equator; R b is the polar radius, the distance from the center of the Earth to the South Pole or the North Pole.

7. The method according to claim 5, characterized in that The converting the local coordinates of the plurality of candidate driving trajectories and the actual driving trajectories in the local coordinate system into pixel coordinates of the plurality of candidate driving trajectories and the actual driving trajectories comprises: The local coordinates of the multiple candidate driving trajectories and the actual driving trajectories in the local coordinate system are converted into pixel coordinates of the multiple candidate driving trajectories and the actual driving trajectories by the following formula: pix x =floor(x-limit x [0])*pixel2dist; pix y =height-floor(y-limity[0])*plxel2dlst; Among them, (pix x ,plx y ) is the pixel coordinate of the bird's-eye view; floor represents the floor function; height is the height of the bird's-eye view; (x, y) is the coordinate of the local coordinate system; limit x and limit y Indicates the size of the region of interest, all of which are lists; pixel2dist indicates the number of pixels in the bird's-eye view corresponding to the target area within 1m.

8. A device for determining a camera calibration method, characterized in that: include: An acquisition module, used to acquire multiple candidate driving trajectories corresponding to vehicles in a target area under multiple camera calibration methods; A comparison module, used for comparing the plurality of candidate driving trajectories with the actual driving trajectories; A selection module, configured to select a target driving trajectory having the highest degree of similarity to the actual driving trajectory from the plurality of candidate driving trajectories according to the comparison result; The determination module is used to determine that the camera calibration method corresponding to the target driving trajectory is the optimal calibration method.

9. A system for determining a camera calibration method, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to implement the method for determining the camera calibration mode according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor corresponding to the camera calibration method determination system, the camera calibration method determination system can implement the camera calibration method determination method as described in any one of claims 1 to 7.

Images