A positioning-based camera calibration effect verification method and system, and a storage medium

Abstract

The present application relates to a kind of camera calibration effect verification method based on positioning, system and storage medium, the method includes R1: in the front of preset position setting target point, and the vehicle to be measured is parked in the preset position with target point;R2: based on the image data of the target point of vehicle-mounted camera of the vehicle to be measured, based on the positioning device of the vehicle to be measured, the positioning data of vehicle and target point are acquired, the image data of the target point of the same time point is extracted Frame image and the positioning data of the vehicle and target point positioning data, output first group of image and positioning information;R3: the vehicle to be measured is moved, the image data of the target point after moving and the positioning data of the vehicle and target point after moving are acquired.The present application not only solves the problem that the constraint of vehicle posture is harsh when result is verified, but also solves the harsh requirement that vehicle and target object are in the same horizontal plane when result is verified.

Description

Technical Field

[0001] This invention relates to the field of camera calibration effect technology, and in particular to a positioning-based camera calibration effect verification method, system, and storage medium. Background Technology

[0002] The use of camera sensors is indispensable in autonomous vehicles. Using cameras requires establishing the correspondence between objects in the three-dimensional world and their coordinates on the image plane. The intrinsic parameter matrix, distortion matrix, and extrinsic parameter matrix obtained from camera calibration represent these correspondences. After obtaining this set of relationships, how effective is it in practice, and how can it be quickly verified?

[0003] In existing technologies, the calibration result is verified by parking the autonomous vehicle at a fixed location to check the target point. The translation of the target point from the vehicle coordinate system in each direction is measured and compared with the translation output by the perception system. If the difference between the two is greater than a certain empirical value, the calibration result is considered unqualified; otherwise, it is qualified. This requires fixing the vehicle at a fixed position (e.g., 90°). This condition is quite stringent, as there is usually human error in parking the vehicle, which has a certain impact on the verification result. When verifying the result, it is often necessary to ensure that the vehicle and the target object are on the same horizontal plane to reduce the error in the rotation direction, and only the result in the horizontal direction is verified. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the present invention provides a positioning-based camera calibration effect verification method, system and storage medium, which not only solves the problem of strict constraints on vehicle attitude during result verification, but also solves the strict requirement that the vehicle and the target object be on the same horizontal plane during result verification.

[0005] To achieve the above and other related objectives, the present invention provides the following technical solution:

[0006] A location-based camera calibration effect verification method, the method comprising:

[0007] R1: Set a target point in front of the preset position and park the vehicle to be tested at the preset position with the target point;

[0008] R2: Based on the on-board camera of the vehicle under test, image data of the target point is collected; based on the positioning device of the vehicle under test, positioning data of the vehicle and the target point are obtained; one frame of image data of the target point at the same time point and one frame of positioning data of the vehicle and the target point are extracted; and the first set of images and positioning information are output.

[0009] R3: Move the vehicle under test, collect image data of the target point after the movement and obtain positioning data of the vehicle and target point after the movement, extract one frame of image data of the target point after the movement and one frame of positioning data of the vehicle and target point at the same time point, and output the second set of images and positioning information.

[0010] R4: Based on the first set of images and positioning information and the second set of images and positioning information, verify the world coordinate error based on pixel coordinates and verify the pixel coordinate error based on world coordinates.

[0011] Furthermore, in step R4, the verification of world coordinate error based on pixel coordinates includes:

[0012] R411: Based on the first set of images and positioning information and the second set of images and positioning information, extract the pixel coordinates of the target points in the images;

[0013] R412: Substitute the pixel coordinates of the target point into the camera parameters and output the corresponding world coordinates; R413: Obtain the reference world coordinates based on the vehicle positioning data and the latitude and longitude information of the target point;

[0014] R414: Calculate the difference between the world coordinates obtained in step R412 and the world coordinates obtained in step R413.

[0015] Furthermore, the vehicle positioning data includes latitude, longitude, and heading angle.

[0016] Furthermore, in step R4, verifying the pixel coordinate error based on world coordinates includes:

[0017] R421: Based on the first set of images and positioning information and the second set of images and positioning information, the reference world coordinates are obtained according to the vehicle positioning information and the latitude and longitude information of the target point;

[0018] R422: Substitute the reference world coordinates into the camera parameters to obtain the corresponding pixel coordinates;

[0019] R423: Extract the pixel coordinates of the target point in the image and use this set of pixel coordinates as the truth value;

[0020] R424: Calculate the difference between the pixel coordinates obtained in step R422 and the pixel coordinates obtained in step R423.

[0021] Furthermore, the world coordinate error is verified based on the pixel coordinates to obtain a set of differences. These differences are evaluated, and a preset calibration error value is set. If these differences are greater than the calibration error value, the camera calibration result is considered to have a large error.

[0022] Furthermore, the pixel coordinate error is verified using world coordinates to obtain a set of differences. The variance of this set of differences is calculated, and a preset variance value is set. If the variance of this set of differences is less than the preset variance value, the camera calibration result is considered unstable. Further, the vehicle positioning data and the latitude and longitude information of the target point are converted into Gaussian coordinates. The target point and the vehicle will be in the same coordinate system. Then, the coordinates of the target point in the vehicle coordinate system are obtained based on the vehicle's heading information. This set of coordinates is the reference world coordinates.

[0023] To achieve the above and other related objectives, the present invention also provides a positioning-based camera calibration effect verification system, including a computer device programmed or configured to perform the steps of any of the positioning-based camera calibration effect verification methods described above.

[0024] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium storing a computer program programmed or configured to perform any of the positioning-based camera calibration effect verification methods described herein.

[0025] The present invention has the following positive effects:

[0026] 1. This invention simplifies the difficulty of obtaining world coordinates by using positioning information, and solves the problem of stringent constraints on vehicle attitude during result verification.

[0027] 2. This invention can evaluate the calibration effect in a standardized way by using quantified difference data, thus solving the stringent requirement that the vehicle and the target object be on the same horizontal plane when verifying the results.

[0028] 3. The verification process of this invention is simple and can be performed quickly, reducing verification costs and thus improving the accuracy of camera calibration results. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0030] Figure 2 This is a schematic diagram showing the position of the vehicle and the target point in this invention;

[0031] Figure 3 This is a schematic diagram of the process for verifying world coordinate errors based on pixel coordinates in this invention;

[0032] Figure 4 This is a schematic diagram of the process for verifying pixel coordinate errors based on world coordinates according to the present invention. Detailed Implementation

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Example 1: As Figure 1 or Figure 2 As shown, a positioning-based camera calibration effect verification method is provided, the method comprising:

[0035] R1: Set a target point in front of the preset position and park the vehicle to be tested at the preset position with the target point;

[0036] R2: Based on the on-board camera of the vehicle under test, image data of the target point is collected; based on the positioning device of the vehicle under test, positioning data of the vehicle and the target point are obtained; one frame of image data of the target point at the same time point and one frame of positioning data of the vehicle and the target point are extracted; and the first set of images and positioning information are output.

[0037] R3: Move the vehicle under test, collect image data of the target point after the movement and obtain positioning data of the vehicle and target point after the movement, extract one frame of image data of the target point after the movement and one frame of positioning data of the vehicle and target point at the same time point, and output the second set of images and positioning information.

[0038] R4: Based on the first set of images and positioning information and the second set of images and positioning information, verify the world coordinate error based on pixel coordinates and verify the pixel coordinate error based on world coordinates.

[0039] In this embodiment, as Figure 3 As shown, in step R4, verifying the world coordinate error based on pixel coordinates includes:

[0040] R411: Based on the first set of images and positioning information and the second set of images and positioning information, extract the pixel coordinates of the target points in the images;

[0041] R412: Substitute the pixel coordinates of the target point into the camera parameters and output the corresponding world coordinates;

[0042] R413: Based on the vehicle positioning data and the latitude and longitude of the target point, obtain the reference world coordinates;

[0043] R414: Calculate the difference between the world coordinates obtained in step R412 and the world coordinates obtained in step R413.

[0044] In this embodiment, the vehicle positioning data includes latitude, longitude, and heading angle.

[0045] In this embodiment, as Figure 4 As shown, in step R4, verifying the pixel coordinate error based on world coordinates includes:

[0046] R421: Based on the first set of images and positioning information and the second set of images and positioning information, the reference world coordinates are obtained according to the vehicle positioning information and the latitude and longitude information of the target point;

[0047] R422: Substitute the reference world coordinates into the camera parameters to obtain the corresponding pixel coordinates;

[0048] R423: Extract the pixel coordinates of the target point in the image and use this set of pixel coordinates as the truth value;

[0049] R424: Calculate the difference between the pixel coordinates obtained in step R422 and the pixel coordinates obtained in step R423.

[0050] In this embodiment, the world coordinate error is verified based on the pixel coordinates to obtain a set of differences. These differences are evaluated, and a preset calibration error value is set. If these differences are greater than the calibration error value, the camera calibration result is considered to have a large error.

[0051] In this embodiment, the pixel coordinate error is verified based on world coordinates to obtain a set of differences. The variance of this set of differences is calculated, and a preset variance value is set. If the variance of this set of differences is less than the preset variance value, the camera calibration result is considered unstable.

[0052] In this embodiment, the vehicle positioning data and the latitude and longitude information of the target point are converted into Gaussian coordinates. The target point and the vehicle will be in the same coordinate system. Then, the coordinates of the target point in the vehicle coordinate system are obtained according to the vehicle's heading information. This set of coordinates is the reference world coordinates.

[0053] To achieve the above and other related objectives, the present invention also provides a positioning-based camera calibration effect verification system, including a computer device programmed or configured to perform the steps of any of the positioning-based camera calibration effect verification methods described above.

[0054] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium storing a computer program programmed or configured to perform any of the positioning-based camera calibration effect verification methods described herein.

[0055] Example 2: Based on the positioning-based camera calibration effect verification method, system and storage medium of Example 1, the present invention will be further described below.

[0056] The software environment of this invention uses ROS as a communication middleware to provide the vehicle sensor data required for the solution. Specifically, it includes vehicle latitude and longitude information and heading positioning information, as well as the raw data information of the camera that needs to be calibrated. The GUI interface is implemented through QT to implement the overall process, including obtaining the pixel coordinates of the target in the image.

[0057] The camera data type is converted to a Qt image type (using cv_bridge to convert ROS format to OpenCV format, and then from OpenCV to QImage format), and then transmitted to the Qt Laber control for real-time display to form a video.

[0058] 1) Synchronize the camera topic and location topic in time. Once the time is synchronized, run the user data collection. Click the button to collect data.

[0059] 2) Switch the interface and display the captured image data in the pixel coordinate acquisition window. The pixel coordinate acquisition window is mainly implemented through QT mouse events and QT label space to display the image. Specifically, the captured image is displayed on the Label control of the window, the label coordinates are set to the upper left corner so that the window coordinates are the same as the origin of the label control coordinates, the label size is set to the image size, and a scroll bar control is added to the window to accommodate situations where the image exceeds the window size.

[0060] 3) Begin listening for right-click activity. Each right-click returns the current mouse coordinates within the window, ultimately returning pixel coordinates. Select the pixel coordinates of the target point by right-clicking; this set of pixel coordinates represents the true values. Upload the camera calibration parameter results. (Generally, calibration parameters are stored in a YAML file. The program parses the YAML file to obtain the parameter matrix; other formats are also acceptable.) Convert the collected vehicle latitude and longitude information and the target point latitude and longitude information into Gaussian coordinates. The target point will now be in the same coordinate system. Using the vehicle's heading information, the target point's coordinates in the vehicle coordinate system can be calculated; this set of coordinates represents the true world coordinates. Substitute the pixel coordinates obtained from mouse clicks and the camera parameters into the corresponding formulas to obtain a set of world coordinates. Compare these with the true world coordinates to obtain a set of differences. Evaluate this set of differences; for example, if the maximum value in this set is greater than 10cm, the calibration error is considered large. Alternatively, calculate the average and variance of the differences; in short, evaluate according to actual needs. Substitute the true values ​​of the world coordinates obtained through the positioning data into the corresponding values ​​to obtain a set of pixel coordinates. Subtract these values ​​from the true values ​​of the pixel coordinates to obtain a set of difference values. Evaluate these difference values ​​in the same way as above.

[0061] In summary, this invention not only solves the problem of stringent constraints on vehicle attitude during result verification, but also addresses the stringent requirement that the vehicle and the target object be on the same horizontal plane during result verification.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for verifying camera calibration results based on positioning, characterized in that, The method includes: R1. Set a target point in front of the preset location and park the vehicle to be tested at the preset location with the target point; R2. Based on the vehicle-mounted camera of the vehicle under test, image data of the target point is collected. Based on the positioning device of the vehicle under test, positioning data of the vehicle and the target point are obtained. One frame of the image data of the target point at the same time point and one frame of the positioning data of the vehicle and the target point are extracted. The first set of images and positioning information are output. R3. Move the vehicle under test, collect image data of the target point after the movement, and obtain positioning data of the vehicle and target point after the movement. Extract one frame of image data of the target point after the movement and one frame of positioning data of the vehicle and target point at the same time point, and output the second set of images and positioning information. R4. Based on the first set of images and positioning information and the second set of images and positioning information, verify the world coordinate error according to the pixel coordinates and verify the pixel coordinate error according to the world coordinates; In step R4, verifying the world coordinate error based on pixel coordinates includes: R411. Based on the first set of images and positioning information and the second set of images and positioning information, extract the pixel coordinates of the target points in the images; R412. Substitute the pixel coordinates of the target point into the camera parameters and output the corresponding world coordinates; R413. Based on the vehicle positioning data and the latitude and longitude of the target point, obtain the reference world coordinates; R414. Calculate the difference between the world coordinates obtained in step R412 and the world coordinates obtained in step R413; In step R4, verifying pixel coordinate errors based on world coordinates includes: R421. Based on the first set of images and positioning information and the second set of images and positioning information, and according to the vehicle positioning information and the latitude and longitude information of the target point, the reference world coordinates are obtained; R422. Substitute the reference world coordinates into the camera parameters to obtain the corresponding pixel coordinates; R423. Extract the pixel coordinates of the target point in the image and use this set of pixel coordinates as the truth value; R424. Calculate the difference between the pixel coordinates obtained in step R422 and the pixel coordinates obtained in step R423; The world coordinate error is verified based on pixel coordinates, and a set of differences is obtained. These differences are evaluated, and a preset calibration error value is set. If these differences are greater than the calibration error value, the camera calibration result is considered to have a large error. The pixel coordinate error is verified based on world coordinates, and a set of differences is obtained. The variance of these differences is calculated, and a preset variance value is set. If the variance of these differences is less than the preset variance value, the camera calibration result is considered to be unstable.

2. The camera calibration effect verification method based on positioning according to claim 1, characterized in that: The vehicle positioning data includes latitude, longitude, and heading angle.

3. The camera calibration effect verification method based on positioning according to claim 1, characterized in that: The vehicle positioning data and the latitude and longitude information of the target point are converted into Gaussian coordinates. The target point and the vehicle will be in the same coordinate system. Then, the coordinates of the target point in the vehicle coordinate system are obtained according to the vehicle's heading information. This set of coordinates is the reference world coordinates.

4. A positioning-based camera calibration effect verification system, comprising computer equipment, characterized in that, The computer device is programmed or configured to perform the steps of the positioning-based camera calibration effect verification method according to any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is programmed or configured to perform the positioning-based camera calibration effect verification method according to any one of claims 1 to 3.

Images