A camera calibration system and calibration method based on video black box

Abstract

The present invention relates to a camera calibration system and calibration method based on a video black box, comprising: a host computer, a dark box, and a camera to be calibrated, a convex lens, and a display arranged in the dark box; the display is fixedly installed in the dark box and connected to an external video input device, and is used to play a real road scene from the camera's perspective; the convex lens is arranged between the camera to be calibrated and the display, and is used to adjust the focal length of the camera to be calibrated; the camera to be calibrated is electrically connected to the host computer, and is used to shoot the real road scene played on the display and upload it to the host computer; the host computer is used to analyze the road scene shot by the camera to be calibrated, identify and calculate the relative distance and relative angle between a target object and a reference object in the road scene shot by the camera to be calibrated, and output the relative position relationship between the camera to be calibrated, the convex lens, and the display according to the difference between the relative distance and the relative angle and the true value.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving simulation testing, and in particular to a camera calibration system and calibration method based on a video black box, and specifically to the application of distortion-free monocular vehicle-mounted camera hardware-in-the-loop simulation. Background Art

[0002] To meet the development requirements of autonomous vehicles, simulation testing has become an essential step. However, the results of SiL (Software-in-the-Loop) / MiL (Model-in-the-Loop) simulation tests are often unconvincing and can only provide preliminary verification of early-stage algorithms and code.

[0003] Therefore, HiL (Hardware-in-the-Loop) testing plays an essential and irreplaceable role in simulation testing, enabling real-world controller and sensor components to be integrated into the loop, enhancing the realism of simulation testing. In the current wave of autonomous driving, core technologies are primarily held by Tier 1 suppliers. Tier 1 suppliers must iteratively update and test real-world controllers and sensors, adapting them to different vehicle models before selling them to OEMs. OEMs struggle to develop their own core technologies, relying on mature black-box hardware from Tier 1 suppliers, often without even access to communication protocols. This forces them to conduct their own research and development through HiL testing. HiL encompasses control-in-the-loop, perception-in-the-loop, and execution-in-the-loop. Within perception-in-the-loop, camera sensors are the most widely used perception sensors due to their mature technology and low cost. These hardware-in-the-loop sensors often utilize camera darkroom cameras, whose calibration techniques vary widely and rely on supplier calibration tools. During the autonomous driving development process, real-world camera calibration is time-consuming and laborious. Currently, autonomous driving functions above Level 3 require trillions of kilometers of simulation test data for reliable implementation. According to the current requirements of autonomous driving simulation testing, camera hardware-in-the-loop simulation testing is particularly important, and calibration is the prerequisite and necessary condition for simulation. Summary of the Invention

[0004] In response to the technical problems existing in the prior art, the present invention provides a camera calibration system and calibration method based on a video black box, which can effectively solve the technical problems of in-loop calibration of camera hardware in the HiL closed-loop simulation test link of autonomous driving. The calibrated black box system can also perform open-loop testing of the camera for autonomous driving.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] On the one hand, the present invention provides a camera calibration system based on a video black box, comprising: a host computer, a black box, and a camera to be calibrated, a convex lens, and a display arranged in the black box;

[0007] The display is fixedly installed in the dark box and connected to an external video input device for playing the real road scene from the camera's perspective;

[0008] The convex lens is arranged between the camera to be calibrated and the display, and is used to adjust the focal length of the camera to be calibrated;

[0009] The camera to be calibrated is electrically connected to the host computer and is used to shoot the real road scene played on the display and upload it to the host computer;

[0010] The host computer is used to analyze the road scene photographed by the camera to be calibrated, identify and calculate the relative distance and relative angle between the target object and the reference object in the road scene photographed by the camera to be calibrated, and output the relative position relationship between the camera to be calibrated, the convex lens and the display based on the difference between the relative distance, relative angle and the true value.

[0011] Furthermore, the system also includes a position adjustment mechanism, which is fixedly connected to the camera to be calibrated and the dark box respectively, and is used to adjust the relative position and angle of the camera to be calibrated, the convex lens and the display.

[0012] Furthermore, the position adjustment mechanism includes a slide rail installed along the axis of the convex lens and the display, and a three-degree-of-freedom camera bracket slidably connected to the slide rail for installing the camera to be calibrated.

[0013] Furthermore, the convex lens is slidably connected to the slide rail via a convex lens fixing bracket.

[0014] Furthermore, the display plays the real road scene from the camera's perspective, including camera data collected from a real vehicle or a simulated road scene produced by scene simulation software; in the simulated road scene, the virtual camera installation position and performance parameters are consistent with the tested camera's installation position on the real vehicle, and can output the true value target data recognized by the camera in real time.

[0015] On the other hand, the present invention provides a camera calibration method based on a video black box, which is implemented based on the above-mentioned camera calibration system and includes the following steps:

[0016] Coarsely adjust the positions of the calibration camera, convex lens, and display so that their centers are on the same straight line;

[0017] Play the road scene from the camera's perspective on the display;

[0018] The camera to be calibrated captures the real road scene played on the display and uploads it to the host computer;

[0019] The host computer analyzes the road scene photographed by the camera to be calibrated, identifies and calculates the relative distance and relative angle between the target object and the reference object in the road scene photographed by the camera to be calibrated;

[0020] The host computer outputs the relative position relationship between the camera to be calibrated, the convex lens and the display according to the difference between the relative distance, the relative angle and the true value, and adjusts the position of the camera to be calibrated and the convex lens according to the relative position relationship.

[0021] The beneficial effect of the present invention is that the present invention can effectively solve the technical problem of in-loop calibration of camera hardware in the HiL closed-loop simulation test link of autonomous driving, and its calibrated darkroom system can also perform open-loop testing of autonomous driving on the camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the structure of a camera calibration system based on a video black box provided by an embodiment of the present invention;

[0023] Figure 2 A schematic flow chart of a camera calibration method based on a video black box is provided in an embodiment of the present invention.

[0024] In the accompanying drawings, the components represented by the reference numerals are as follows:

[0025] 1. Host computer, 2. Dark box, 3. Camera to be calibrated, 4. Convex lens, 5. Display, 6. Slide rail, 7. Three-degree-of-freedom camera bracket, 8. Convex lens fixing bracket, 9. External video input device. DETAILED DESCRIPTION

[0026] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0027] Figure 1 The following is a schematic diagram of the camera calibration system structure based on a video black box provided by an embodiment of the present invention. Figure 1 As shown, the camera calibration system includes: a host computer 1, a dark box 2, and a camera to be calibrated 3, a convex lens 4, and a display 5 arranged in the dark box 1;

[0028] The display 5 is fixedly installed in the dark box and connected to the external video input device 9 for playing the real road scene from the camera's perspective;

[0029] The convex lens 4 is arranged between the camera to be calibrated 3 and the display 5, and is used to adjust the focal length of the camera to be calibrated;

[0030] The camera to be calibrated 3 is electrically connected to the host computer 1, and is used to shoot the real road scene played on the display 5 and upload it to the host computer 1;

[0031] The host computer 1 is used to analyze the road scene photographed by the camera to be calibrated, identify and calculate the relative distance and relative angle between the target object and the reference object in the road scene photographed by the camera to be calibrated, and output the relative position relationship between the camera to be calibrated, the convex lens and the display based on the difference between the relative distance, relative angle and the true value.

[0032] Furthermore, the system also includes a position adjustment mechanism, which is fixedly connected to the camera to be calibrated and the dark box respectively, and is used to adjust the relative positions and angles among the camera to be calibrated, the convex lens and the display.

[0033] Furthermore, the position adjustment mechanism includes a slide rail 6 installed along the axis of the convex lens and the display, and a three-degree-of-freedom camera bracket 7 slidably connected to the slide rail for mounting the camera to be calibrated.

[0034] Furthermore, the convex lens is slidably connected to the slide rail via a convex lens fixing bracket 8 .

[0035] Furthermore, the display plays the real road scene from the camera's perspective, including camera data collected from a real vehicle or a simulated road scene produced by scene simulation software; in the simulated road scene, the virtual camera installation position and performance parameters are consistent with the tested camera's installation position on the real vehicle, and can output the true value target data recognized by the camera in real time.

[0036] On the basis of the above embodiment, the embodiment of the present invention further provides a camera calibration method based on a video black box, such as Figure 2 As shown, the following steps are included:

[0037] Coarsely adjust the positions of the calibration camera, convex lens, and display so that their centers are on the same straight line;

[0038] Play the road scene from the camera's perspective on the display;

[0039] The camera to be calibrated captures the real road scene played on the display and uploads it to the host computer;

[0040] The host computer analyzes the road scene photographed by the camera to be calibrated, identifies and calculates the relative distance and relative angle between the target object and the reference object in the road scene photographed by the camera to be calibrated;

[0041] The host computer outputs the relative position relationship between the camera to be calibrated, the convex lens and the display according to the difference between the relative distance, the relative angle and the true value, and adjusts the position of the camera to be calibrated and the convex lens according to the relative position relationship.

[0042] Specifically, the calibration is performed according to the following process:

[0043] 1. Camera dark box environment preparation: First, prepare the entire camera dark box environment system (as shown in the attached Figure 1 ) is debugged, and the interior includes a three-degree-of-freedom camera bracket, a convex lens and its fixing bracket, a display, a slide rail, etc. The interior is painted black to ensure that the entire dark box environment is free of light interference.

[0044] 2. Camera debugging preparation: prepare a normal camera, debug it in an external environment first, supply the required voltage to the camera, debug its communication interface, host computer visualization interface, and data display interface to facilitate subsequent darkroom calibration and data analysis.

[0045] 3. Display-borne scene debugging: The display plays the road scene from the camera's perspective, and its road scene coverage must include the requirements for calibration. There are two ways to obtain the display scene: 1. Camera data collected from the actual vehicle, including timestamps, target data, images, videos, etc.; 2. Simulation using mainstream scene simulation software. The virtual camera's installation position and performance parameters are consistent with the tested camera's installation position on the actual vehicle, and the real-value target data recognized by the camera can be output in real time.

[0046] 4. Camera installation and fixation: Use the three-degree-of-freedom camera bracket to fix the camera on the slide rail, adjust the camera installation height up and down, and use a laser pointer to keep the center of the camera, the center of the convex lens and the center of the monitor on the same horizontal line.

[0047] 5. Lane Marking Calibration: Calibrate and identify lane marking distances. Select a scenario where the vehicle is in the center of the lane, deviating 0.5 meters to the left, and 0.5 meters to the right. Output the distances to the left and right lane markings through the camera host interface. Determine whether the error with the distance set on the display is within the threshold. If the difference is too large, adjust the distances of the slide rail and camera bracket.

[0048] 6. Relative distance calibration: Select the measured vehicle and the target vehicle in the middle of the lane, and the target vehicle is 3m, 5m, 10m, 15m, 20m, 30m, 50m away from the measured vehicle. The closer the distance, the higher the calibration accuracy. The measured vehicle and the target vehicle keep running at the same speed. The target distance recognition result is output through the host computer interface to determine whether the error with the set distance is within the threshold. If the difference is too large, adjust the front and rear distances of the slide rail, camera bracket, and convex lens.

[0049] 7. Relative angle calibration: Select at least three lanes in the scene, with the vehicle under test in the middle lane, and the target vehicle in the left lane, the middle lane with a deviation of 0.5m to the left, the middle lane with a deviation of 0.5m to the right, and the right lane, respectively. The longitudinal distance is 3m, 5m, 10m, 15m, 20m, 30m, 50m, etc. from the vehicle under test. The closer the distance, the higher the calibration accuracy. The vehicle under test and the target vehicle keep running at the same speed. The target relative angle recognition result is output through the camera host computer interface to determine whether the angle error with the true value is within the threshold. If the difference is too large, adjust the left and right distance of the slide rail and camera bracket.

[0050] 8. The calibrated camera darkroom system can be used to conduct open-loop and HiL closed-loop simulation testing of the camera for autonomous driving, which is conducive to the faster implementation of autonomous driving.

[0051] The present invention can effectively solve the technical problem of in-loop calibration of camera hardware in the HiL closed-loop simulation test of autonomous driving, and its calibrated darkroom system can also perform open-loop testing of autonomous driving on the camera.

[0052] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0053] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A camera calibration system based on a video black box, characterized in that: include: A host computer, a dark box, and a camera to be calibrated, a convex lens, and a display arranged in the dark box; The display is fixedly installed in the dark box and connected to an external video input device for playing the real road scene from the camera's perspective; The convex lens is arranged between the camera to be calibrated and the display, and is used to adjust the focal length of the camera to be calibrated; The camera to be calibrated is electrically connected to the host computer and is used to shoot the real road scene played on the display and upload it to the host computer; The host computer is used to analyze the road scene photographed by the camera to be calibrated, identify and calculate the relative distance and relative angle between the target object and the reference object in the road scene photographed by the camera to be calibrated, and output the relative position relationship between the camera to be calibrated, the convex lens and the display based on the difference between the relative distance, relative angle and the true value.

2. The system according to claim 1, wherein: It also includes a position adjustment mechanism, which is fixedly connected to the camera to be calibrated and the dark box respectively, and is used to adjust the relative position and angle of the camera to be calibrated, the convex lens and the display.

3. The system according to claim 2, characterized in that The position adjustment mechanism includes a slide rail installed along the axis direction of the convex lens and the display, and a three-degree-of-freedom camera bracket slidably connected to the slide rail for installing the camera to be calibrated.

4. The system according to claim 3, characterized in that The convex lens is slidably connected to the slide rail via a convex lens fixing bracket.

5. The system according to claim 1, wherein: The display plays the real road scene from the camera's perspective, including camera data collected from a real vehicle or a simulated road scene produced by scene simulation software; in the simulated road scene, the virtual camera's installation position and performance parameters are consistent with the tested camera's installation position on the real vehicle, and can output the true value target data recognized by the camera in real time.

6. A camera calibration method based on a video black box, the method being implemented based on the camera calibration system according to any one of claims 1 to 5, characterized in that: The following steps are involved: Coarsely adjust the positions of the calibration camera, convex lens, and display so that their centers are on the same straight line; Play the road scene from the camera's perspective on the display; The camera to be calibrated captures the real road scene played on the display and uploads it to the host computer; The host computer analyzes the road scene photographed by the camera to be calibrated, identifies and calculates the relative distance and relative angle between the target object and the reference object in the road scene photographed by the camera to be calibrated; The host computer outputs the relative position relationship between the camera to be calibrated, the convex lens and the display according to the difference between the relative distance, the relative angle and the true value, and adjusts the position of the camera to be calibrated and the convex lens according to the relative position relationship.

Images