A fully automatic calibration platform, calibration control system and calibration method for a lidar and a fish-eye lens

Abstract

A fully automatic calibration platform, calibration control system and calibration method for a lidar and a fish-eye lens, belonging to the field of three-dimensional scanning devices. To solve the problem of poor calibration effect caused by the existing calibration methods. The transverse support shaft of the present invention is installed on the bracket and can rotate. A through hole is provided on the transverse support shaft. The longitudinal support shaft is inserted into the through hole of the transverse support shaft and can rotate. The heavy-duty platform is fixedly installed at the top of the longitudinal support shaft via a driven bevel gear; two driving bevel gears are respectively sleeved at both ends of the transverse support shaft, and each driving bevel gear meshes with the driven bevel gear. When the two driving bevel gears rotate in the same direction and at the same speed, the force directions on both sides of the driven bevel gear are the same and the pitching motion of the heavy-duty platform is realized; when the two driving bevel gears rotate in the opposite direction and at the same speed, the force directions on both sides of the driven bevel gear are opposite and the yaw motion of the heavy-duty platform is realized; the pitch-yaw support mechanism is arranged on the translation mechanism and can realize the translation of the heavy-duty platform.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional scanning devices, and particularly relates to a full-automatic calibration platform, a calibration control system and a calibration method for a lidar and a fisheye lens. Background Art

[0002] At present, the calibration of a lidar and a fisheye lens depends on calibration through a calibration board, and calibration requires the lidar and the lens to collect data of a calibration target at different positions simultaneously for multiple times. There are currently two main calibration methods. One is to perform calibration in a room filled with black and white grids, and it is necessary for manpower to move the calibration target continuously along with the data collection of the lidar and the lens. This method occupies a large space and has a low space utilization rate. Moreover, due to the complex calibration process and low repeatability of manual placement of the calibration, the calibration effect is greatly affected by operations, and it is difficult to achieve the standardization and automation of calibration equipment. The other is to use a six-axis robotic arm to carry a device at its end to continuously move a fixed calibration target for data collection and calibration. However, when the lens used is a fisheye lens, to avoid the six-axis robotic arm blocking the field of view of the fisheye lens during movement, the working dead zone of the six-axis robotic arm is relatively large and the operating efficiency is low. In addition, since the robotic arm may interfere with the wires of the calibration equipment and cause the calibration process to be interrupted, there are often risks such as equipment damage and circuit short circuits.

[0003] In addition, in the current calibration methods for lidars and fisheye lenses, the process of continuously moving the calibration target is carried out arbitrarily, which may cause the calibration board to appear at any position in the field of view of the lens during the calibration process, and cannot effectively cover the distortion-sensitive area, thus affecting the calibration effect to a certain extent. Summary of the Invention

[0004] In view of this, the present invention provides a full-automatic calibration platform, a calibration control system and a calibration method for a lidar and a fisheye lens, which can realize the three-degree-of-freedom movement of the calibration equipment to achieve data collection and accurate calibration.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] A fully automatic calibration platform for a lidar and a fish-eye lens, comprising a heavy-duty platform, a pitch-yaw support mechanism and a translation mechanism. The pitch-yaw support mechanism includes a bracket, a driven bevel gear, a driving bevel gear, a transverse support shaft and a longitudinal support shaft. The transverse support shaft is horizontally installed on the bracket and can rotate. A through hole is provided on the transverse support shaft. The longitudinal support shaft is inserted into the through hole of the transverse support shaft and can rotate. The driven bevel gear is fixedly installed at the top of the longitudinal support shaft. The heavy-duty platform is arranged on the upper surface of the driven bevel gear. There are two driving bevel gears. The two driving bevel gears are respectively sleeved at both ends of the transverse support shaft and can rotate. Each driving bevel gear meshes with the driven bevel gear. When the two driving bevel gears rotate in the same direction and at the same speed, the force directions on both sides of the driven bevel gear are the same and it rotates to one side to realize the pitch motion of the heavy-duty platform. When the two driving bevel gears rotate in opposite directions and at the same speed, the force directions on both sides of the driven bevel gear are opposite and it rotates to realize the yaw motion of the heavy-duty platform. The pitch-yaw support mechanism is arranged on the translation mechanism and is controlled by the translation mechanism to move horizontally to realize the translation of the heavy-duty platform.

[0007] Further, the bracket includes a support plate and a pitch limit plate. There are two support plates arranged side by side left and right. There are two pitch limit plates, which are configured on the front and back sides of the two support plates. The pitch limit plates connect the two support plates and are used to limit the pitch angle of the heavy-duty platform.

[0008] Further, the pitch-yaw support mechanism further includes two first synchronous belt drive assemblies. Each first synchronous belt drive assembly corresponds to a driving bevel gear and can drive the corresponding driving bevel gear to rotate. Each first synchronous belt drive assembly includes a pitch-yaw motor, a driving pulley, a driven pulley and a first synchronous belt. The two pitch-yaw motors are configured on the front and back sides between the two support plates and are arranged left and right in opposite directions. The driving pulley is connected to the motor shaft of the pitch-yaw motor and can rotate with the motor shaft. The driven pulley is sleeved on the transverse support shaft and is fixedly connected to the driving bevel gear. The first synchronous belt is sleeved on the driving pulley and the driven pulley and is tensioned.

[0009] Further, the first synchronous belt drive assembly further includes a first tensioning pulley. The first tensioning pulley is rotatably installed on the support plate and abuts against the first synchronous belt to realize the tensioning of the first synchronous belt.

[0010] Further, the translation mechanism includes a fixed seat, a sliding pair and a second synchronous belt drive assembly. The pitch-yaw support mechanism is connected to the fixed seat via the sliding pair. The second synchronous belt drive assembly can drive the pitch-yaw support mechanism to move along the length direction of the fixed seat to realize the horizontal movement of the heavy-duty platform.

[0011] Further, the second synchronous belt drive assembly includes a translation motor, a second synchronous belt, a translation pulley, a tensioning seat, and a second tensioning pulley. The translation motor is installed on the bracket. The translation pulley is connected to the motor shaft of the translation motor and can rotate with the motor shaft. The tensioning seat is installed on the bracket. There are two groups of second tensioning pulleys installed on the front and rear sides of the tensioning seat. The second synchronous belt bypasses the translation pulley and then converges between the two groups of second tensioning pulleys. One end of the second synchronous belt abuts against the front second tensioning pulley and extends out of the tensioning seat. The other end of the second synchronous belt abuts against the rear second tensioning pulley and extends out of the tensioning seat. The two ends of the second synchronous belt respectively extend towards the two end portions of the fixed seat and are fixed to the front and rear end portions of the fixed seat.

[0012] The present invention also proposes a full-automatic calibration control system for a lidar and a fish-eye lens. The system includes a full-automatic calibration platform for a lidar and a fish-eye lens, a controller, a calibration device, and a backend server. The controller communicates with the calibration platform, the calibration device, and the backend server respectively. The controller controls the real-time movement of the calibration platform, and controls the calibration device to take pictures and collect lidar point cloud data. The calibration platform drives the calibration device to collect data at a specific angle. The backend server performs calibration calculations. The calibration device also communicates with the backend server, and the backend server transmits the processed data results back to the calibration device.

[0013] The present invention also proposes a full-automatic calibration method for a lidar and a fish-eye lens. Using a full-automatic calibration control system for a lidar and a fish-eye lens to calibrate the lidar and the fish-eye lens, the method includes the following steps:

[0014] First, fix the required calibration device to the heavy-duty platform of the calibration platform, start the calibration platform, the calibration platform automatically resets, the calibration device communicates with the calibration platform normally, and start data collection. The data collection process includes:

[0015] S1. Collection of calibration left camera internal parameter data:

[0016] The calibration platform moves the calibration device to make the optical center line of the left camera of the calibration device perpendicular to the calibration target and intersect with the center point of the calibration target. Move the optical center of the left camera of the calibration device to a distance of d 1 from the calibration target. Under the drive of the calibration platform, make the spatial intersection point of the left camera optical center line and the calibration target plane move in a closed trajectory formed by four straight lines connected end to end in sequence from the initial center point. According to the traveling direction of the last line segment of the closed trajectory, continue to move forward along the straight line, and then move in a closed trajectory formed by four straight lines connected end to end in sequence again. The final stop point is the initial center point. Record the movement method of forming two closed trajectories as double closed trajectory movement. Take N1 calibration target photos evenly on the trajectory of each straight line movement, and each photo is taken with the calibration device in a stationary condition.

[0017] Then the calibration platform moves the calibration device to a distance of d 2 from the calibration target. During this period, the optical center line of the left camera of the calibration device is always perpendicular to the calibration target, and N2 calibration target photos are evenly taken;

[0018] Under the drive of the calibration platform, the spatial intersection point of the optical center line of the left camera and the calibration target plane moves in the same motion mode as the double closed trajectory and finally returns to the initial center point; N3 calibration target photos are evenly taken on the trajectory of each straight-line motion, and each photo is taken with the device in a stationary condition;

[0019] All the collected data is uploaded to the server;

[0020] S2. Acquisition of external parameter data of the left camera calibration:

[0021] The left camera takes a calibration target photo at a distance of d 2 meters from the calibration target. After waiting for the radar point cloud to be generated, it is saved as a set of external parameter calibration data; then the calibration platform drives the calibration device to evenly collect external parameter calibration data 5 times within the range of ±60 degrees with the optical center of the left camera as the symmetry center line along the Yaw axis. The motion trajectories are yaw equal to +60°, +30°, 0°, -30°, -60° in sequence, or -60°, -30°, 0°, +30°, +60°;

[0022] All the collected data is uploaded to the backend server;

[0023] S3. Acquisition of external parameter and internal parameter data of the right camera calibration. The acquisition methods of external parameter calibration data and internal parameter calibration data of the right camera are the same as those of the left camera calibration data; after completing the external parameter and internal parameter calibration of the right camera, all the collected data is uploaded to the backend server;

[0024] S4. Wait for all the data to be uploaded. The backend server starts the calibration calculation. After the calibration calculation is completed, the calibration parameters are uploaded to the calibration device to complete the automatic calibration of the lidar and the lens.

[0025] Furthermore, d 1 is 0.5 meters, d 2 is 2 meters.

[0026] Furthermore, the process of the backend server starting the calibration calculation includes:

[0027] S401. The backend server completes the internal parameter calibration based on the taken photos. In this process, the cv2.fisheye related module of OpenCV is used to complete the camera calibration;

[0028] S402. The backend server completes the external parameter calibration based on the taken photos:

[0029] During the calibration process, the external parameters of the relative position between the camera and the radar are calibrated in a calibration board-based manner. In this process, it is necessary to ensure that the calibration board is placed in the common view area of the camera and the radar, and the corners of the calibration board are identified by the camera and the radar respectively.

[0030] Before the camera identifies the calibration board, according to the previously calibrated internal parameters, the image is projected into a panoramic image through equidistant rectangular projection; on the panoramic image, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corners on the panoramic image.

[0031] The method for the radar to identify the calibration board: First, accumulate the radar point cloud for t seconds; after t seconds, all the accumulated point clouds are cropped according to the position of the calibration board to remove irrelevant environmental content, and then projected into a panoramic image through equidistant rectangular projection. The intensity information of the radar is used as the gray value of the image; on this panoramic image of the radar intensity, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corners on the panoramic image.

[0032] After the camera and the radar respectively identify the calibration board corners, based on the 2D coordinates of the identified calibration board corners on the image and the 3D coordinates in the radar coordinate system, scipy is used to construct an optimization problem to optimize the rotation angles in the horizontal and vertical directions of the two panoramic images, as well as the corresponding displacements; finally, an external parameter transformation matrix from the radar coordinate system to the camera coordinate system is constructed, and at the same time, the reprojection error is obtained. According to this error, the calibration result is evaluated. When the error is less than the preset threshold, the calibration is considered successful.

[0033] The beneficial effects of the present invention compared with the prior art are as follows:

[0034] 1. The calibration device can achieve three-degree-of-freedom movement under the control of the calibration platform, occupies a small overall space, can significantly reduce the time and effort of manual input during the calibration process, greatly improve the accuracy and consistency of calibration, realize the batch and standardization of the entire calibration process operation, and ultimately achieve the purpose of improving the calibration efficiency.

[0035] 2. The overall structure of the calibration platform of the present invention is simple, has a high integration degree and a small size, can avoid the occlusion of the calibration platform to the calibration device, and can achieve a good calibration effect.

[0036] 3. The heavy-duty platform of the present invention is supported by a bracket. The heavy-duty platform can achieve heavy-duty, and can avoid the heavy-duty being transmitted to the driving bevel gear and the first synchronous belt drive assembly, reduce the load of the first synchronous belt drive assembly, and further reduce the impact of the load on the pitch-yaw motor, thereby improving the service life of the pitch-yaw motor.

[0037] 4. Fully consider the data acquisition process of the internal and external parameters of the left and right lenses in the calibration process and the connection between processes, design the shortest calibration sequence and calibration path, and maximize the calibration efficiency.

[0038] 5. As Figure 7 shown in the data acquisition trajectory, the calibration board can completely appear at various positions in the fish-eye lens, making the calibration board evenly distributed in the lens field of view, covering the edge area sensitive to distortion, and making the calibration more accurate. In addition, the double-distance calibration of the present invention combines different trajectory densities and the repeated positioning accuracy of the synchronous belt to ensure high-frequency sampling at close range and wide-area coverage at long range. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings, as part of this application, are used to provide a further understanding of the present invention.

[0040] Figure 1 FIG. is a schematic structural diagram of a full-automatic calibration platform for a lidar and a fish-eye lens according to the present invention.

[0041] Figure 2 FIG. is a partial enlarged view of the calibration platform.

[0042] Figure 3 FIG. is an internal sectional view of the calibration platform.

[0043] Figure 4 FIG. is a schematic structural diagram after the driven pulley and the driving bevel gear are integrally formed.

[0044] Figure 5 FIG. is an overall logic block diagram of the calibration control system.

[0045] Figure 6 FIG. is the overall calibration process of the calibration device.

[0046] Figure 7 FIG. is the movement trajectory of the internal parameter calibration.

[0047] Figure 8 FIG. is the movement trajectory of the external parameter calibration.

[0048] Description of the reference numerals:

[0049] 1 - Heavy-duty platform;

[0050] 2 - Pitch-yaw support mechanism, 21 - Bracket, 211 - Support plate, 212 - Pitch limit plate, 22 - Driven bevel gear, 23 - Driving bevel gear, 24 - Lateral support shaft, 25 - Longitudinal support shaft, 26 - First synchronous belt drive assembly, 261 - Pitch-yaw motor, 262 - Driving pulley, 263 - Driven pulley, 264 - First tensioning pulley, 265 - First synchronous belt;

[0051] 3 - Translation mechanism, 31 - Fixed seat, 32 - Sliding pair, 321 - Translation base, 322 - Slide block, 323 - Guide rail, 33 - Second synchronous belt drive assembly, 331 - Translation motor, 332 - Second synchronous belt, 333 - Translation pulley, 334 - Tensioning seat, 335 - Second tensioning pulley. Detailed implementation manners

[0052] The following describes the present invention in detail with reference to specific embodiments.

[0053] Figure 1 The structural schematic diagram of a full - automatic calibration platform for a lidar and a fish - eye lens in this embodiment is shown. As Figure 1 shown, the full - automatic calibration platform includes a heavy - duty platform 1, a pitch - yaw support mechanism 2, and a translation mechanism 3. The heavy - duty platform 1 is installed on the pitch - yaw support mechanism 2 and is controlled by the pitch - yaw support mechanism 2 to perform pitch motion or yaw motion; the pitch - yaw support mechanism 2 is installed on the translation mechanism 3 and is controlled by the translation mechanism 3 to perform horizontal movement, thereby realizing the translation of the heavy - duty platform 1. It can be seen that the heavy - duty platform 1 in this embodiment has three degrees of freedom. Calibration devices such as lidars and fish - eye lenses are detachably installed on the heavy - duty platform 1 and can move with the heavy - duty platform 1 to achieve data acquisition and precise calibration.

[0054] Figure 2 and Figure 3 The structural schematic diagram of the pitch - yaw support mechanism 2 in this embodiment is shown. For the convenience of subsequent description, a three - dimensional space coordinate system is established, where x the axis is the pitch axis, z the axis is the yaw axis. As Figure 2 and Figure 3 shown, the pitch - yaw support mechanism 2 in this embodiment includes a bracket 21, a driven bevel gear 22, a driving bevel gear 23, a lateral support shaft 24, a longitudinal support shaft 25, and a first synchronous belt drive assembly 26. Combining Figure 2 , the bracket 21 includes a support plate 211 and a pitch limiting plate 212. There are two support plates 211 arranged side by side left and right. There are two pitch limiting plates 212, which are configured on the front and rear sides of the two support plates 211. The pitch limiting plates 212 connect the two support plates 211 and are used to limit the pitch angle of the heavy - duty platform 1. The lateral support shaft 24 is horizontally installed on the two support plates 211 via flange bearings. The axial direction of the lateral support shaft 24 is the same as that of xThe directions of the axes are the same. A through hole is provided in the middle position of the transverse support shaft 24 in the diametrical direction. The longitudinal support shaft 25 is inserted into the through hole of the transverse support shaft 24 via a flange bearing and can rotate around the z-axis. The driven bevel gear 22 is coaxially and fixedly installed at the top of the longitudinal support shaft 25 and can rotate with the longitudinal support shaft 25. The heavy-duty platform 1 is arranged on the upper surface of the driven bevel gear 22. The size of the driving bevel gear 23 is smaller than that of the driven bevel gear 22. There are two driving bevel gears 23, and the two driving bevel gears 23 are symmetrically arranged on both sides of the driven bevel gear 22. Each driving bevel gear 23 is sleeved on the end of the transverse support shaft 24 through a flange bearing and meshes with the driven bevel gear 22. There are two first synchronous belt drive assemblies 26, and each first synchronous belt drive assembly 26 corresponds to a driving bevel gear 23 and can drive the corresponding driving bevel gear 23 to rotate. When the two driving bevel gears 23 rotate in the same direction and at the same speed, the two sides of the driven bevel gear 22 are subjected to two driving forces with the same vector. The driven bevel gear 22 rotates around the x-axis under these two driving forces, so as to realize the pitching motion of the heavy-duty platform 1. The pitching limit plate 212 can limit the pitching angle of the heavy-duty platform 1 to ±85°, avoiding excessive deviation of the center of gravity of the calibration device on the heavy-duty platform 1 and unable to reset. When the two driving bevel gears 23 rotate in the opposite direction and at the same speed, the driving forces received by the two sides of the driven bevel gear 22 are in different directions. The driven bevel gear 22 rotates around the z-axis under these two driving forces, so as to realize the yaw motion of the heavy-duty platform 1. Since there is no limit in the yaw direction of the heavy-duty platform 1, a 360° rotation of the calibration device can be realized, increasing the calibration range of the calibration device. It can be seen that the heavy-duty platform 1 of this embodiment is located at the top of the entire calibration platform. The heavy-duty platform 1 and the driven bevel gear 22 can be integrally formed. The transverse support shaft 24 and the longitudinal support shaft 25 are integrated between the driven bevel gear 22 and the two driving bevel gears 23, making the overall structure of the pitch-yaw support mechanism 2 simple, with high integration and small size. It can avoid the calibration platform from blocking the calibration device. After the calibration device is installed on the heavy-duty platform 1, there is no blockage around it, and a better calibration effect can be achieved. Especially when the calibration device uses a fish-eye lens, the fish-eye lens has a wide field of view and a large shooting range, and the calibration platform needs to avoid the shooting range of the lens to avoid affecting the calibration effect. The heavy-duty platform 1 of this embodiment can realize the pitching motion and the yaw motion under the coordinated cooperation of the driven bevel gear 22, the two driving bevel gears 23, the transverse support shaft 24 and the longitudinal support shaft 25. The whole execution process is simple and convenient. In addition, by Figure 3It can be seen that the horizontal support shaft 24 is supported by the support 21, and the driven bevel gear 22 is mounted on the horizontal support shaft 24 via the longitudinal support shaft 25. Therefore, the heavy load on the heavy-load platform 1 is sequentially transmitted to the support 21 via the driven bevel gear 22, the longitudinal support shaft 25, and the horizontal support shaft 24. Thus, the heavy-load platform 1 can carry heavy loads and avoid transmitting the load to the driving bevel gear 23 and the first synchronous belt drive assembly 26, reducing the load on the first synchronous belt drive assembly 26. Furthermore, the impact of the load on the pitch-yaw motor 261 can be reduced, and the service life of the pitch-yaw motor 261 can be extended.

[0055] As Figure 2 and Figure 3 shown, the two first synchronous belt drive assemblies 26 of this embodiment are respectively arranged on both sides of the two support plates 211 and are arranged in a mirror image. Each first synchronous belt drive assembly 26 includes a pitch-yaw motor 261, a driving pulley 262, a driven pulley 263, a first tensioning pulley 264, and a first synchronous belt 265. The two pitch-yaw motors 261 are arranged on the front and rear sides between the two support plates 211 and are arranged in opposite directions left and right, that is, the motor shaft of each pitch-yaw motor 261 corresponds to a support plate 211. The motor shaft of the pitch-yaw motor 261 extends out of the corresponding support plate 211 and is connected to the driving pulley 262. The driven pulley 263 is sleeved on the horizontal support shaft 24 via a flange bearing and is fixedly connected to the driving bevel gear 23. From Figure 4 it can be seen that the driven pulley 263 and the driving bevel gear 23 can be integrally formed. The first synchronous belt 265 is sleeved on the driving pulley 262 and the driven pulley 263. The first tensioning pulley 264 is rotatably mounted on the support plate 211 and abuts against the first synchronous belt 265 to achieve the tensioning of the first synchronous belt 265. When the pitch-yaw motor 261 drives the driving pulley 262 to rotate, the driven pulley 263 is driven by the first synchronous belt 265 and the driven pulley 263 drives the driving bevel gear 23 to rotate, realizing the pitch movement or yaw movement of the driven bevel gear 22. The driving bevel gear 23 of this embodiment is driven by the first synchronous belt 265, so that the pitch-yaw motor 261 can be placed below, realizing remote transmission. At the same time, the two pitch-yaw motors 261 are integrated between the two support plates 211 and are arranged below the heavy-load platform 1, reducing the space occupation and avoiding the occlusion of the lens.

[0056] Figure 1 and Figure 2 The structural schematic diagram of the translation mechanism 3 is also shown. As Figure 1 and Figure 2As shown in the figure, the translation mechanism 3 of this embodiment includes a fixed base 31, a sliding pair 32, and a second synchronous belt drive assembly 33. The fixed base 31 is a strip-shaped base. The pitch-yaw support mechanism 2 is connected to the fixed base 31 via the sliding pair 32, that is, the pitch-yaw support mechanism 2 has the degree of freedom of translation. The sliding pair 32 includes a translation base 321, a slider 322, and a guide rail 323. There are two guide rails 323, which are arranged side by side on the upper surface of the fixed base 31. The axial direction of the guide rail 323 is the same as the length direction of the fixed base 31. The translation base 321 is installed on the two guide rails 323 via the slider 322 and can slide. The pitch-yaw support mechanism 2 is installed on the upper surface of the translation base 321. The second synchronous belt drive assembly 33 is connected to the bracket 21 and can drive the pitch-yaw support mechanism 2 to move along the length direction of the fixed base 31 to realize the horizontal movement of the heavy-duty platform 1.

[0057] Combined with Figure 2 , the second synchronous belt drive assembly 33 of this embodiment includes a translation motor 331, a second synchronous belt 332, a translation pulley 333, a tensioning seat 334, and a second tensioning pulley 335. The translation motor 331 is installed on the support plate 211. The translation pulley 333 is connected to the motor shaft of the translation motor 331 and rotates with the motor shaft. The tensioning seat 334 is square. The tensioning seat 334 is installed on the support plate 211 and is arranged below the translation pulley 333. There are four second tensioning pulleys 335, which are evenly divided into two groups. The two groups of second tensioning pulleys 335 are arranged side by side on the front and rear sides of the tensioning seat 334. The second synchronous belt 332 bypasses the translation pulley 333 and then gathers downward and is inserted between the two groups of second tensioning pulleys 335. One end of the second synchronous belt 332 abuts against the front two second tensioning pulleys 335 and extends out of the tensioning seat 334. One end of the second synchronous belt 332 extends forward to the front end of the fixed base 31 and is fixed to the front end of the fixed base 31. The other end of the second synchronous belt 332 abuts against the rear two second tensioning pulleys 335 and extends out of the tensioning seat 334. The other end of the second synchronous belt 332 extends backward to the rear end of the fixed base 31 and is fixed to the rear end of the fixed base 31. Since the two ends of the second synchronous belt 332 are fixed, and the pitch-yaw support mechanism 2 has the degree of freedom of horizontal movement through the design of the sliding pair. During the process of the translation motor 331 driving the second synchronous belt 332 to move via the translation pulley 333, the second synchronous belt 332 drives the bracket 21 to move in the opposite direction via the translation pulley 333 and the translation motor 331. At this time, the bracket 21 drives the pitch-yaw support mechanism 2 and the translation base 321 to move along the axial direction of the guide rail 323, thereby realizing the translation of the heavy-duty platform 1. It can be seen from this that the heavy-duty platform 1 of this embodiment can realize long-distance horizontal movement under the drive of the second synchronous belt drive assembly 33 and the guidance of the sliding pair 32 to increase the calibration range.

[0058] The calibration device in this embodiment can achieve three-degree-of-freedom motion under the control of the calibration platform, occupies a small overall space, can significantly reduce the time and effort invested by humans during the calibration process, greatly improve the accuracy and consistency of calibration, realize the batch and standardization of the entire calibration process operation, and ultimately achieve the purpose of improving the calibration efficiency.

[0059] As Figure 5 shown, this embodiment also provides a full-automatic calibration control system for a lidar and a fish-eye lens. The system includes a full-automatic calibration platform, a controller, a calibration device, and a backend server. The controller uses an edge computing device (such as a Raspberry Pi). The controller communicates with the calibration platform and the calibration device respectively for real-time control. The backend server is based on a general-purpose computer and undertakes the tasks of data processing and running calibration algorithms.

[0060] In terms of the communication architecture, the controller is connected to the backend server through a wired network to provide high-reliability and low-latency data transmission; the controller communicates with the calibration platform through CAN to ensure precise motion control. The controller sends control signals to the calibration platform, and the calibration platform returns feedback signals such as position information and temperature information to the controller; the controller communicates wirelessly with the calibration device through WiFi, flexibly adapts to various devices, and controls the calibration device to take pictures, record lidar point cloud data, and transmit the collected data to the backend server by calling the internal API of the calibration device; the calibration device transmits data to the backend server through WiFi, and the backend server also wirelessly returns the processed data results to the calibration device through this path; the calibration platform and the calibration device are tightly and rigidly connected through a quick-release structure, and the calibration platform drives the calibration device to collect data at a specific angle. Among them, the camera used by the calibration device is a 185° wide-angle fish-eye camera. During calibration, in order to ensure the accuracy of calibration data (especially the camera internal parameters), it is necessary to make the calibration board appear at various positions in the field of view of the calibration lens as much as possible (that is, to make the calibration board cover the lens by rotating the heavy-duty platform 1). Thus, through the transmission method of the lower-mounted pitch-yaw motor 261 and the structure driven by two extremely narrow active bevel gears 23, when calibrating a camera with a 185° large FOV, the situation where the calibration platform blocks the lens will not occur. It can well improve the proportion of useful data. If a traditional calibration platform is used here, the structure itself will block the field of view and make the calibration result worse. In addition, the power supply of the calibration device is also realized together with the quick-release structure on the calibration platform; the overall control system logic block diagram is as Figure 5 shown.

[0061] The front end uses the NiceGUI framework and has a simple and intuitive operation interface, enabling the calibration task to be started quickly. The back end uses the FastAPI framework to process calibration data rapidly. The controller controls both the calibration platform and the calibration device simultaneously, enabling precise control over the pitch-yaw motor 261, the movement of the translation motor, and the timing of taking pictures by the calibration device, thus avoiding motion blur during picture taking by a rolling shutter camera.

[0062] In the present invention, the operation of the motor is closely coupled with data acquisition; based on the NiceGUI framework, the front end constructs an intuitive and user-friendly graphical user interface, enabling operators to efficiently complete the calibration task without programming knowledge, thereby reducing the training work for calibration personnel. The back end is based on the FastAPI framework and interacts efficiently with the front end, supporting the rapid iteration of calibration processing algorithms. The controller and the back-end server are connected via a wired network, providing more reliable and low-latency data transmission. The controller and the calibration device are connected via WiFi, eliminating the need for a wired connection and preventing the calibration process from being interrupted due to the influence of the calibration platform on the wires. Moreover, the design of the present invention fully considers the coordinated operation of the motor operation and the data acquisition of the calibration device. The movement trajectory of the camera is completely recorded, ensuring the traceability of the calibration process. At each shooting point, the camera is stationary for taking pictures, effectively avoiding image quality problems caused by motion blur, thereby improving the accuracy of the calibration result. With the flexible architecture of front-end and back-end separation, the system adopts a front-end and back-end separation design, improving the operation convenience and development efficiency.

[0063] When the calibration device is collecting data, since the camera lens used is a rolling shutter, if the lens shakes during the exposure of taking pictures, it will cause the collected images to be blurred, thereby leading to calibration failure. However, during the calibration process, more than 200 pictures are collected continuously for fifteen minutes. If collected manually, due to human fatigue or the inevitable natural shaking of the human hand, blurred pictures are likely to be collected. Therefore, by using this set of linked control systems, it can be ensured that the motor control and the triggering of the camera API are efficiently linked throughout the calibration process. Ensure that each picture is taken at a completely stationary state and at the specific angle we need, improving the quality of data collection;

[0064] In addition, to reduce the requirements for the quality of operators, we use the niceGUI front end to make an operation panel and simplify the operation to a few simple button clicks. In this way, it is very easy to expand the calibration lines, and even one person can operate multiple calibration lines.

[0065] In this embodiment, the described full-automatic calibration platform for lidar and fisheye lens is used to calibrate the lidar and the fisheye lens. The calibration includes internal parameter calibration and external parameter calibration. The internal parameter calibration is to calibrate the spatial parameters of the camera lens, and the corner points of the calibration target need to be evenly distributed in the camera's field of view. The external parameter calibration is to calibrate the relative position between the camera and the lidar. It is necessary to identify the corner points of the calibration target in the lidar point cloud, match them with the corner points of the calibration target in the picture, and determine the spatial relationship between the camera and the lidar according to the matching information. After completing the data acquisition for internal parameter calibration and external parameter calibration, the data is uploaded to the backend server. After the backend server processes the calibration data, the calibration parameters are sent to the calibration device. The overall calibration process of the calibration device is as Figure 6 shown. The calibration process is carried out in the order of left camera internal parameter calibration - left camera external parameter calibration - right camera external parameter calibration - right camera internal parameter calibration. The calibration platform has the shortest total travel distance and total rotation angle for completing the required processes in this order and the shortest required time.

[0066] First, fix the required calibration device on the heavy-duty platform 1 of the calibration platform, start the calibration platform, and the calibration platform automatically resets. The communication between the calibration device and the calibration platform is normal, and the data acquisition process begins.

[0067] S1. Calibrate the internal parameters of the left camera:

[0068] The calibration platform moves the calibration device so that the optical center line of the left camera of the calibration device is perpendicular to the calibration target and intersects the center point of the calibration target. Move the optical center of the left camera of the calibration device to a distance of 0.5 meters from the calibration target. Under the drive of the calibration platform, the spatial intersection point of the left camera optical center line and the calibration target plane moves along the Figure 7 shown closed trajectory formed by the sequence of arrows 1 to 8 starting from the initial center point, that is, starting from the initial center point, moving in four straight lines in the upper left, upper right, lower right, and lower left directions to form a closed trajectory (returning to the initial center point to form a square), then continuing to move in four straight lines in the lower left, lower right, upper right, and upper left directions to form a closed trajectory (returning to the initial center point to form a square). That is, each number represents a straight-line movement trajectory. Four calibration target photos are evenly taken on each straight-line movement trajectory, and each photo is taken with the calibration device in a stationary condition. Finally, return to the initial center point. Then the calibration platform moves the calibration device to a distance of 2 meters from the calibration target. During this period, the optical center line of the left camera of the calibration device is always perpendicular to the calibration target, and 10 calibration target photos are evenly taken. Then, under the drive of the calibration platform, the spatial intersection point of the left camera optical center line and the calibration target plane, according to Figure 7Move along the closed trajectory formed in sequence by the arrows 9-16 starting from the initial center point, that is, perform one more movement according to the aforementioned movement mode. Similarly, each number represents a straight-line movement trajectory. Four calibration target photos are evenly taken on each straight-line movement trajectory, and each time the photo is taken, the device is in a stationary condition. Finally, return to the initial center point and remain stationary, and upload all the collected data to the server.

[0069] S2. Calibrate the external parameters of the left camera:

[0070] The left camera takes a calibration target photo at a distance of 2 meters from the optical center to the calibration target, waits for the radar point cloud to be generated and then saves it as a set of external parameter calibration data. Then the calibration platform drives the calibration device to evenly collect external parameter calibration data 5 times within the range of ±60 degrees with the optical center of the left camera as the symmetry center line along the Yaw axis (yaw is equal to 0°, +60°, +30°, -60°, -30° respectively). The external parameter calibration movement trajectory is as Figure 8 shown. The calibration platform uploads all the collected data to the backend server.

[0071] S3. Start calibrating the external parameters and internal parameters of the right camera. The calibration principles of the external parameters and internal parameters of the right camera are the same as those of the left camera, so they will not be elaborated here. After completing the calibration of the external parameters and internal parameters of the right camera, upload all the collected data to the backend server.

[0072] S4. Wait for all the data to be uploaded. The backend server starts to process. After the processing is completed, if the processing result meets the requirements, upload the calibration parameters to the calibration device to complete the automatic calibration of the lidar and the lens, and voice broadcast "Perfect! The calibration data has been uploaded successfully". Otherwise, give a voice alarm to remind "Oops! The data collection failed." and remind manual processing.

[0073] S401. The backend server completes the internal parameter calibration based on the taken photos:

[0074] The calibration process uses OpenCV to implement the internal parameter calibration of the fisheye camera. Specifically, the relevant modules of cv2.fisheye in OpenCV are used to complete the camera calibration: First, the cv.findChessboardCorners() function in OpenCV is used to detect the feature points of the calibration board; the sub-pixel positioning method (such as the cv2.cornerSubPix() function in OpenCV) is adopted for the fine positioning of the feature points to improve the accuracy of feature point extraction; after the corner points are located, the fisheye camera calibration interface cv2.fisheye.calibrate() in OpenCV is called to calculate the 3x3 internal parameter matrix and distortion coefficients k1, k2, k3, k4 of the fisheye camera model, and at the same time obtain the reprojection error. According to this error, the calibration result is evaluated. When the error is less than the preset threshold (such as 0.5 pixels, related to the actual image resolution), the calibration is considered successful.

[0075] This method can efficiently and accurately complete the internal parameter calibration of the fisheye camera by combining the distortion model of the fisheye camera and the optimization algorithm.

[0076] S402. The backend server completes the external parameter calibration based on the taken photos:

[0077] In the calibration process, the external parameter calibration of the direct relative position between the camera and the radar is completed in a calibration board-based manner. In this process, it is necessary to ensure that the calibration board is placed in the common viewing area of the camera and the radar, and the camera and the radar are used to identify the corners of the calibration board respectively;

[0078] Before the camera identifies the calibration board, according to the previously calibrated internal parameters, the picture is projected into a panoramic image through equirectangular projection. On the panoramic image, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corner points on the panoramic image.

[0079] The method for the radar to identify the calibration board: First, accumulate the radar point cloud for 10 seconds. Since the used radar is non-repetitive scanning, the density of the radar points will gradually increase; after 10 seconds, all the accumulated point clouds are cropped according to the approximate position of the calibration board to remove the irrelevant environmental content, and then projected into a panoramic image through equirectangular projection. The intensity information of the radar is used as the gray value of the image. On this panoramic image of the radar intensity, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corner points on the panoramic image.

[0080] After separately identifying the corner points of the calibration board using a camera and a radar, an optimization problem is constructed using scipy based on the 2D coordinates of the identified calibration board corner points on the image and the 3D coordinates in the radar coordinate system to optimize the rotation angles in the horizontal and vertical directions of the two panoramic images, as well as the corresponding displacements. Finally, an external parameter transformation matrix from the radar coordinate system to the camera coordinate system is constructed, and at the same time, the reprojection error is obtained. The calibration result is evaluated according to this error. When the error is less than a preset threshold (such as 0.2°), the calibration is considered successful.

[0081] This solution has low requirements for the environment. During the data acquisition process, it only needs to ensure that the calibration device and the calibration board are relatively stationary and unobstructed, enabling the data acquisition to be deployed in a relatively complex environment and only occupying less space.

[0082] The processing method of the present invention can make good use of the structural design and control advantages of the calibration platform (the pitch and yaw axes are obtained by synthesizing the angles of two motors). By optimizing the control method, it can enable precise control by the servo motor to move along a preset closed polygon trajectory ( Figure 7 ), so that the calibration board is evenly distributed in the lens field of view, covering the edge areas sensitive to distortion. Moreover, the double-distance calibration (0.5m close distance + 2m far distance) in the present invention combines different trajectory densities and utilizes the repeated positioning accuracy of the synchronous belt (±0.1mm), which can ensure high-frequency sampling at close distances and wide-area coverage at far distances, and also guarantees the overall calibration effect. The moving method of the present invention can make the corner points of the calibration board evenly distributed in the fisheye image, reducing the solution error of the high-order distortion parameters (k3, k4) by 58%.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting the protection scope of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A fully automatic calibration platform for lidar and fisheye lens, characterized in that, It includes a heavy-duty platform, a pitch-yaw support mechanism and a translation mechanism. The pitch-yaw support mechanism includes a bracket, a driven bevel gear, a driving bevel gear, a transverse support shaft and a longitudinal support shaft. The transverse support shaft is horizontally installed on the bracket and can rotate. A through hole is provided on the transverse support shaft. The longitudinal support shaft is inserted into the through hole of the transverse support shaft and can rotate. The driven bevel gear is fixedly installed at the top of the longitudinal support shaft. The heavy-duty platform is arranged on the upper surface of the driven bevel gear. There are two driving bevel gears. The two driving bevel gears are respectively sleeved on both ends of the transverse support shaft and can rotate. Each driving bevel gear meshes with the driven bevel gear. When the two driving bevel gears rotate in the same direction and at the same speed, the force directions on both sides of the driven bevel gear are the same and it rotates to one side to realize the pitch movement of the heavy-duty platform. When the two driving bevel gears rotate in the opposite direction and at the same speed, the force directions on both sides of the driven bevel gear are opposite and it rotates to realize the yaw movement of the heavy-duty platform. The pitch-yaw support mechanism is arranged on the translation mechanism and is controlled by the translation mechanism to move horizontally to realize the translation of the heavy-duty platform. The bracket includes a support plate and a pitch limit plate. There are two support plates arranged side by side left and right. There are two pitch limit plates, which are configured on the front and rear sides of the two support plates. The pitch limit plates connect the two support plates and are used to limit the pitch angle of the heavy-duty platform. The pitch-yaw support mechanism further includes two first synchronous belt drive components. Each first synchronous belt drive component corresponds to a driving bevel gear and can drive the corresponding driving bevel gear to rotate. Each first synchronous belt drive component includes a pitch-yaw motor, a driving pulley, a driven pulley and a first synchronous belt. The two pitch-yaw motors are configured on the front and rear sides between the two support plates and are arranged in opposite directions left and right. The driving pulley is connected to the motor shaft of the pitch-yaw motor and can rotate with the motor shaft. The driven pulley is sleeved on the transverse support shaft and is fixedly connected to the driving bevel gear. The first synchronous belt is sleeved on the driving pulley and the driven pulley and is tensioned.

2. The fully automatic calibration platform for lidar and fish-eye lens according to claim 1, wherein, The first synchronous belt drive component further includes a first tensioning pulley. The first tensioning pulley is rotatably installed on the support plate and abuts against the first synchronous belt to realize the tensioning of the first synchronous belt.

3. The fully automatic calibration platform for lidar and fish-eye lens according to claim 1, characterized in that, The translation mechanism includes a fixed seat, a sliding pair and a second synchronous belt drive component. The pitch-yaw support mechanism is connected to the fixed seat via the sliding pair. The second synchronous belt drive component can drive the pitch-yaw support mechanism to move along the length direction of the fixed seat to realize the horizontal movement of the heavy-duty platform.

4. A fully automatic calibration platform for lidar and fish-eye lens according to claim 3, characterized in that, The second synchronous belt drive component includes a translation motor, a second synchronous belt, translation pulleys, a tensioning seat and second tensioning pulleys. The translation motor is installed on the bracket. The translation pulleys are connected to the motor shaft of the translation motor and can rotate with the motor shaft. The tensioning seat is installed on the bracket. There are two groups of second tensioning pulleys installed on the front and rear sides of the tensioning seat. The second synchronous belt bypasses the translation pulleys and then converges between the two groups of second tensioning pulleys. One end of the second synchronous belt abuts against the front second tensioning pulley and extends out of the tensioning seat. The other end of the second synchronous belt abuts against the rear second tensioning pulley and extends out of the tensioning seat. The two ends of the second synchronous belt respectively extend towards the two end parts of the fixed seat and are fixed to the front and rear end parts of the fixed seat.

5. A fully automatic calibration control system for a lidar and a fish-eye lens, characterized in that, The system includes a fully automatic calibration platform for lidar and fish-eye lens as described in any one of claims 1 to 4, a controller, a calibration device, and a backend server; the controller communicates with the calibration platform, the calibration device, and the backend server respectively. The controller controls the real-time movement of the calibration platform, and controls the calibration device to take pictures and collect lidar point cloud data; the calibration platform drives the calibration device to collect data at specific angles; the backend server performs calibration calculations, and the calibration device also communicates with the backend server, and the backend server transmits the processed data results back to the calibration device.

6. A fully automatic calibration method for lidar and fish-eye lens, characterized in that, Using the fully automatic calibration control system for lidar and fish-eye lens described in claim 5 to calibrate the lidar and fish-eye lens, including the following steps: First, fix the required calibration device on the heavy-duty platform of the calibration platform, start the calibration platform, the calibration platform automatically resets, the calibration device communicates with the calibration platform normally, and start data collection. The data collection process includes: S1. Collection of internal parameter data of the left camera: Calibrate the platform to move the calibration device so that the optical center line of the left camera of the calibration device is perpendicular to the calibration target and intersects the center point of the calibration target. Move the optical center of the left camera of the calibration device to a distance of d 1 from the calibration target. Under the drive of the calibration platform, make the spatial intersection point of the optical center line of the left camera and the calibration target plane move in four straight lines that are sequentially connected end to end from the initial center point to form a closed trajectory. Along the advancing direction of the last line segment of the closed trajectory, continue to move forward in a straight line, and then move in four straight lines that are sequentially connected end to end again to form a closed trajectory. The final stop point is the initial center point. Record the movement method of forming two closed trajectories as double closed trajectory movement; Uniformly take N1 calibration target photos on the trajectory of each straight-line movement, and each time the photo is taken, the calibration device is in a stationary condition; Then the calibration platform moves the calibration device to a distance of d 2 from the calibration target. During this period, the optical center line of the left camera of the calibration device is always perpendicular to the calibration target, and N2 calibration target photos are taken evenly. Under the drive of the calibration platform, the spatial intersection point of the optical center line of the left camera and the calibration target plane moves in the same motion mode as the double closed trajectory, and finally returns to the initial center point; N3 calibration target photos are evenly taken on the trajectory of each straight-line motion, and each time the photo is taken, the calibration device is in a static condition. Upload all the collected data to the backend server; S2. Collection of external parameter data of the left camera: The left camera is at the optical center distance calibration target d Take a calibration target photo at a distance of 2 meters, wait for the radar point cloud to be generated and save it as a set of external parameter calibration data; then the calibration platform drives the calibration device to uniformly collect external parameter calibration data 5 times within the range of ±60 degrees with the optical center of the left camera as the symmetric center line along the Yaw axis. The movement trajectories are yaw equal to +60°, +30°, 0°, -30°, -60°, or -60°, -30°, 0°, +30°, +60° in sequence; Upload all the collected data to the backend server; S3. Collection of external and internal parameter data of the right camera. The collection methods of external parameter calibration data and internal parameter calibration data of the right camera are the same as those of the left camera calibration data; after completing the external and internal parameter calibrations of the right camera, upload all the collected data to the backend server; S4. Wait for all the data to be uploaded. The backend server starts calibration calculations. After the calibration calculations are completed, upload the calibration parameters to the calibration device to complete the automatic calibration of the lidar and the lens.

7. A fully automatic calibration method for a lidar and a fish-eye lens according to claim 6, characterized in that d 1 is 0.5 meters, d 2 is 2 meters.

8. A fully automatic calibration method for a lidar and a fish-eye lens according to claim 6 or 7, characterized in that, The process of the backend server starting calibration calculations includes: S401. The backend server completes the internal parameter calibration based on the taken photos. In this process, the relevant modules of cv2.fisheye in OpenCV are used to complete the camera calibration; S402. The backend server completes the external parameter calibration based on the taken photos: During the calibration process, the external parameter calibration of the direct relative position between the camera and the radar is completed in a calibration board-based manner. In this process, it is necessary to ensure that the calibration board is placed in the common viewing area of the camera and the radar, and the camera and the radar are used to identify the corners of the calibration board respectively; Before the camera identifies the calibration board, according to the previously calibrated internal parameters, project the picture into a panoramic view through equidistant rectangular projection; call cv.findChessboardCorners() and cv2.cornerSubPix() on the panoramic view to find the coordinates of the calibration board corner points on the panoramic view; Method for calibrating a radar recognition calibration board: First, accumulate radar point clouds for t seconds; after t seconds, crop all the accumulated point clouds according to the position of the calibration board to remove irrelevant environmental content, and then project them into a panoramic image through equidistant rectangular projection, using the intensity information of the radar as the gray value of the image; call cv.findChessboardCorners() and cv2.cornerSubPix() on this panoramic radar intensity image to find the coordinates of the calibration board corners on the panoramic image. After using the camera and radar to identify the calibration board corners respectively, based on the 2D coordinates of the identified calibration board corners in the image and the 3D coordinates in the radar coordinate system, use scipy to construct an optimization problem to optimize the rotation angles in the horizontal and vertical directions of the two panoramic images, as well as the corresponding displacements; finally, construct the external parameter transformation matrix from the radar coordinate system to the camera coordinate system, and at the same time obtain the reprojection error, and evaluate the calibration result according to this error. When the error is less than the preset threshold, the calibration is considered successful.

Images