Laser radar and fisheye lens full-automatic calibration platform, calibration control system and calibration method

Abstract

The invention discloses a laser radar and fisheye lens full-automatic calibration platform, a calibration control system and a calibration method, and belongs to the field of three-dimensional scanning equipment. The problem that the calibration effect is poor due to an existing calibration mode is solved. A transverse supporting shaft is installed on a support and can rotate, a through hole is formed in the transverse supporting shaft, a longitudinal supporting shaft is inserted into the through hole of the transverse supporting shaft and can rotate, and a heavy load platform is fixedly installed at the top end of the longitudinal supporting shaft through a driven bevel gear; the two driving bevel gears are arranged at the two ends of the transverse supporting shaft in a sleeving mode respectively, each driving bevel gear is meshed with the corresponding driven bevel gear, and when the two driving bevel gears rotate in the same direction at the same speed, the stress directions of the two sides of each driven bevel gear are the same, and pitching motion of the heavy-load platform is achieved; when the two driving bevel gears rotate in the opposite directions at the same speed, the stress directions of the two sides of the driven bevel gear are opposite, and yawing motion of the heavy-load platform is achieved. And the pitching-yawing supporting mechanism is arranged on the translation mechanism and can realize translation of the heavy-load platform.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional scanning devices, and particularly relates to a full-automatic calibration platform, a calibration control system and a calibration method for a lidar and a fish-eye lens. Background Art

[0002] Currently, the calibration of a lidar and a fish-eye lens relies on calibration through a calibration board. The calibration requires the lidar and the lens to collect data of a calibration target at different positions simultaneously for multiple times. Currently, there are mainly two calibration methods. One is to perform calibration in a room filled with black and white grids. It is necessary for manpower to move the calibration target continuously along with the data collection of the lidar and the lens. This method occupies a large space and has a low space utilization rate. Moreover, due to the complex calibration process and low repeatability of manual placement of the calibration target, the calibration effect is greatly affected by operations, and it is difficult to achieve the standardization and automation of calibration equipment. The other is to use a six-axis robotic arm to carry a device at the end to continuously move a fixed calibration target for data collection and calibration. However, when the lens used is a fish-eye lens, in order to avoid blocking the field of view of the fish-eye lens during the movement of the robotic arm, the working dead zone of the six-axis robotic arm is relatively large, and the operating efficiency is low. In addition, since the robotic arm may interfere with the wires of the calibration equipment, resulting in the interruption of the calibration process, there are often risks such as equipment damage and short circuits.

[0003] In addition, in the current calibration methods for lidars and fish-eye lenses, the process of continuously moving the calibration target is carried out arbitrarily. This leads to the calibration board may appear at any position in the field of view of the lens during the calibration process, and cannot effectively cover the distortion-sensitive area, thus affecting the calibration effect to a certain extent. Summary of the Invention

[0004] In view of this, the present invention provides a full-automatic calibration platform, a calibration control system and a calibration method for a lidar and a fish-eye lens, which can realize the three-degree-of-freedom movement of the calibration equipment to achieve data collection and accurate calibration.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: A fully automatic calibration platform for lidar and fish-eye lens, comprising a heavy-duty platform, a pitch-yaw support mechanism and a translation mechanism. The pitch-yaw support mechanism includes a bracket, a driven bevel gear, a driving bevel gear, a horizontal support shaft and a vertical support shaft. The horizontal support shaft is horizontally installed on the bracket and can rotate. A through hole is provided on the horizontal support shaft. The vertical support shaft is inserted into the through hole of the horizontal support shaft and can rotate. The driven bevel gear is fixedly installed at the top end of the vertical support shaft. The heavy-duty platform is arranged on the upper surface of the driven bevel gear. There are two driving bevel gears. The two driving bevel gears are respectively sleeved at both ends of the horizontal support shaft and can rotate. Each driving bevel gear meshes with the driven bevel gear. When the two driving bevel gears rotate in the same direction and at the same speed, the force directions on both sides of the driven bevel gear are the same and rotate to one side to realize the pitch motion of the heavy-duty platform. When the two driving bevel gears rotate in opposite directions and at the same speed, the force directions on both sides of the driven bevel gear are opposite and rotate to realize the yaw motion of the heavy-duty platform. The pitch-yaw support mechanism is arranged on the translation mechanism and is controlled by the translation mechanism to move horizontally to realize the translation of the heavy-duty platform.

[0006] Further, the bracket includes a support plate and a pitch limit plate. There are two support plates arranged side by side left and right. There are two pitch limit plates, which are configured on the front and rear sides of the two support plates. The pitch limit plates connect the two support plates and are used to limit the pitch angle of the heavy-duty platform.

[0007] Further, the pitch-yaw support mechanism further includes two first synchronous belt drive assemblies. Each first synchronous belt drive assembly corresponds to a driving bevel gear and can drive the corresponding driving bevel gear to rotate. Each first synchronous belt drive assembly includes a pitch-yaw motor, a driving pulley, a driven pulley and a first synchronous belt. The two pitch-yaw motors are configured on the front and rear sides between the two support plates and are arranged in opposite directions left and right. The driving pulley is connected to the motor shaft of the pitch-yaw motor and can rotate with the motor shaft. The driven pulley is sleeved on the horizontal support shaft and is fixedly connected to the driving bevel gear. The first synchronous belt is sleeved on the driving pulley and the driven pulley and is tensioned.

[0008] Further, the first synchronous belt drive assembly further includes a first tensioning pulley. The first tensioning pulley is rotatably installed on the support plate and abuts against the first synchronous belt to realize the tensioning of the first synchronous belt.

[0009] Further, the translation mechanism includes a fixed seat, a sliding pair and a second synchronous belt drive assembly. The pitch-yaw support mechanism is connected to the fixed seat via the sliding pair. The second synchronous belt drive assembly can drive the pitch-yaw support mechanism to move along the length direction of the fixed seat to realize the horizontal movement of the heavy-duty platform.

[0010] Further, the second synchronous belt drive assembly includes a translation motor, a second synchronous belt, a translation pulley, a tensioning seat, and a second tensioning pulley. The translation motor is installed on the bracket. The translation pulley is connected to the motor shaft of the translation motor and can rotate with the motor shaft. The tensioning seat is installed on the bracket. There are two groups of second tensioning pulleys, which are installed on the front and rear sides of the tensioning seat. The second synchronous belt bypasses the translation pulley and then converges between the two groups of second tensioning pulleys. One end of the second synchronous belt abuts against the front second tensioning pulley and extends out of the tensioning seat. The other end of the second synchronous belt abuts against the rear second tensioning pulley and extends out of the tensioning seat. The two ends of the second synchronous belt extend towards the two end parts of the fixed seat respectively and are fixed to the front and rear end parts of the fixed seat.

[0011] The present invention also provides a full-automatic calibration control system for a lidar and a fish-eye lens. The system includes a full-automatic calibration platform for a lidar and a fish-eye lens, a controller, a calibration device, and a backend server. The controller communicates with the calibration platform, the calibration device, and the backend server respectively. The controller controls the real-time movement of the calibration platform, and controls the calibration device to take pictures and collect lidar point cloud data. The calibration platform drives the calibration device to collect data at a specific angle. The backend server performs calibration calculations. The calibration device also communicates with the backend server, and the backend server sends the processed data results back to the calibration device.

[0012] The present invention also provides a full-automatic calibration method for a lidar and a fish-eye lens. Using a full-automatic calibration control system for a lidar and a fish-eye lens to calibrate the lidar and the fish-eye lens, the method includes the following steps: First, fix the required calibration device on the heavy-duty platform of the calibration platform, start the calibration platform, the calibration platform automatically resets, the calibration device communicates with the calibration platform normally, and start data collection. The data collection process includes: S1. Collection of calibration data for the left camera internal parameters: The calibration platform moves the calibration device so that the optical center line of the left camera of the calibration device is perpendicular to the calibration target and intersects the center point of the calibration target. Move the optical center of the left camera of the calibration device to a distance from the calibration target d 1 At this position, under the drive of the calibration platform, the spatial intersection point of the optical center line of the left camera and the calibration target plane moves from the initial center point in a closed trajectory formed by four straight lines connected end to end in sequence. According to the traveling direction of the last line segment of the closed trajectory, continue to move forward along the straight line, and then form a closed trajectory again by four straight lines connected end to end in sequence. The final stop point is the initial center point. The movement mode of forming two closed trajectories is recorded as double-closed trajectory movement. Take N 1 calibration target photos evenly on the trajectory of each straight-line movement, and each photo is taken with the calibration device in a stationary condition; Then the calibration platform moves the calibration device to a distance from the calibration target d2 At this point, the left camera optical center line of the calibration device is always perpendicular to the calibration target, and N calibration target photos are evenly taken during the period. 2 photos of the calibration target; Under the drive of the calibration platform, the spatial intersection point of the left camera optical center line and the calibration target plane moves in the same motion mode as the double closed trajectory and finally returns to the initial center point; N calibration target photos are evenly taken on the trajectory of each straight-line motion, and each time the photo is taken, the device is in a stationary condition. 3 photos of the calibration target, and each time the photo is taken, the device is in a stationary condition; Upload all the collected data to the server; S2. Acquisition of external parameter data of the left camera calibration: The left camera takes a photo of the calibration target at a distance of d 2 meters from the optical center, waits for the radar point cloud to be generated and then saves it as a set of external parameter calibration data; then the calibration platform drives the calibration device to evenly collect external parameter calibration data 5 times within the range of ±60 degrees with the left camera optical center as the symmetric center line along the Yaw axis. The motion trajectories are yaw equal to +60°, +30°, 0°, -30°, -60° in sequence, or -60°, -30°, 0°, +30°, +60°; Upload all the collected data to the backend server; S3. Acquisition of external parameter and internal parameter data of the right camera calibration. The acquisition methods of external parameter calibration data and internal parameter calibration data of the right camera are the same as those of the left camera calibration data; after completing the external parameter and internal parameter calibration of the right camera, upload all the collected data to the backend server; S4. Wait for all the data to be uploaded. The backend server starts the calibration calculation. After the calibration calculation is completed, upload the calibration parameters to the calibration device to complete the automatic calibration of the lidar and the lens.

[0013] Furthermore, d 1 is 0.5 meters, d 2 is 2 meters.

[0014] Furthermore, the process of the backend server starting the calibration calculation includes: S401. The backend server completes the internal parameter calibration based on the taken photos. In this process, the relevant modules of cv2.fisheye in OpenCV are used to complete the camera calibration; S402. The backend server completes the external parameter calibration based on the taken photos: During the calibration process, the external parameter calibration of the direct relative position between the camera and the radar is completed by using the calibration plate method. In this process, it is necessary to ensure that the calibration plate is placed in the common view area of the camera and the radar, and the camera and the radar are used to identify the corners of the calibration plate respectively; Before the camera recognizes the calibration board, according to the previously calibrated internal parameters, the image is projected into a panoramic image through equidistant rectangular projection; on the panoramic image, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corners on the panoramic image; The method for the radar to recognize the calibration board: First, accumulate the radar point cloud for t seconds; after t seconds, all the accumulated point clouds are cropped according to the position of the calibration board to remove irrelevant environmental content, and then projected into a panoramic image through equidistant rectangular projection. The intensity information of the radar is used as the gray value of the image; on this panoramic image of the radar intensity, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corners on the panoramic image; After using the camera and radar to recognize the calibration board corners respectively, based on the 2D coordinates of the recognized calibration board corners on the image and the 3D coordinates in the radar coordinate system, scipy is used to construct an optimization problem to optimize the rotation angles in the horizontal and vertical directions of the two panoramic images, as well as the corresponding displacements; finally, an external parameter transformation matrix from the radar coordinate system to the camera coordinate system is constructed, and at the same time, the reprojection error is obtained. According to this error, the calibration result is evaluated. When the error is less than the preset threshold, the calibration is considered successful.

[0015] The beneficial effects of the present invention compared with the prior art are: 1. The calibration device can achieve three-degree-of-freedom movement under the control of the calibration platform. The overall occupied space is small, which can significantly reduce the time and energy invested by humans during the calibration process, greatly improve the accuracy and consistency of calibration, realize the batch and standardization of the entire calibration process operation, and ultimately achieve the purpose of improving the calibration efficiency.

[0016] 2. The overall structure of the calibration platform of the present invention is simple, has a high integration degree and a small size, can avoid the occlusion of the calibration platform to the calibration device, and can achieve a good calibration effect.

[0017] 3. The heavy-duty platform of the present invention is supported by a bracket. The heavy-duty platform can achieve heavy-duty, and can avoid the heavy-duty being transmitted to the driving bevel gear and the first synchronous belt drive assembly, reduce the load on the first synchronous belt drive assembly, and further reduce the impact of the load on the pitch-yaw motor, thereby improving the service life of the pitch-yaw motor.

[0018] 4. Fully consider the data acquisition process of the internal and external parameters of the left and right lenses in the calibration process and the connection between the processes, design the shortest calibration sequence and calibration path, and maximize the calibration efficiency.

[0019] 5. As Figure 7The shown data acquisition trajectory can make the calibration board fully appear at various positions in the fish-eye lens, evenly distribute the calibration board in the lens field of view, cover the edge area sensitive to distortion, and make the calibration more accurate. In addition, the dual-distance calibration of the present invention combines different trajectory densities and the repeated positioning accuracy of the synchronous belt, ensuring high-frequency sampling at close range and wide-area coverage at long range. Brief Description of the Drawings

[0020] The drawings, as a part of this application, are used to provide a further understanding of the present invention.

[0021] Figure 1 It is a schematic structural diagram of a full-automatic calibration platform for a lidar and a fish-eye lens of the present invention.

[0022] Figure 2 It is a partial enlarged view of the calibration platform.

[0023] Figure 3 It is an internal sectional view of the calibration platform.

[0024] Figure 4 It is a schematic structural diagram after the driven pulley and the driving bevel gear are integrally formed.

[0025] Figure 5 It is an overall logic block diagram of the calibration control system.

[0026] Figure 6 It is the overall calibration process of the calibration device.

[0027] Figure 7 It is the motion trajectory of the internal parameter calibration.

[0028] Figure 8 It is the motion trajectory of the external parameter calibration.

[0029] Description of the Reference Numerals: 1 - Heavy-duty platform; 2 - Pitch-yaw support mechanism, 21 - Bracket, 211 - Support plate, 212 - Pitch limit plate, 22 - Driven bevel gear, 23 - Driving bevel gear, 24 - Lateral support shaft, 25 - Longitudinal support shaft, 26 - First synchronous belt drive assembly, 261 - Pitch-yaw motor, 262 - Driving pulley, 263 - Driven pulley, 264 - First tensioning pulley, 265 - First synchronous belt; 3 - Translation mechanism, 31 - Fixed seat, 32 - Sliding pair, 321 - Translation base, 322 - Slide block, 323 - Guide rail, 33 - Second synchronous belt drive assembly, 331 - Translation motor, 332 - Second synchronous belt, 333 - Translation pulley, 334 - Tensioning seat, 335 - Second tensioning pulley. Detailed Description of the Preferred Embodiments

[0030] The present invention will be described in detail below with reference to specific embodiments.

[0031] Figure 1 Fig. shows a schematic structural diagram of a full-automatic calibration platform for a lidar and a fish-eye lens in this embodiment. As Figure 1 shown, the full-automatic calibration platform includes a heavy-duty platform 1, a pitch-yaw support mechanism 2, and a translation mechanism 3. The heavy-duty platform 1 is installed on the pitch-yaw support mechanism 2 and is controlled by the pitch-yaw support mechanism 2 to perform pitch motion or yaw motion; the pitch-yaw support mechanism 2 is installed on the translation mechanism 3 and is controlled by the translation mechanism 3 to perform horizontal movement, thereby realizing the translation of the heavy-duty platform 1. It can be seen that the heavy-duty platform 1 in this embodiment has three degrees of freedom. Calibration devices such as lidar and fish-eye lens are detachably installed on the heavy-duty platform 1 and can move with the heavy-duty platform 1 to realize data acquisition and precise calibration.

[0032] Figure 2 and Figure 3 Fig. shows a schematic structural diagram of the pitch-yaw support mechanism 2 in this embodiment. For the convenience of subsequent description, a three-dimensional space coordinate system is established, where x the axis is the pitch axis, z the axis is the yaw axis. As Figure 2 and Figure 3 shown, the pitch-yaw support mechanism 2 in this embodiment includes a bracket 21, a driven bevel gear 22, a driving bevel gear 23, a lateral support shaft 24, a longitudinal support shaft 25, and a first synchronous belt drive assembly 26. Combining Figure 2 with, the bracket 21 includes a support plate 211 and a pitch limit plate 212. There are two support plates 211 arranged side by side left and right. There are two pitch limit plates 212, which are arranged on the front and rear sides of the two support plates 211. The pitch limit plates 212 connect the two support plates 211 and are used to limit the pitch angle of the heavy-duty platform 1. The lateral support shaft 24 is horizontally installed on the two support plates 211 via flange bearings. The axial direction of the lateral support shaft 24 is the same as xThe directions of the axes are the same. A through hole is provided in the middle position of the transverse support shaft 24 in the diametrical direction. The longitudinal support shaft 25 is inserted into the through hole of the transverse support shaft 24 via a flange bearing and can rotate around the z-axis. The driven bevel gear 22 is coaxially and fixedly installed at the top of the longitudinal support shaft 25 and can rotate with the longitudinal support shaft 25. The heavy-duty platform 1 is arranged on the upper surface of the driven bevel gear 22. The size of the driving bevel gear 23 is smaller than that of the driven bevel gear 22. There are two driving bevel gears 23, and the two driving bevel gears 23 are symmetrically arranged on both sides of the driven bevel gear 22. Each driving bevel gear 23 is sleeved on the end of the transverse support shaft 24 through a flange bearing and meshes with the driven bevel gear 22. There are two first synchronous belt drive assemblies 26, and each first synchronous belt drive assembly 26 corresponds to a driving bevel gear 23 and can drive the corresponding driving bevel gear 23 to rotate. When the two driving bevel gears 23 rotate in the same direction and at the same speed, the two sides of the driven bevel gear 22 are subjected to two driving forces with the same vector. The driven bevel gear 22 rotates around the x-axis under the action of the two driving forces, so as to realize the pitching motion of the heavy-duty platform 1. The pitching limit plate 212 can limit the pitching angle of the heavy-duty platform 1 to ±85°, avoiding excessive deviation of the center of gravity of the calibration device on the heavy-duty platform 1 and unable to reset. When the two driving bevel gears 23 rotate in opposite directions and at the same speed, the directions of the driving forces received by the two sides of the driven bevel gear 22 are different. The driven bevel gear 22 rotates around the z-axis under the action of the two driving forces, so as to realize the yaw motion of the heavy-duty platform 1. Since there is no limit in the yaw direction of the heavy-duty platform 1, the calibration device can rotate 360°, increasing the calibration range of the calibration device. It can be seen that the heavy-duty platform 1 of this embodiment is located at the top of the entire calibration platform. The heavy-duty platform 1 and the driven bevel gear 22 can be integrally formed. The transverse support shaft 24 and the longitudinal support shaft 25 are integrated between the driven bevel gear 22 and the two driving bevel gears 23, making the overall structure of the pitching-yaw support mechanism 2 simple, with high integration and small size. It can avoid the calibration platform from blocking the calibration device. After the calibration device is installed on the heavy-duty platform 1, there is no blockage around it, and a better calibration effect can be achieved. Especially when the calibration device uses a fish-eye lens, the fish-eye lens has a wide field of view and a large shooting range, and the calibration platform needs to avoid the shooting range of the lens to avoid affecting the calibration effect. The heavy-duty platform 1 of this embodiment can realize the pitching motion and the yaw motion under the coordinated cooperation of the driven bevel gear 22, the two driving bevel gears 23, the transverse support shaft 24 and the longitudinal support shaft 25. The entire execution process is simple and convenient. In addition, by Figure 3It can be seen that the horizontal support shaft 24 is supported by the support 21, and the driven bevel gear 22 is installed on the horizontal support shaft 24 via the longitudinal support shaft 25. Therefore, the heavy load on the heavy-load platform 1 is sequentially transmitted to the support 21 via the driven bevel gear 22, the longitudinal support shaft 25, and the horizontal support shaft 24. Thus, the heavy-load platform 1 can carry heavy loads and avoid the load being transmitted to the driving bevel gear 23 and the first synchronous belt drive assembly 26, reducing the load on the first synchronous belt drive assembly 26. Furthermore, the impact of the load on the pitch-yaw motor 261 can be reduced, and the service life of the pitch-yaw motor 261 can be extended.

[0033] As Figure 2 and Figure 3 shown, the two first synchronous belt drive assemblies 26 of this embodiment are respectively arranged on both sides of the two support plates 211 and are arranged in a mirror image. Each first synchronous belt drive assembly 26 includes a pitch-yaw motor 261, a driving pulley 262, a driven pulley 263, a first tensioning pulley 264, and a first synchronous belt 265. The two pitch-yaw motors 261 are arranged on the front and rear sides between the two support plates 211 and are arranged in opposite directions left and right, that is, the motor shaft of each pitch-yaw motor 261 corresponds to one support plate 211. The motor shaft of the pitch-yaw motor 261 extends out of the corresponding support plate 211 and is connected to the driving pulley 262. The driven pulley 263 is sleeved on the horizontal support shaft 24 via a flange bearing and is fixedly connected to the driving bevel gear 23. From Figure 4 it can be seen that the driven pulley 263 and the driving bevel gear 23 can be integrally formed. The first synchronous belt 265 is sleeved on the driving pulley 262 and the driven pulley 263. The first tensioning pulley 264 is rotatably installed on the support plate 211 and abuts against the first synchronous belt 265 to achieve the tensioning of the first synchronous belt 265. When the pitch-yaw motor 261 drives the driving pulley 262 to rotate, the driven pulley 263 is driven by the first synchronous belt 265 and the driven pulley 263 drives the driving bevel gear 23 to rotate, realizing the pitch movement or yaw movement of the driven bevel gear 22. The driving bevel gear 23 of this embodiment is driven by the first synchronous belt 265, so that the pitch-yaw motor 261 can be placed below, realizing remote transmission. At the same time, the two pitch-yaw motors 261 are integrated between the two support plates 211 and are arranged below the heavy-load platform 1, reducing the space occupation and avoiding the occlusion of the lens.

[0034] Figure 1 and Figure 2 The structural schematic diagram of the translation mechanism 3 is also shown. As Figure 1 and Figure 2As shown in the figure, the translation mechanism 3 of this embodiment includes a fixed base 31, a sliding pair 32, and a second synchronous belt drive assembly 33. The fixed base 31 is a strip-shaped base, and the pitch-yaw support mechanism 2 is connected to the fixed base 31 via the sliding pair 32, that is, the pitch-yaw support mechanism 2 has the degree of freedom of translation. The sliding pair 32 includes a translation base 321, a slider 322, and a guide rail 323. There are two guide rails 323, which are arranged side by side on the upper surface of the fixed base 31. The axial direction of the guide rail 323 is the same as the length direction of the fixed base 31. The translation base 321 is installed on the two guide rails 323 via the slider 322 and can slide. The pitch-yaw support mechanism 2 is installed on the upper surface of the translation base 321. The second synchronous belt drive assembly 33 is connected to the bracket 21 and can drive the pitch-yaw support mechanism 2 to move along the length direction of the fixed base 31 to achieve the horizontal movement of the heavy-duty platform 1.

[0035] Combined with Figure 2 , the second synchronous belt drive assembly 33 of this embodiment includes a translation motor 331, a second synchronous belt 332, a translation pulley 333, a tensioning seat 334, and a second tensioning pulley 335. The translation motor 331 is installed on the support plate 211. The translation pulley 333 is connected to the motor shaft of the translation motor 331 and rotates with the motor shaft. The tensioning seat 334 is square. The tensioning seat 334 is installed on the support plate 211 and is arranged below the translation pulley 333. There are four second tensioning pulleys 335, which are evenly divided into two groups. The two groups of second tensioning pulleys 335 are arranged side by side on the front and rear sides of the tensioning seat 334. The second synchronous belt 332 bypasses the translation pulley 333 and then converges downward and is inserted between the two groups of second tensioning pulleys 335. One end of the second synchronous belt 332 abuts against the two front second tensioning pulleys 335 and extends out of the tensioning seat 334. One end of the second synchronous belt 332 extends forward to the front end of the fixed base 31 and is fixed to the front end of the fixed base 31. The other end of the second synchronous belt 332 abuts against the two rear second tensioning pulleys 335 and extends out of the tensioning seat 334. The other end of the second synchronous belt 332 extends backward to the rear end of the fixed base 31 and is fixed to the rear end of the fixed base 31. Since both ends of the second synchronous belt 332 are fixed, and the pitch-yaw support mechanism 2 has the degree of freedom of horizontal movement through the design of the sliding pair. During the process of the translation motor 331 driving the second synchronous belt 332 to move via the translation pulley 333, the second synchronous belt 332 drives the bracket 21 to move in the reverse direction via the translation pulley 333 and the translation motor 331. At this time, the bracket 21 drives the pitch-yaw support mechanism 2 and the translation base 321 to move along the axial direction of the guide rail 323, thereby realizing the translation of the heavy-duty platform 1. It can be seen from this that the heavy-duty platform 1 of this embodiment can achieve long-distance horizontal movement under the drive of the second synchronous belt drive assembly 33 and the guidance of the sliding pair 32 to increase the calibration range.

[0036] The calibration device in this embodiment can achieve three-degree-of-freedom motion under the control of the calibration platform, occupies a small overall space, can significantly reduce the time and effort invested by humans during the calibration process, greatly improve the accuracy and consistency of calibration, realize the batch and standardization of the entire calibration process operation, and ultimately achieve the purpose of improving the calibration efficiency.

[0037] As Figure 5 shown, this embodiment also provides a full-automatic calibration control system for a lidar and a fish-eye lens. The system includes a full-automatic calibration platform, a controller, a calibration device, and a backend server. The controller uses an edge computing device (such as a Raspberry Pi), and the controller communicates with the calibration platform and the calibration device respectively for real-time control. The backend server is based on a general-purpose computer and undertakes the tasks of data processing and running calibration algorithms.

[0038] In terms of the communication architecture, the controller and the backend server are connected through a wired network to provide high-reliability and low-latency data transmission; the controller and the calibration platform communicate through CAN to ensure precise motion control. The controller sends control signals to the calibration platform, and the calibration platform returns feedback signals such as position information and temperature information to the controller; the controller and the calibration device achieve wireless communication through WiFi, flexibly adapt to various devices, and control the calibration device to take pictures, record lidar point cloud data, and transmit the collected data to the backend server by invoking the internal API of the calibration device; the calibration device transmits data to the backend server through WiFi, and the backend server also wirelessly returns the processed data results to the calibration device through this path; the calibration platform and the calibration device are tightly and rigidly connected through a quick-release structure, and the calibration platform drives the calibration device to collect data at a specific angle. Among them, the camera used by the calibration device is a 185° wide-angle fish-eye camera. During calibration, in order to ensure the accuracy of calibration data (especially the camera internal parameters), it is necessary to make the calibration board appear at various positions in the field of view of the calibration lens as much as possible (that is, to make the calibration board cover the lens by rotating the heavy-duty platform 1). Thus, through the transmission method of the lower-mounted pitch-yaw motor 261 and the structure driven by two extremely narrow active bevel gears 23, when calibrating a camera with a 185° large FOV, the situation where the calibration platform blocks the lens will not occur. It can well improve the proportion of useful data. If a traditional calibration platform is used here, the structure itself will block the field of view and make the calibration result worse. In addition, the power supply of the calibration device is also realized together with the quick-release structure on the calibration platform; the overall control system logic block diagram is as Figure 5 shown.

[0039] The front end uses the NiceGUI framework and has a simple and intuitive operation interface, enabling the calibration operation to be started quickly. The back end uses the FastAPI framework to process calibration data rapidly. The controller controls both the calibration platform and the calibration device simultaneously, enabling precise control over the pitch-yaw motor 261, the movement of the translation motor, and the timing of taking pictures by the calibration device, thus avoiding motion blur during picture taking by a rolling shutter camera.

[0040] In the present invention, the operation of the motor is closely coupled with data acquisition; based on the NiceGUI framework, the front end constructs an intuitive and user-friendly graphical user interface, enabling operators to efficiently complete calibration tasks without programming knowledge, reducing the training work for calibration personnel. The back end is based on the FastAPI framework, interacting efficiently with the front end and supporting rapid iteration of calibration processing algorithms. The controller and the back-end server are connected via a wired network, providing more reliable and low-latency data transmission. The controller and the calibration device are connected via WiFi, eliminating the need for wired connection and preventing the calibration process from being interrupted due to the influence of the calibration platform on the wires. Moreover, the design of the present invention fully considers the coordinated operation of the motor operation and the data acquisition of the calibration device. The movement trajectory of the camera is completely recorded to ensure the traceability of the calibration process. At each shooting point, the camera is stationary for picture taking, effectively avoiding image quality problems caused by motion blur, thereby improving the accuracy of the calibration result. With the flexible architecture of separating the front and back ends, the system adopts a front-end and back-end separation design, enhancing the operation convenience and development efficiency.

[0041] When the calibration device collects data, since the camera lens used is a rolling shutter, if the lens shakes during the exposure of taking pictures, it will cause the collected images to be blurred, thereby leading to calibration failure. However, during the calibration process, more than 200 pictures are collected continuously for fifteen minutes. If collected manually, due to human fatigue or the inevitable natural shaking of the human hand, blurred pictures are likely to be collected. Therefore, by using this set of linked control systems, it can be ensured that the motor control and the triggering of the camera taking pictures API are efficiently linked throughout the calibration process. Ensure that each picture is taken at a completely stationary state and at a specific angle we need, improving the quality of data collection; In addition, to reduce the requirements for the quality of operators, we use the niceGUI front end as an operation panel and simplify the operation to a few simple button clicks. In this way, it is very easy to expand the calibration lines, and even one person can operate multiple calibration lines.

[0042] In this embodiment, the above-mentioned full-automatic calibration platform for lidar and fisheye lens is used to calibrate the lidar and the fisheye lens. The calibration includes internal parameter calibration and external parameter calibration. The internal parameter calibration is to calibrate the spatial parameters of the camera lens, and the corner points of the calibration target need to be evenly distributed in the camera's field of view. The external parameter calibration is to calibrate the relative position between the camera and the lidar. It is necessary to identify the corner points of the calibration target on the lidar point cloud, match them with the corner points of the calibration target in the picture, and determine the spatial relationship between the camera and the lidar according to the matching information. After completing the data acquisition of the internal parameter calibration and the external parameter calibration, the data is uploaded to the backend server. After the backend server processes the calibration data, the calibration parameters are sent to the calibration device. The overall calibration process of the calibration device is as Figure 6 shown. The calibration process is carried out in the order of left camera internal parameter calibration - left camera external parameter calibration - right camera external parameter calibration - right camera internal parameter calibration. The calibration platform has the minimum total travel distance and total rotation angle for the calibration required processes in this order, and the shortest required time.

[0043] First, fix the required calibration device to the heavy-duty platform 1 of the calibration platform, start the calibration platform, the calibration platform automatically resets, the communication between the calibration device and the calibration platform is normal, and the data acquisition process begins.

[0044] S1. Calibrate the internal parameters of the left camera: The calibration platform moves the calibration device so that the optical center line of the left camera of the calibration device is perpendicular to the calibration target and intersects the center point of the calibration target. Move the optical center of the left camera of the calibration device to a distance of 0.5 meters from the calibration target. Under the drive of the calibration platform, the spatial intersection point of the left camera optical center line and the calibration target plane moves along the closed trajectory composed of arrows 1-8 in the order Figure 7 shown, that is, starting from the initial center point, it moves in four straight lines in the upper left, upper right, lower right, and lower left directions to form a closed trajectory (returning to the initial center point to form a square), then continues to move in four straight lines in the lower left, lower right, upper right, and upper left directions to form a closed trajectory (returning to the initial center point to form a square). That is, each number represents a straight-line movement trajectory. Four calibration target photos are evenly taken on each straight-line movement trajectory, and each photo is taken with the calibration device in a stationary condition. Finally, it returns to the initial center point. Then the calibration platform moves the calibration device to a distance of 2 meters from the calibration target. During this period, the optical center line of the left camera of the calibration device is always perpendicular to the calibration target, and 10 calibration target photos are evenly taken. Then, under the drive of the calibration platform, the spatial intersection point of the left camera optical center line and the calibration target plane moves along the closed trajectory composed of arrows 9-16 in the order Figure 7 shown, that is, it moves again in the aforementioned movement mode. Similarly, each number represents a straight-line movement trajectory. Four calibration target photos are evenly taken on each straight-line movement trajectory, and each photo is taken with the device in a stationary condition. Finally, it returns to the initial center point and remains stationary, and all the collected data is uploaded to the server.

[0045] S2. Calibrate the extrinsic parameters of the left camera: The left camera takes a calibration target photo at a distance of 2 meters from the optical center to the calibration target. Wait for the lidar point cloud to be generated and then save it as a set of extrinsic parameter calibration data. Then, the calibration platform drives the calibration device to uniformly collect extrinsic parameter calibration data 5 times within the range of ±60 degrees with the optical center of the left camera as the symmetry center line along the Yaw axis (yaw is equal to 0°, +60°, +30°, -60°, -30° respectively). The motion trajectory of the extrinsic parameter calibration is as Figure 8 shown. The calibration platform uploads all the collected data to the backend server.

[0046] S3. Start calibrating the extrinsic parameters and intrinsic parameters of the right camera. The calibration principles of the extrinsic and intrinsic parameters of the right camera are the same as those of the left camera, so they will not be elaborated here. After completing the calibration of the extrinsic and intrinsic parameters of the right camera, upload all the collected data to the backend server.

[0047] S4. Wait for all the data to be uploaded. The backend server starts to process. After the processing is completed, if the processing result meets the requirements, upload the calibration parameters to the calibration device to complete the automatic calibration of the lidar and the lens, and the voice broadcasts "Perfect! The calibration data has been uploaded successfully". Otherwise, a voice alarm will be given to remind "Oops! The data acquisition failed." to remind manual processing.

[0048] S401. The backend server completes the intrinsic parameter calibration based on the taken photos: The calibration process uses the OpenCV-based fisheye camera intrinsic parameter calibration. Specifically, use the relevant modules of cv2.fisheye in OpenCV to complete the camera calibration: First, use the cv.findChessboardCorners() function to detect the feature points of the calibration board; use the sub-pixel localization method (such as the cv2.cornerSubPix() function in OpenCV) for fine localization of the feature points to improve the accuracy of feature point extraction; after the corners are located, call the fisheye camera calibration interface cv2.fisheye.calibrate() in OpenCV to solve the 3x3 matrix of the intrinsic parameters of the fisheye camera model and the distortion coefficients k1, k2, k3, k3, and at the same time obtain the reprojection error. Evaluate the calibration result according to this error. When the error is less than the preset threshold (such as 0.5 pixels, related to the actual image resolution), the calibration is considered successful.

[0049] This method can efficiently and accurately complete the intrinsic parameter calibration of the fisheye camera by combining the distortion model and optimization algorithm of the fisheye camera.

[0050] S402. The backend server completes the extrinsic parameter calibration based on the taken photos: During the calibration process, the external parameters of the relative position between the camera and the radar are calibrated based on the calibration board. In this process, it is necessary to ensure that the calibration board is placed in the common view area of the camera and the radar, and the corners of the calibration board are identified by the camera and the radar respectively; Before the camera identifies the calibration board, according to the previously calibrated internal parameters, the image is projected into a panoramic image through equirectangular projection. On the panoramic image, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corners on the panoramic image.

[0051] The method for the radar to identify the calibration board: First, accumulate the radar point cloud for 10 seconds. Since the used radar is non-repetitive scanning, the density of the radar points will gradually increase; after 10 seconds, all the accumulated point clouds are cropped according to the approximate position of the calibration board to remove irrelevant environmental content, and then projected into a panoramic image through equirectangular projection. The intensity information of the radar is used as the gray value of the image. On this panoramic image of the radar intensity, cv.findChessboardCorners() and cv2.cornerSubPix() are called to find the coordinates of the calibration board corners on the panoramic image.

[0052] After the corners of the calibration board are identified by the camera and the radar respectively, based on the 2D coordinates of the identified calibration board corners on the image and the 3D coordinates in the radar coordinate system, scipy is used to construct an optimization problem to optimize the rotation angles in the horizontal and vertical directions of the two panoramic images, as well as the corresponding displacements. Finally, an external parameter transformation matrix from the radar coordinate system to the camera coordinate system is constructed, and the reprojection error is obtained. The calibration result is evaluated according to this error. When the error is less than the preset threshold (such as 0.2°), the calibration is considered successful.

[0053] This solution has low requirements for the environment. During the data acquisition process, it only needs to ensure that the calibration device and the calibration board are relatively stationary and unobstructed, so that the data acquisition can be deployed in a relatively complex environment and only occupy less space.

[0054] The processing method of the present invention can make good use of the structural design and control advantages of the calibration platform (the pitch and yaw axes are obtained by synthesizing the angles of two motors). By optimizing the control method, it is possible to accurately control the servo motor to follow the preset closed polygon trajectory ( Figure 7The movement then enables the calibration board to be evenly distributed in the lens field of view, covering the edge areas sensitive to distortion. Moreover, in the present invention, the dual-distance calibration (0.5 m near distance + 2 m far distance) combines different trajectory densities. By utilizing the repetitive positioning accuracy (±0.1 mm) of the synchronous belt, it can ensure high-frequency sampling at close range and wide-area coverage at far range, and also guarantee the overall calibration effect. The movement mode of the present invention can make the corner points of the calibration board evenly distributed in the fish-eye image, reducing the calculation error of the high-order distortion parameters (k3, k4) by 58%.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting the protection scope of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A fully automatic calibration platform for laser radar and fisheye lens, characterized in that: It includes a heavy-loaded platform, a pitch-yaw support mechanism and a translation mechanism, wherein the pitch-yaw support mechanism includes a bracket, a driven bevel gear, a driving bevel gear, a transverse support shaft and a longitudinal support shaft, the transverse support shaft is horizontally installed on the bracket and can rotate, a through hole is opened on the transverse support shaft, the longitudinal support shaft is inserted into the through hole of the transverse support shaft and can rotate, the driven bevel gear is fixed to the top end of the longitudinal support shaft, and the heavy-loaded platform is arranged on the upper surface of the driven bevel gear; there are two driving bevel gears, the two driving bevel gears are respectively sleeved on the two ends of the transverse support shaft and can rotate, each driving bevel gear is meshed with the driven bevel gear, when the two driving bevel gears rotate in the same direction and at the same speed, the two sides of the driven bevel gear are subjected to the same force direction and rotate to one side, so as to realize the pitching movement of the heavy-loaded platform; when the two driving bevel gears rotate in opposite directions and at the same speed, the two sides of the driven bevel gear are subjected to opposite force directions and rotate, so as to realize the yaw movement of the heavy-loaded platform; The pitch-yaw support mechanism is arranged on the translation mechanism and is controlled by the translation mechanism to perform horizontal movement, so as to realize the translation of the heavy-load platform.

2. The fully automatic calibration platform for laser radar and fisheye lens according to claim 1, characterized in that: The bracket includes a support plate and a pitch limit plate. There are two support plates arranged side by side. There are two pitch limit plates arranged on the front and rear sides of the two support plates. The pitch limit plate connects the two support plates and is used to limit the pitch angle of the heavy-load platform.

3. The fully automatic calibration platform for laser radar and fisheye lens according to claim 2, characterized in that: The pitch-yaw support mechanism also includes two first synchronous belt drive assemblies, each of which corresponds to a driving bevel gear and can drive the corresponding driving bevel gear to rotate; each first synchronous belt drive assembly includes a pitch-yaw motor, a driving pulley, a driven pulley and a first synchronous belt, the two pitch-yaw motors are arranged on the front and rear sides between the two support plates and are arranged in opposite directions left and right, the driving pulley is connected to the motor shaft of the pitch-yaw motor and can rotate with the motor shaft, the driven pulley is sleeved on the transverse support shaft and fixedly connected to the driving bevel gear, and the first synchronous belt is sleeved on the driving pulley and the driven pulley and is tensioned.

4. The laser radar and fisheye lens fully automatic calibration platform according to claim 3, characterized in that: The first synchronous belt driving assembly also includes a first tensioning wheel, which is rotatably mounted on the support plate and abuts against the first synchronous belt to achieve tensioning of the first synchronous belt.

5. The fully automatic calibration platform for laser radar and fisheye lens according to claim 1, characterized in that: The translation mechanism includes a fixed seat, a sliding pair and a second synchronous belt drive assembly. The pitch-yaw support mechanism is connected to the fixed seat via the sliding pair. The second synchronous belt drive assembly can drive the pitch-yaw support mechanism to move along the length direction of the fixed seat to achieve horizontal movement of the heavy-load platform.

6. The fully automatic calibration platform for laser radar and fisheye lens according to claim 5, characterized in that: The second synchronous belt driving assembly includes a translation motor, a second synchronous belt, a translation pulley, a tensioning seat and a second tensioning pulley. The translation motor is installed on the bracket, the translation pulley is connected to the motor shaft of the translation motor and can rotate with the motor shaft, the tensioning seat is installed on the bracket, the second tensioning pulley is provided with two groups and is installed on the front and rear sides of the tensioning seat, the second synchronous belt passes around the translation pulley and is retracted between the two groups of second tensioning pulleys, one end of the second synchronous belt abuts against the second tensioning pulley in the front and extends out of the tensioning seat, the other end of the second synchronous belt abuts against the second tensioning pulley in the rear and extends out of the tensioning seat, and the two ends of the second synchronous belt extend to the two end ends of the fixed seat respectively and are fixed to the front and rear end ends of the fixed seat.

7. A fully automatic calibration control system for laser radar and fisheye lens, characterized in that: The system includes a fully automatic calibration platform for laser radar and fisheye lens as described in any one of claims 1 to 6, a controller, a calibration device and a back-end server; the controller communicates with the calibration platform, the calibration device and the back-end server respectively, and the controller controls the real-time movement of the calibration platform, and controls the calibration device to take pictures and collect radar point cloud data; the calibration platform drives the calibration device to collect data at a specific angle; the back-end server performs calibration calculations, and the calibration device also communicates with the back-end server, and the back-end server sends the processed data results back to the calibration device.

8. A fully automatic calibration method for laser radar and fisheye lens, characterized in that: Using the laser radar and fisheye lens fully automatic calibration control system described in claim 7 to calibrate the radar and fisheye lens, the following steps are included: First, fix the required calibration equipment to the heavy-load platform of the calibration platform, start the calibration platform, and the calibration platform will automatically reset. The calibration equipment and the calibration platform communicate normally, and start data collection. The data collection process includes: S1. Calibrate the left camera internal reference data collection: The calibration platform moves the calibration device so that the optical center line of the left camera of the calibration device is perpendicular to the calibration target and intersects with the center point of the calibration target. Move the optical center of the left camera of the calibration device to the distance from the calibration target. d 1. Driven by the calibration platform, the spatial intersection of the optical center line of the left camera and the calibration target plane starts from the initial center point and moves in four straight lines connected end to end in sequence to form a closed trajectory. According to the direction of the last line segment of the closed trajectory, it continues to move forward along the straight line, and again moves in four straight lines connected end to end in sequence to form a closed trajectory. The final stop point is the initial center point. The movement mode that forms two closed trajectories is recorded as a double closed trajectory movement. Evenly take N1 calibration target photos on the trajectory of each straight line motion, and each shooting is taken with the calibration equipment in a static condition. The calibration platform then moves the calibration device to the calibration target d 2. During this period, the optical center line of the left camera of the calibration equipment is always perpendicular to the calibration target, and N2 calibration target photos are evenly taken; Driven by the calibration platform, the spatial intersection of the optical center line of the left camera and the calibration target plane moves in the same way as the double closed trajectory motion, and finally returns to the initial center point; N3 calibration target photos are evenly taken on each linear motion trajectory, and each shot is taken with the calibration device in a stationary condition; Upload all collected data to the backend server; S2. Calibrate the left camera external parameter data collection: The left camera is at the optical center distance calibration target d Take a calibration target photo at a distance of 2 meters, wait for the radar point cloud to be generated and save it as a set of external parameter calibration data; then the calibration platform drives the calibration device to uniformly collect external parameter calibration data 5 times along the Yaw axis within the range of ±60 degrees with the optical center of the left camera as the symmetrical midline, and the motion trajectory is yaw equal to +60°, +30°, 0°, -30°, -60°, or -60°, -30°, 0°, +30°, +60°; Upload all collected data to the backend server; S3, calibrate the right camera external parameters and the right camera internal parameters data collection. The right camera external parameter calibration data collection and internal parameter calibration data collection are the same as the left camera calibration data collection method. After completing the right camera external parameter and internal parameter calibration, upload all collected data to the backend server; S4: After all data is uploaded, the backend server starts calibration calculation. After the calibration calculation is completed, the calibration parameters are uploaded to the calibration device to complete the automatic calibration of the lidar and lens.

9. The method for fully automatic calibration of laser radar and fisheye lens according to claim 8, characterized in that: d 1 is 0.5 meters, d 2 is 2 meters.

10. A fully automatic calibration method for laser radar and fisheye lens according to claim 8 or 9, characterized in that: The process of the backend server starting calibration calculation includes: S401, the backend server completes the internal reference calibration based on the taken photos, and in this process, the cv2.fisheye related modules of OpenCV are used to complete the camera calibration; S402, the backend server completes external parameter calibration based on the taken photos: During the calibration process, the external parameter calibration of the direct relative position of the camera and radar is completed based on the calibration plate. This process requires that the calibration plate be placed in the common view area of ​​the camera and radar, and the camera and radar are used to identify the calibration plate angle respectively; Before the camera recognizes the calibration plate, the image is projected into a panorama through equirectangular projection according to the previously calibrated internal parameters; cv2.cornerSubPix() is called on the panorama to find the coordinates of the corner points of the calibration plate on the panorama; The method of radar recognition calibration plate: first accumulate radar point cloud for t seconds; after t seconds, cut all accumulated point clouds according to the position of the calibration plate, remove irrelevant environmental content, and then project them into a panoramic image through equidistant rectangular projection, using the radar intensity information as the grayscale value of the image; call cv.findChessboardCorners() and cv2.cornerSubPix() on this radar intensity panorama to find the coordinates of the calibration plate corner points on the panorama; After using the camera and radar to identify the corner points of the calibration plate respectively, scipy is used to construct an optimization problem based on the 2D coordinates of the identified corner points of the calibration plate on the image and the 3D coordinates in the radar coordinate system to optimize the horizontal and vertical rotation angles of the two panoramas, as well as the corresponding displacements; finally, an extrinsic parameter transformation matrix from the radar coordinate system to the camera coordinate system is constructed, and the reprojection error is obtained. The calibration result is evaluated based on the error. When the error is less than the preset threshold, the calibration is considered successful.

Images