A roadside laser radar-camera-utm coordinate system joint calibration method and system

Abstract

The application provides a roadside laser radar-camera-UTM coordinate system joint calibration method and system, and the method comprises the following steps: selecting GNSS RTK equipment collection points, and obtaining a UTM coordinate sequence P UTM ; sequentially extracting the GNSS RTK equipment collection points in a laser radar point cloud, and obtaining a coordinate sequence P Lidar under a laser radar coordinate system; sequentially extracting the GNSS RTK equipment collection points in a camera image, and obtaining a coordinate sequence P Pixel under a pixel coordinate system; based on P UTM , P Lidar and P Pixel , an external parameter transformation relationship among the laser radar coordinate system, the camera coordinate system and the UTM coordinate system is calculated through nonlinear optimization, so that the joint external parameter calibration of the roadside laser radar-camera-UTM coordinate system can be more convenient, economical and accurate. The application avoids the high cost of obtaining a high-precision map, and the calibration precision is not lost, and the collection can be carried out for many times at different traffic intersections, the normal traffic is not disturbed, and the safety hidden danger of collection personnel is reduced.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation, specifically relating to a method and system for joint calibration of roadside lidar-camera-UTM coordinate system. Background Technology

[0002] With the development of intelligent transportation, there is a need for intelligent roadside equipment to perceive traffic conditions in real time and accurately. Currently, roadside equipment mainly includes sensors such as LiDAR and RGB cameras. To ensure that LiDAR and RGB cameras have a consistent spatial description of the scene, the extrinsic parameters of LiDAR and RGB cameras need to be calibrated.

[0003] To integrate with upstream and downstream algorithms, the perception information of roadside devices needs to be transformed into a unified coordinate system. In the field of intelligent transportation, the UTM coordinate system is generally chosen as the unified coordinate system. Currently, a common method for transforming roadside devices into the UTM coordinate system is to match LiDAR point clouds with high-precision maps to locate the LiDAR's coordinates in the UTM coordinate system. Then, by using the extrinsic parameter calibration relationship between the LiDAR and the RGB camera, the camera's coordinates in the UTM coordinate system are obtained.

[0004] However, obtaining high-precision maps is extremely difficult, incurring very high economic and policy costs. Establishing high-precision maps at every intersection to calibrate roadside equipment to the UTM coordinate system is a very low-cost-performance solution. Therefore, existing technologies urgently need improvement. Summary of the Invention

[0005] To address the problems existing in the prior art, the purpose of this invention is to propose a joint calibration method, system, and device that can achieve joint extrinsic parameter calibration of the roadside lidar-camera-UTM coordinate system more conveniently, economically, and accurately.

[0006] To achieve the above objectives, the first aspect of the present invention provides a method for calibrating the extrinsic parameters of a roadside lidar-camera-UTM coordinate system, comprising the following steps:

[0007] Select the acquisition points of the GNSS RTK device and obtain the UTM coordinate sequence P. UTM ;

[0008] Extract the GNSS RTK device acquisition points sequentially from the lidar point cloud and obtain the coordinate sequence P in the lidar coordinate system. Lidar ;

[0009] Extract the GNSS RTK device acquisition points sequentially from the camera image and obtain the coordinate sequence P in the pixel coordinate system. Pixel ;

[0010] Based on PUTM P Lidar and P Pixel The external parameter transformation relationship between the lidar coordinate system, camera coordinate system and UTM coordinate system is obtained through nonlinear optimization calculation.

[0011] Furthermore, GNSS RTK equipment acquisition points were selected, and the UTM coordinate sequence P was obtained. UTM This includes: selecting the common area perceived by the camera and LiDAR as the acquisition area; when the extrinsic parameters of the camera and LiDAR are unknown, determining the common area by observing the scene characteristics in the camera image and LiDAR point cloud; selecting at least 3 acquisition points in the common area, with the acquisition points distributed as evenly as possible in the acquisition area and not on the same straight line; converting the Earth's latitude and longitude coordinates into planar coordinates through a projection algorithm, and UTM coordinates are planar coordinates with added altitude information.

[0012] Furthermore, the GNSS RTK device acquisition points are sequentially extracted from the lidar point cloud, and the coordinate sequence P in the lidar coordinate system is obtained. Lidar Includes the following steps:

[0013] Based on the point cloud corresponding to the ground plane, the plane equation is obtained through plane fitting;

[0014] For each acquisition point, the point cloud corresponding to the GNSS RTK device is extracted, and the straight line equation is obtained through line fitting. The GNSS RTK measurement device is in the shape of a straight rod, and it needs to be kept perpendicular to the ground during measurement. The RANSAC algorithm is used to perform line fitting on the GNSS RTK measurement device in the lidar point cloud to obtain the straight line equation.

[0015] Based on the fitted linear equation and the ground plane equation, the coordinates of each acquisition point in the lidar coordinate system are calculated.

[0016] Furthermore, GNSS RTK device acquisition points are sequentially selected in the camera image, and the coordinate sequence P in the pixel coordinate system is obtained. Pixel Includes the following steps:

[0017] Detect GNSS RTK devices in camera images and obtain their accurate locations in the images;

[0018] The intersection of the GNSS RTK device and the ground is the acquisition point, and its pixel coordinates (u, v) are obtained. The pixel coordinates are based on the top left corner of the image as the origin, with the positive direction of the u axis to the right and the positive direction of the v axis downward.

[0019] Furthermore, when detecting GNSS RTK devices in camera images, the detection method adopts target detection algorithms such as YOLO or SSD deep learning. Detection is achieved by labeling and training the GNSS RTK devices in advance, or by manual selection.

[0020] Furthermore, based on P UTM P Lidar and P Pixel The steps to obtain the extrinsic parameter transformation relationship between the lidar coordinate system, the camera coordinate system, and the UTM coordinate system through nonlinear optimization calculation include:

[0021] Based on UTM coordinate sequence P UTM and lidar coordinate sequence P Lidar The external parameter relationship between the lidar coordinate system and the UTM coordinate system is obtained through nonlinear optimization.

[0022] Based on the lidar coordinate sequence P Lidar Projection onto the image and pixel coordinate sequence P Pixel The error was determined by nonlinear optimization to obtain the external parameter relationship between the lidar coordinate system and the camera coordinate system.

[0023] Furthermore, based on the UTM coordinate sequence P UTM and lidar coordinate sequence P Lidar When obtaining the extrinsic parameter relationship between the lidar coordinate system and the UTM coordinate system through nonlinear optimization, a total of n matching point pairs are selected, where the coordinate sequence P UTM and P Lidar Write them in matrix form respectively:

[0024]

[0025] The equation that needs to be optimized is:

[0026]

[0027] in It is the optimal extrinsic parameter obtained from the lidar coordinate system to the UTM coordinate system.

[0028] Furthermore, based on the lidar coordinate sequence P Lidar Projection onto the image and pixel coordinate sequence P Pixei To determine the error, the external parameter relationship between the lidar coordinate system and the camera coordinate system is obtained through nonlinear optimization, including the following steps:

[0029] a) First, obtain the camera's intrinsic parameter matrix K and distortion coefficients D. The camera's intrinsic parameter K is a 3×3 matrix, in the form of:

[0030]

[0031] Where f x and f y c represents the focal length, which is horizontal and vertical. x and c y Represents pixel offset; the distortion coefficient vector D is a 5-dimensional vector with the following form:

[0032] D = [k1, k2, p1, p2, k3]

[0033] Where k1, k2, and k3 are the first three coefficients of radial distortion, and p1 and p2 are the first two coefficients of tangential distortion;

[0034] b) P Lidar Transform the points in the image from the lidar coordinate system to the camera coordinate system:

[0035] c) Perform a projection transformation based on the intrinsic parameter matrix K, and convert it to homogeneous coordinates to obtain... Where Z is KP Camera The third-dimensional coordinates;

[0036] d) Increase the distortion according to the Taylor expressions for radial and tangential distortion, as follows:

[0037] Radial distortion:

[0038] Tangential distortion:

[0039] Where (x0, y0) are the coordinates before distortion, and (x, y) are the coordinates after distortion;

[0040] e) Construct the optimization equation:

[0041]

[0042] Where n is the number of sampling points, and D(·) represents the process of increasing distortion. It is the optimal extrinsic parameter obtained from the lidar coordinate system to the camera coordinate system.

[0043] A second aspect of the present invention provides a joint calibration system, including a data acquisition and uploading module, a target detection and extraction module, and an external parameter calculation and verification module;

[0044] The data acquisition and uploading module uses GNSS RTK equipment to collect latitude, longitude, and altitude information and uploads it to the database; the user inputs camera intrinsic parameters and distortion coefficients and uploads them to the database; the lidar acquires point clouds during sampling and uploads them to the database; the camera acquires images during sampling and uploads them to the database.

[0045] The target detection and extraction module detects and extracts the intersection points between the GNSS RTK device and the ground plane in the point cloud, and detects and extracts the intersection points between the GNSS RTK device and the ground in the image;

[0046] The extrinsic parameter calculation and verification module is used to calculate the extrinsic parameters of the LiDAR coordinate system and the UTM coordinate system, calculate the extrinsic parameters of the LiDAR coordinate system and the camera coordinate system, transform the LiDAR point cloud to the UTM coordinate system and compare it with the high-precision map for verification, and project the LiDAR point cloud onto the image for verification.

[0047] A third aspect of the present invention provides a joint calibration device, comprising a GNSS RTK device, a lidar, a camera, a memory, and a processor; the GNSS RTK device is used to measure and determine the UTM coordinates of a location; the lidar is used to acquire three-dimensional point cloud data of the target device; the camera is used to acquire image data of the target device; the memory is used to store data from the aforementioned devices and user-uploaded data; and the processor is used to execute the steps of the roadside lidar-camera-UTM coordinate system joint calibration method described in the present invention.

[0048] Compared with the prior art, the present invention has at least the following beneficial effects:

[0049] The joint calibration method of roadside lidar-camera-UTM coordinate system described in this invention collects UTM coordinate information of multiple points using a GNSS RTK device, while the lidar and camera collect 3D point cloud and image data at corresponding times. The 3D point cloud and image data of each collected point are detected and extracted to obtain the coordinates of the collected point in the lidar coordinate system and pixel coordinate system. By calculating the coordinate values ​​of each collected point in different coordinate systems, the external parameter transformation relationship between the lidar coordinate system, camera coordinate system, and UTM coordinate system can be obtained. By directly selecting points to collect coordinates using a GNSS RTK device, the high cost of obtaining high-precision maps is eliminated without loss of accuracy. Because the GNSS RTK device is flexible and portable, in the form of a straight stick, it can be used by one person holding it, enabling data collection at different traffic intersections without interfering with normal traffic. This avoids the traffic impact caused by using large calibration boards or calibration vehicles placed in the middle of the road at different poses, and also reduces the safety hazards for data collection personnel. By utilizing data from multiple acquisition points, the extrinsic parameters of the LiDAR and camera can be calibrated. This allows for the selection of more matching point pairs, whereas the traditional method using calibration boards only allows for four matching point pairs per board, and these points are relatively concentrated, which is not conducive to calibration of large-angle roadside views. Therefore, this method can obtain more accurate extrinsic parameters for the LiDAR and camera. Attached Figure Description

[0050] Figure 1 A schematic flowchart illustrating the joint calibration method of roadside lidar-camera-UTM coordinate system for high-precision maps without reliance on external maps.

[0051] Figure 2 A schematic diagram of measurements taken with a handheld GNSS RTK device.

[0052] Figure 3 A schematic flowchart for processing point clouds using lidar.

[0053] Figure 4 This is a schematic diagram showing the matching of the lidar point cloud with the verification high-precision map after calibration.

[0054] Figure 5 This is a schematic diagram showing the projection of the lidar point cloud onto the image after calibration.

[0055] Figure 6 A schematic diagram of the web application interface for the designed joint calibration system. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be noted that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0057] The first aspect of this invention provides a joint calibration method for the lidar-camera-UTM coordinate system. Figure 1 This is a flowchart of the method.

[0058] like Figure 1 As shown, the method may include the following steps:

[0059] 1) Select the GNSS RTK equipment acquisition points and obtain the UTM coordinate sequence P. UTM ;

[0060] 2) Sequentially extract the GNSS RTK device acquisition points from the lidar point cloud and obtain the coordinate sequence P in the lidar coordinate system. Lidar ;

[0061] 3) Sequentially extract the GNSS RTK device acquisition points from the camera image and obtain the coordinate sequence P in the pixel coordinate system. Pixel ;

[0062] 4) Based on P UTM P Lidar and P Pixei The external parameter transformation relationship between the lidar coordinate system, camera coordinate system and UTM coordinate system is obtained through nonlinear optimization calculation.

[0063] It should be noted that the coordinate system concepts used in the embodiments of this invention include: UTM (Universal Transverse Mercator) coordinate system T UTM The lidar coordinate system T Lidar Camera coordinate system T Camera and pixel coordinate system T Pixel The pixel coordinate system T Pixel It is a two-dimensional coordinate system, while the others are three-dimensional coordinate systems.

[0064] In embodiments of the present invention, the transformation from the lidar coordinate system to the UTM coordinate system... Transformation from camera coordinate system to UTM coordinate system These are the parameters that the calibration method in this invention ultimately requires. The transformation parameters are a 4×4 matrix, including a 3×3 rotation matrix and a three-dimensional translation vector, where the unit of the rotation matrix is ​​radians and the unit of the translation vector is meters.

[0065] According to an embodiment of the present invention, GNSS RTK equipment acquisition points are selected, and UTM coordinate sequence P is obtained. UTM This includes selecting the common area perceived by the camera and LiDAR as the acquisition area. It should be noted that when the external parameters of the camera and LiDAR are unknown, it is necessary to determine the common area by observing the scene characteristics in the camera image and LiDAR point cloud.

[0066] According to an embodiment of the present invention, GNSS RTK equipment acquisition points are selected, and UTM coordinate sequence P is obtained. UTM This also includes selecting at least three sampling points in a public area. These sampling points should be distributed as evenly as possible within the sampling area and should not lie on a straight line. It should be noted that the more sampling points there are, the more accurate the final calibration result will be. The projections of the sampling points onto the image should be distributed as evenly as possible, avoiding clustering at the image edges.

[0067] According to an embodiment of the present invention, GNSS RTK equipment acquisition points are selected, and UTM coordinate sequence P is obtained. UTMThis also includes: acquiring latitude, longitude, and altitude information for each acquisition point via GNSS RTK equipment, and converting this information to obtain the corresponding UTM coordinates (E, N, H). It should be noted that the UTM coordinate system is a projected coordinate system, which is a planar coordinate system converted from Earth's latitude and longitude coordinates using a projection algorithm. The UTM coordinate system divides the Earth into 60 regions along the east-west direction (longitude intervals 1-60) and 20 regions along the north-south direction (latitude intervals CX) from 80°S to 84°N. Each latitude and longitude grid is uniquely identified, called a coordinate zone. Once the latitude, longitude, and local UTM coordinate zone are known, the latitude and longitude can be converted to UTM coordinates, i.e., (E, N). In the embodiments of this invention, the UTM coordinates used are based on the above-mentioned two-dimensional UTM coordinate system with the addition of altitude information, thus becoming a three-dimensional coordinate system, i.e., (E, N, H), which makes it easier to describe its conversion relationship with the lidar coordinate system and camera coordinate system.

[0068] According to one embodiment of the present invention, GNSS RTK device acquisition points are sequentially extracted from the lidar point cloud, and a coordinate sequence P in the lidar coordinate system is obtained. Lidar Includes the following steps:

[0069] 1) Based on the point cloud corresponding to the ground plane, the plane equation is obtained through plane fitting. In one embodiment of the present invention, the plane fitting adopts the RANSAC (Random Sample Consensus) algorithm, and the obtained plane equation is Ax+By+Cz+D=0.

[0070] 2) For each acquisition point, the point cloud corresponding to the GNSS RTK device is extracted, and the linear equation is obtained through line fitting. It should be noted that the GNSS RTK measuring device is a straight rod with a level on it; the device must be kept perpendicular to the ground during measurement. Therefore, in one embodiment of this invention, the RANSAC algorithm is used to perform line fitting on the GNSS RTK measuring device in the lidar point cloud, obtaining the linear equation as follows: Where a0, b0, c0 ≠ 0.

[0071] 3) Based on the fitted linear equation and the ground plane equation, the coordinates (x, y, z) of each acquisition point in the lidar coordinate system are calculated. In this embodiment of the invention, the acquisition point is the intersection of the linear equation and the ground plane equation. By solving the two equations simultaneously, its coordinates can be obtained as follows:

[0072]

[0073] According to one embodiment of the present invention, GNSS RTK device acquisition points are sequentially selected in the camera image, and a coordinate sequence P in the pixel coordinate system is obtained. Pixel Includes the following steps:

[0074] 1) Detecting GNSS RTK devices in the image. Detection methods can employ deep learning-based object detection algorithms such as YOLO and SSD, which can be achieved by pre-labeling and training the GNSS RTK devices. Alternatively, detection can be performed manually. While this method requires manual intervention, it is more accurate and saves on time and computing power. In one embodiment of this invention, manual selection is used to locate the precise position of the GNSS RTK device in the image.

[0075] 2) The intersection of the GNSS RTK device and the ground is the acquisition point, and its pixel coordinates (u, v) are obtained. In one embodiment of the present invention, the pixel coordinates are defined with the upper left corner of the image as the origin, the positive direction of the u-axis to the right, and the positive direction of the v-axis downwards.

[0076] According to an embodiment of the present invention, based on P UTM P Lidar and P Pixel The steps to obtain the extrinsic parameter transformation relationship between the lidar coordinate system, the camera coordinate system, and the UTM coordinate system through nonlinear optimization calculation include:

[0077] 1) Based on UTM coordinate sequence P UTM and lidar coordinate sequence P Lidar The extrinsic parameter relationship between the lidar coordinate system and the UTM coordinate system is obtained through nonlinear optimization. In one embodiment of the present invention, a total of n matching point pairs are selected, wherein the coordinate sequence P UTM and P Lidar They can be written in matrix form respectively:

[0078]

[0079] The equation that needs to be optimized is

[0080]

[0081] in It is the optimal extrinsic parameter obtained from the lidar coordinate system to the UTM coordinate system.

[0082] 2) Based on the lidar coordinate sequence P Lidar Projection onto the image and pixel coordinate sequence P Pixel The error is determined by nonlinear optimization to obtain the extrinsic parameter relationship between the lidar coordinate system and the camera coordinate system. In one embodiment of the present invention, it is necessary to:

[0083] a) First, obtain the camera's intrinsic parameter matrix K and distortion coefficients D. The camera's intrinsic parameter K is a 3×3 matrix, in the form of:

[0084]

[0085] Where f x and f y c represents the focal length, which is horizontal and vertical. x and c y Represents pixel offset; the distortion coefficient vector D is a 5-dimensional vector with the following form:

[0086] D = [k1, k2, p1, p2, k3]

[0087] Where k1, k2, and k3 are the first three coefficients of radial distortion, and p1 and p2 are the first two coefficients of tangential distortion.

[0088] b) P Lidar Transform the points in the image from the lidar coordinate system to the camera coordinate system:

[0089] c) Perform a projection transformation based on the intrinsic parameter matrix K, and convert it to homogeneous coordinates to obtain... Where Z is KP Camera The third-dimensional coordinates.

[0090] d) Increase the distortion according to the Taylor expressions for radial and tangential distortion, as follows:

[0091] Radial distortion:

[0092] Tangential distortion: Where (x0, y0) are the coordinates before distortion, and (x, y) are the coordinates after distortion.

[0093] e) Construct the optimization equation:

[0094]

[0095] Where n is the number of sampling points, and D(·) represents the process of increasing distortion. It is the optimal extrinsic parameter obtained from the lidar coordinate system to the camera coordinate system.

[0096] A second aspect of the present invention provides a joint calibration system, the system comprising:

[0097] 1) Data Acquisition and Upload Module: Uses GNSS RTK equipment to collect latitude, longitude, and altitude information and uploads it to the database; users input camera intrinsic parameters and distortion coefficients and upload them to the database; LiDAR acquires point clouds during sampling and uploads them to the database; the camera acquires images during sampling and uploads them to the database.

[0098] 2) Target detection and extraction module: Detects and extracts the intersection points of GNSS RTK devices and the ground plane in the point cloud, and detects and extracts the intersection points of GNSS RTK devices and the ground in the image.

[0099] 3) Extrinsic parameter calculation and verification module: Calculates the extrinsic parameters of the LiDAR coordinate system and the UTM coordinate system, calculates the extrinsic parameters of the LiDAR coordinate system and the camera coordinate system, transforms the LiDAR point cloud into the UTM coordinate system and compares it with the high-precision map for verification, and projects the LiDAR point cloud onto the image for verification.

[0100] A third aspect of the present invention provides a joint calibration apparatus comprising: a GNSS RTK device for measuring and determining the UTM coordinates of a location; a lidar for acquiring three-dimensional point cloud data of a target device; a camera for acquiring image data of the target device; a memory for storing data from the aforementioned devices and user-uploaded data; and a processor for executing the aforementioned method steps and calculating extrinsic parameters.

Claims

1. A method for joint calibration of roadside lidar-camera-UTM coordinate system, characterized in that, Includes the following steps: Select GNSS RTK device acquisition points and obtain UTM coordinate sequences. ; Extract the GNSS RTK device acquisition points sequentially from the lidar point cloud and obtain the coordinate sequence in the lidar coordinate system. ; Extract the GNSS RTK device acquisition points sequentially from the camera images and obtain the coordinate sequence in the pixel coordinate system. ; based on , and The external parameter transformation relationship between the lidar coordinate system, camera coordinate system, and UTM coordinate system is obtained through nonlinear optimization calculation, including: based on UTM coordinate sequence. and lidar coordinate sequence The extrinsic parameter relationship between the lidar coordinate system and the UTM coordinate system was obtained through nonlinear optimization; specifically, a total of [number] parameters were selected. A set of matching point pairs, where the coordinate sequence is... and Write them in matrix form respectively: The equation that needs to be optimized is: in It is the optimal extrinsic parameter obtained from the lidar coordinate system to the UTM coordinate system; Based on lidar coordinate sequence Projection on the image and pixel coordinate sequence The error is determined by nonlinear optimization to obtain the extrinsic parameter relationship between the lidar coordinate system and the camera coordinate system, including the following steps: a) First, obtain the camera's intrinsic parameter matrix. and distortion coefficient Obtain camera internal parameters K It is a 3×3 matrix, in the form of: in and The focal length represents the horizontal and vertical directions. and Represents pixel offset; obtains distortion coefficient vector D It is a 5-dimensional vector, in the form of: Where k1, k2, and k3 are the first three coefficients of radial distortion, and p1 and p2 are the first two coefficients of tangential distortion; b) Transform the points in the image from the lidar coordinate system to the camera coordinate system: c) Based on the intrinsic parameter matrix Perform a projection transformation and convert to homogeneous coordinates to obtain... ,in for The third-dimensional coordinates; d) Increase the distortion according to the Taylor expressions for radial and tangential distortion, as follows: Radial distortion: Tangential distortion: in The coordinates before distortion. The coordinates are after distortion; e) Construct the optimization equation: in The number of sampling points. This represents the process of increasing distortion. It is the optimal extrinsic parameter obtained from the lidar coordinate system to the camera coordinate system.

2. The roadside lidar-camera-UTM coordinate system joint calibration method according to claim 1, characterized in that, Select GNSS RTK device acquisition points and obtain UTM coordinate sequences. This includes: selecting the common area perceived by the camera and LiDAR as the acquisition area; when the extrinsic parameters of the camera and LiDAR are unknown, determining the common area by observing the scene characteristics in the camera image and LiDAR point cloud; selecting at least 3 acquisition points in the common area, with the acquisition points evenly distributed in the acquisition area and not on the same straight line; converting the Earth's latitude and longitude coordinates into planar coordinates through a projection algorithm, and UTM coordinates are planar coordinates with added altitude information.

3. The method for joint calibration of roadside lidar-camera-UTM coordinate system according to claim 1, characterized in that, The GNSS RTK device acquisition points are sequentially extracted from the lidar point cloud, and the coordinate sequence in the lidar coordinate system is obtained. Includes the following steps: Based on the point cloud corresponding to the ground plane, the plane equation is obtained through plane fitting; For each acquisition point, the point cloud corresponding to the GNSS RTK device is extracted, and the straight line equation is obtained through line fitting. The GNSS RTK measurement device is in the shape of a straight rod, and it needs to be kept perpendicular to the ground during measurement. The RANSAC algorithm is used to perform line fitting on the GNSS RTK measurement device in the lidar point cloud to obtain the straight line equation. Based on the fitted linear equation and the ground plane equation, the coordinates of each acquisition point in the lidar coordinate system are calculated.

4. The method for joint calibration of roadside lidar-camera-UTM coordinate system according to claim 1, characterized in that, Select GNSS RTK device acquisition points sequentially in the camera image and obtain the coordinate sequence in the pixel coordinate system. Includes the following steps: Detect GNSS RTK devices in camera images and obtain their accurate locations in the images; The intersection of the GNSS RTK device and the ground is identified as the data acquisition point, and its pixel coordinates are obtained. The pixel coordinates are based on the top left corner of the image as the origin, with the positive u-axis pointing to the right and the positive v-axis pointing downwards.

5. The method for joint calibration of roadside lidar-camera-UTM coordinate system according to claim 1, characterized in that, When detecting GNSS RTK devices in camera images, the detection method uses YOLO or SSD deep learning-based target detection algorithms. Detection is achieved by labeling and training the GNSS RTK devices in advance, or by manual selection.

6. A roadside lidar-camera-UTM coordinate system joint calibration system, used to implement the roadside lidar-camera-UTM coordinate system joint calibration method as described in claim 1, characterized in that, It includes a data acquisition and uploading module, a target detection and extraction module, and an external parameter calculation and verification module; The data acquisition and uploading module uses GNSS RTK equipment to collect latitude, longitude, and altitude information and uploads it to the database; the user inputs camera intrinsic parameters and distortion coefficients and uploads them to the database; the lidar acquires point clouds during sampling and uploads them to the database; the camera acquires images during sampling and uploads them to the database. The target detection and extraction module detects and extracts the intersection points between the GNSS RTK device and the ground plane in the point cloud, and detects and extracts the intersection points between the GNSS RTK device and the ground in the image; The extrinsic parameter calculation and verification module is used to calculate the extrinsic parameters of the LiDAR coordinate system and the UTM coordinate system, calculate the extrinsic parameters of the LiDAR coordinate system and the camera coordinate system, transform the LiDAR point cloud to the UTM coordinate system and compare it with the high-precision map for verification, and project the LiDAR point cloud onto the image for verification.

7. A combined calibration device, characterized in that, It includes GNSS RTK equipment, lidar, camera, memory, and processor; the GNSS RTK equipment is used to measure and determine the UTM coordinates of a location; the lidar is used to acquire 3D point cloud data of the target equipment; the camera is used to acquire image data of the target equipment; and the memory is used to store data from the above devices and user-uploaded data. A processor for executing the steps of the roadside lidar-camera-UTM coordinate system joint calibration method according to any one of claims 1-5.

Images