High-precision map-based roadside perception device joint calibration method and system

Abstract

The embodiment of the application discloses a kind of based on high-precision map's roadside perception device joint calibration method, system, which based on high-precision map's roadside perception device joint calibration method includes: based on three-dimensional point cloud, utilize the roadside laser radar and movable instant positioning and map construction SLAM laser radar, determine first calibration parameter;Based on the three-dimensional point cloud of high-precision map, utilize the roadside camera and visual interactive device, determine second calibration parameter;Based on the first calibration parameter and the second calibration parameter, determine the joint calibration parameter of the roadside perception device. Thus, in the case where no additional hardware is added at the vehicle end, any target in the natural scene within the roadside perception area can be identified, and the joint calibration of the high-precision roadside perception device is completed.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of location technology, and in particular to a method and system for joint calibration of roadside sensing devices based on high-precision maps. Background Technology

[0002] With the development of autonomous driving technology, the requirements for the application and implementation of artificial intelligence in autonomous driving are becoming increasingly stringent. The development of roadside multi-source fusion perception system capabilities is a key area of ​​focus for vehicle-road cooperative autonomous driving.

[0003] When initializing and deploying a vehicle-road cooperative roadside multi-source fusion sensing system, spatial synchronization using roadside sensing devices is required. In related technologies, the device calibration method involves collecting data on-site at the roadside, then bringing the collected data back to the laboratory for calibration parameter calculation. This method makes it difficult to obtain data at optimal sampling points, hindering dynamic parameter adjustment and making it difficult to achieve productization and large-scale market deployment in the later stages of the project. Summary of the Invention

[0004] In view of this, embodiments of this application provide at least one method and system for joint calibration of roadside sensing devices based on high-precision maps.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a joint calibration method for roadside sensing devices based on high-precision maps. The roadside sensing devices include roadside lidar and roadside cameras. The joint calibration method for roadside sensing devices based on high-precision maps includes:

[0007] A high-precision map based on 3D point clouds is constructed using the roadside lidar and a mobile real-time positioning and mapping system to determine the first calibration parameters. The high-precision map of 3D point clouds is constructed based on the 3D point clouds of the natural scene in the roadside sensing area. The 3D point clouds are collected by the mobile SLAM lidar. The first calibration parameters are the transformation parameters between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the 3D point clouds.

[0008] Based on the high-precision map of the 3D point cloud, the second calibration parameter is determined using the roadside camera and the visualization interaction device; the second calibration parameter is the transformation parameter between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera.

[0009] Based on the first calibration parameters and the second calibration parameters, the joint calibration parameters of the roadside sensing device are determined; the joint calibration parameters are the transformation parameters between the coordinate system corresponding to the roadside lidar and the pixel coordinate system corresponding to the image of the roadside camera.

[0010] This application provides a joint calibration system for roadside sensing devices based on high-precision maps. The system includes: roadside sensing devices, a mobile SLAM lidar, a visualization and interactive device, and a data processing unit; wherein...

[0011] The roadside sensing equipment includes roadside lidar and roadside cameras;

[0012] The roadside lidar is used to collect radar data of natural scenes;

[0013] The roadside camera is used to collect image data of the natural scene;

[0014] The mobile SLAM lidar is used to collect radar data along the SLAM lidar's moving path;

[0015] The visualization and interactive device is used to visualize and display a high-precision map of 3D point clouds, radar data collected by the roadside lidar, radar data collected by the mobile SLAM lidar, and image data collected by the roadside camera; and to provide visual instructions and interactive operations.

[0016] The data processing unit is used to determine a first calibration parameter based on the high-precision map of the 3D point cloud, the radar data collected by the roadside lidar, and the radar data collected by the mobile SLAM lidar. The high-precision map of the 3D point cloud is constructed based on the 3D point cloud of the natural scene in the roadside sensing area, and the 3D point cloud is collected by the mobile SLAM lidar. The first calibration parameter is a transformation parameter between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the 3D point cloud. Based on the high-precision map of the 3D point cloud, the image data collected by the roadside camera, and the radar data collected by the mobile SLAM lidar, a second calibration parameter is determined. The second calibration parameter is a transformation parameter between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera. Based on the first calibration parameter and the second calibration parameter, a joint calibration parameter for the roadside sensing device is determined. The joint calibration parameter is a transformation parameter between the coordinate system corresponding to the roadside lidar and the pixel coordinate system corresponding to the image of the roadside camera.

[0017] According to the technical solution provided in this application, a high-precision map of a 3D point cloud constructed based on the natural scene of the roadside perception area is generated. First calibration parameters are determined using roadside LiDAR and a mobile SLAM LiDAR. Second calibration parameters are determined using a roadside camera and a visual interactive device. Finally, based on the first and second calibration parameters, joint calibration parameters for the roadside perception devices are determined. This allows for the identification of any target in the natural scene within the roadside perception area without adding additional hardware to the vehicle, thus achieving high-precision joint calibration of the roadside perception devices.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0019] Figure 1 A schematic diagram of the implementation process of a joint calibration method for roadside sensing devices based on high-precision maps provided in this application embodiment. Figure 1 ;

[0020] Figure 2A A schematic diagram illustrating the implementation architecture of a joint calibration system for roadside sensing devices based on a high-precision map, provided in an embodiment of this application;

[0021] Figure 2B A schematic diagram of the composition of a backpack device in a roadside sensing device joint calibration system based on a high-precision map, provided as an embodiment of this application;

[0022] Figure 2C A schematic diagram of the implementation process of a joint calibration method for roadside sensing devices based on high-precision maps provided in this application embodiment;

[0023] Figure 3 This is a schematic diagram of the composition structure of a roadside sensing device joint calibration system based on a high-precision map, provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the following description, the terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first / second / third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application. It should also be noted that, for ease of description, only the parts relevant to the application are shown in the accompanying drawings.

[0026] Based on this, embodiments of this application provide a joint calibration method for roadside sensing devices based on high-precision maps. This joint calibration method for roadside sensing devices based on high-precision maps can be executed by a processor. The processor may include, but is not limited to, a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), or a field-programmable gate array (FPGA). In some embodiments, this joint calibration method for roadside sensing devices based on high-precision maps can be applied to a joint calibration system for roadside sensing devices based on high-precision maps. This joint calibration system for roadside sensing devices may include: roadside sensing devices, a mobile SLAM lidar, a visual interactive device, and a data processing unit; wherein, the roadside sensing devices may include, but are not limited to, roadside lidar and roadside cameras. The joint calibration method for roadside sensing devices based on high-precision maps can be executed by the data processing unit, or jointly executed by the processor in the roadside sensing device, the processor in the mobile SLAM lidar, and / or the processor in the visualization and interactive device, together with the data processing unit. This application embodiment is not limited to this method.

[0027] Figure 1 A schematic diagram of the implementation process of a joint calibration method for roadside sensing devices based on high-precision maps provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the joint calibration method for roadside sensing devices based on high-precision maps includes the following steps S101 to S103:

[0028] Step S101: Based on the high-precision map of the 3D point cloud, a SLAM LiDAR is constructed using roadside LiDAR and a mobile real-time positioning and mapping system to determine the first calibration parameters. The high-precision map of the 3D point cloud is constructed based on the 3D point cloud of the natural scene in the roadside sensing area. The 3D point cloud is collected by the mobile SLAM LiDAR. The first calibration parameters are the transformation parameters between the coordinate system corresponding to the roadside LiDAR and the coordinate system corresponding to the 3D point cloud.

[0029] The roadside sensing area refers to the area covered by the sensing range of the roadside sensing device. In some implementations, the roadside sensing area can be determined based on the sensing data collected by the roadside sensing device.

[0030] The natural scene in the roadside sensing area includes scene information within the area covered by the sensing range of the roadside sensing device. This scene information includes, but is not limited to, environmental information of the road scene, dynamic targets and / or static targets at the road scene.

[0031] The 3D point cloud is a 3D laser point cloud obtained by scanning the roadside sensing area using a Simultaneous Localization and Mapping (SLAM) lidar.

[0032] In some implementations, roadside lidar can be installed at locations including but not limited to roadside poles, light poles, etc.

[0033] In some implementations, the SLAM lidar can be mounted on a mobile device, including but not limited to robots, autonomous driving devices, backpack devices, etc.; the SLAM lidar can be a mechanical rotating lidar, including but not limited to a 16-line mechanical rotating lidar.

[0034] In some implementations, the first calibration parameter can be a projection matrix that includes three rotational degrees of freedom and three translational degrees of freedom.

[0035] In some implementations, a data processing unit can calculate the transformation parameters between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the 3D point cloud, based on radar data collected by the roadside lidar, radar data collected by the mobile SLAM lidar, and a high-precision map of the 3D point cloud constructed using the mobile SLAM lidar. These transformation parameters are then used as the first calibration parameters. The radar data from the mobile SLAM lidar may include, but is not limited to, the laser point cloud from the mobile SLAM lidar and its positioning data. It is understood that the laser point cloud from the mobile SLAM lidar corresponds to the 3D point cloud in the high-precision map constructed using the mobile SLAM lidar. Therefore, the transformation parameters between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the 3D point cloud are also the transformation parameters between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the mobile SLAM lidar.

[0036] In some implementations, the construction of a high-precision map of a 3D point cloud is performed simultaneously with the determination of the first calibration parameters.

[0037] Step S102: Based on the high-precision map of the 3D point cloud, determine the second calibration parameters using a roadside camera and a visualization interaction device; the second calibration parameters are the transformation parameters between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera.

[0038] Visual interactive devices include, but are not limited to, flat panel displays and laptops.

[0039] In some implementations, roadside cameras can be installed at locations including, but not limited to, roadside poles, light poles, etc.

[0040] In some implementations, the visual interactive device can be mounted on a mobile device or held by a person.

[0041] In some implementations, the second calibration parameter can be a projection matrix that includes three rotational degrees of freedom and three translational degrees of freedom.

[0042] In some implementations, a data processing unit can calculate the transformation parameters between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera, based on the image data collected by the roadside camera and the high-precision map of the 3D point cloud, and use the transformation parameters between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera as the second calibration parameter.

[0043] In some implementations, the construction of a high-precision map of a 3D point cloud is performed simultaneously with the determination of the second calibration parameters.

[0044] Step S103: Based on the first calibration parameters and the second calibration parameters, determine the joint calibration parameters of the roadside sensing device; the joint calibration parameters are the transformation parameters between the coordinate system corresponding to the roadside lidar and the pixel coordinate system corresponding to the image of the roadside camera.

[0045] In some implementations, the data processing unit can use the first calibration parameters and the second calibration parameters to convert the coordinate system corresponding to the roadside lidar into the pixel coordinate system corresponding to the image of the roadside camera, and determine the joint calibration parameters of the roadside sensing device based on the conversion result of converting the coordinate system corresponding to the roadside lidar into the pixel coordinate system corresponding to the image of the roadside camera.

[0046] In some implementations, the construction of a high-precision map of a 3D point cloud is carried out simultaneously with the determination of joint calibration parameters for roadside sensing devices.

[0047] In this embodiment, based on a high-precision map of the 3D point cloud of the roadside sensing area, first calibration parameters are determined using roadside LiDAR and a mobile SLAM LiDAR. Second calibration parameters are determined using a roadside camera and a visual interactive device. Finally, based on the first and second calibration parameters, joint calibration parameters for the roadside sensing devices are determined. This allows for the identification of any target in the natural scene within the roadside sensing area without adding any additional hardware to the vehicle, achieving high-precision joint calibration of the roadside sensing devices.

[0048] In some embodiments, a portable SLAM lidar and a visual interactive device are integrated into a backpack device, which also includes a data processing unit, a communication unit, and a positioning terminal. Prior to step S101, the method may further include steps S111 to S115:

[0049] Step S111: Use the communication unit to connect to the roadside sensing device and obtain the raw sensing data from the roadside sensing device.

[0050] The communication unit is used for data communication between various functional modules, devices or units inside and outside the backpack device. For example, it is used to configure the connection information of the roadside camera, or to transmit the raw data acquired from the roadside sensing device to the visual interactive device.

[0051] The raw sensing data from roadside sensing devices includes, but is not limited to, images captured by roadside cameras and laser point clouds obtained from roadside lidar scanning.

[0052] Step S112: Using a visual interactive device, determine the roadside sensing area based on the original sensing data.

[0053] The roadside sensing area refers to the sensing area of ​​roadside sensing devices such as roadside cameras and / or roadside lidar at the road site, including but not limited to the range, start point, end point, and edge of the sensed road site.

[0054] In some implementations, the roadside sensing area can be determined by manual selection in real time on a visual interactive device. For example, an operator can mark all highway areas in the images captured by roadside cameras as the roadside sensing area on a flat panel display showing images of the road scene.

[0055] Step S113: Using the data processing unit, determine the target's movement path based on the roadside sensing area and the sensing capability range of the SLAM lidar.

[0056] The target movement path is the movement path of the backpack device within the roadside sensing area.

[0057] In some implementations, the range and direction of the target's movement path can be determined based on the roadside sensing area and the sensing capability range of the SLAM lidar. For example, the start and end points of the roadside sensing area can be used as the start and end points of the target's movement path; another example is that the target's movement path turns at a bend in the roadside sensing area; yet another example is that the maximum sensing capability range of the SLAM lidar can be used as the minimum width of the target's movement path.

[0058] In some implementations, the target movement path can be automatically generated by the data processing unit, or it can be generated manually by selecting the path's range and / or direction based on the roadside sensing area and the SLAM lidar's sensing range using a visual interactive device. For example, the data processing unit can automatically generate a reference path that runs through the entire roadside sensing area, based on the roadside sensing area and the SLAM lidar's sensing range, as the backpack device's movement path.

[0059] In some implementations, after the target movement path is generated by the data processing unit, the robot can obtain the reference path and the backpack device can automatically move along the reference path.

[0060] In some implementations, after the target movement path is generated by the data processing unit, the target movement path can be visualized through a visual interactive device, and a person carrying a backpack moves along the target movement path in real time based on the visualization display result of the visual interactive device.

[0061] In some implementations, after the target movement path is generated by the data processing unit, the target movement path can be converted into voice instructions by a voice navigation device, and the backpack device carried by the operator can move along the target movement path in real time based on the voice instructions of the voice navigation device.

[0062] In some implementations, the camera intrinsics of the roadside camera are pre-set in the data processing unit so that the data processing unit can acquire image data from the roadside camera based on the camera intrinsics. The image data may include, but is not limited to, pixel coordinate data of the image.

[0063] Step S114: Use the positioning terminal to acquire the positioning data of the SLAM lidar moving along the target's moving path.

[0064] The positioning terminal may include, but is not limited to, a real-time kinematic (RTK) positioning terminal, and the positioning data may include, but is not limited to, latitude and longitude coordinates, elevation information, etc.

[0065] Step S115: Using the data processing unit, a high-precision map of three-dimensional point cloud is constructed based on the positioning data and radar data collected by the SLAM lidar along the target's movement path.

[0066] Among them, through the data processing unit, based on the positioning data obtained by the positioning terminal and the radar data collected by the SLAM lidar, a high-precision three-dimensional laser point cloud map of the roadside perception area is constructed using the SLAM algorithm and the principle of four points coplanarity.

[0067] In some implementations, as the SLAM lidar moves along the target's path, its speed can be kept at a stable value. Several points along the target's path are randomly selected as markers. A data processing unit then uses the SLAM algorithm and the principle of four points being coplanar to construct a high-precision three-dimensional map of the roadside sensing area using the lidar point cloud. These markers can be selected automatically or manually via a visual interactive device.

[0068] In some implementations, as the SLAM lidar moves along the target's path, the recording module in the visual interactive device can be triggered to switch from a waiting state to a recording state. The positioning terminal then acquires the positioning data of the marker points, and the recording module records this data. In this case, each 3D lidar point cloud in the constructed high-precision map corresponds to a set of positioning data.

[0069] In this embodiment, a mobile SLAM lidar, a visualization interaction device, a data processing unit, a communication unit, and a positioning terminal are integrated into a backpack device. The communication unit connects to roadside sensing devices to acquire raw sensing data. The visualization interaction device determines the roadside sensing area. The data processing unit determines the target's movement path. The positioning terminal acquires the positioning data of the SLAM lidar moving along the target's movement path. The data processing unit uses the SLAM algorithm and the principle of four points being coplanar to construct a high-precision three-dimensional laser point cloud map of the roadside sensing area. Thus, through this backpack-integrated joint calibration and point collection device, a high-precision roadside map can be constructed in real time without adding additional hardware to the vehicle.

[0070] In some embodiments, the backpack device further includes a balancing device, and step S101 may include steps S116 to S119:

[0071] Step S116: Based on the high-precision map of 3D point cloud, use a visualization interactive device to instruct the backpack device to move directly below the roadside lidar.

[0072] The system uses a visual interactive device to display a high-precision map of a 3D point cloud that is built in real time, and indicates the location of the roadside lidar on the map. The backpack device moves based on the location indication of the roadside lidar.

[0073] In some implementations, the backpack device can be moved by a robot or by a person carrying the backpack device.

[0074] Step S117: Use a balancing device to keep the backpack horizontal during movement.

[0075] Among them, the balancing device is used to adjust the posture of the backpack device, and the balancing device may include, but is not limited to, a balancer.

[0076] Step S118: Using the communication unit, acquire multiple sets of synchronized data collected during the movement of the backpack device in real time; wherein, each set of synchronized data includes radar data collected by the roadside lidar and SLAM lidar respectively.

[0077] The synchronized data includes radar data collected by the roadside lidar and SLAM lidar respectively when the backpack device moves directly under the roadside lidar.

[0078] Step S119: Using the data processing unit, determine the first calibration parameter based on multiple sets of synchronized data.

[0079] The data processing unit is used to perform coordinate conversion on the radar data collected by the roadside lidar and SLAM lidar, converting the data in the coordinate system corresponding to the roadside lidar into the coordinate system corresponding to the three-dimensional point cloud, and determining the first calibration parameter based on the converted data in the coordinate system corresponding to the three-dimensional point cloud.

[0080] In this embodiment, a balancing device is also integrated into the backpack device. A visual interactive device instructs the backpack device to move directly beneath the roadside lidar. The balancing device keeps the backpack device horizontal during movement. A communication unit acquires real-time synchronized data from multiple sets of roadside lidar and SLAM lidar during the backpack device's movement. A data processing unit determines the first calibration parameters. Thus, through this integrated backpack calibration and point-collecting device, automatic conversion from the roadside lidar coordinate system to a 3D point cloud coordinate system can be achieved based on real-time synchronized SLAM lidar and roadside lidar data.

[0081] In some implementations, determining the first calibration parameter based on multiple sets of synchronization data in step S119 above may include the following steps S121 to S124:

[0082] Step S121: Determine the conversion parameters between each group of synchronized data as the first candidate calibration parameters;

[0083] Specifically, after acquiring the synchronized data, the data processing unit determines the transformation parameters for converting the data in the coordinate system corresponding to the roadside lidar in the synchronized data into the coordinate system corresponding to the three-dimensional point cloud as the first candidate calibration parameters.

[0084] Step S122: For each set of synchronized data, perform road surface fitting on the set of synchronized data to obtain the first roadside normal vector and the knapsack normal vector of the road surface; the first roadside normal vector is the normal vector in the coordinate system corresponding to the roadside lidar, and the knapsack normal vector is the normal vector in the coordinate system corresponding to the three-dimensional point cloud.

[0085] In each set of fitted data, the first roadside normal vector of the road surface is the normal vector of the fitted road surface constructed by the roadside lidar, and the knapsack normal vector of the road surface in each set of fitted data is the normal vector of the fitted road surface constructed by the SLAM lidar. Based on the two synchronous data of each set, a first roadside normal vector and a knapsack normal vector can be obtained.

[0086] Step S123: Based on the first candidate calibration parameters, convert the first roadside normal vector into the second roadside normal vector in the coordinate system corresponding to the three-dimensional point cloud.

[0087] The second path-side normal vector is the normal vector in the coordinate system corresponding to the 3D point cloud, calculated by the data processing unit using the first candidate calibration parameter. Based on the two synchronized data sets in each group, one first candidate calibration parameter and one second path-side normal vector can be determined. The number of synchronized data sets is equal to the number of first candidate calibration parameters and also the number of second path-side normal vectors.

[0088] For example, the conversion process between the first roadside normal vector and the second roadside normal vector is shown in formula (1):

[0089]

[0090] in, This is the first path side normal vector. This is the second-side normal vector. The first candidate calibration parameters are α, β, and γ, which are the three rotational degrees of freedom, and T. x T y T z Each has three translational degrees of freedom.

[0091] Step S124: Based on the error between the second roadside normal vector and the knapsack normal vector, determine the first calibration parameter from multiple sets of first candidate calibration parameters.

[0092] Specifically, based on the second pathside normal vector and knapsack normal vector obtained from two sets of synchronized data, an error can be determined. Multiple sets of synchronized data correspond to multiple errors, and each error corresponds to a first candidate calibration parameter that converts the first pathside normal vector into the second pathside normal vector. The number of sets of synchronized data is equal to the number of errors. Through the data processing unit, based on multiple errors, the first candidate calibration parameter corresponding to the error that meets specific requirements is determined as the first calibration parameter.

[0093] In some implementations, the inverse cosine function can be used to calculate the inverse cosine of the angle between the second roadside normal vector and the knapsack normal vector as the error, and the first candidate calibration parameter corresponding to the smallest absolute value of the error can be selected as the first calibration parameter.

[0094] In some implementations, during the process of adjusting the horizontal direction using a balancing device, the levelness of the backpack device can be assessed based on the magnitude of the error between the second-side normal vector and the backpack normal vector. For example, the levelness can be divided into three levels: red, yellow, and green, and displayed on a visual interactive device; where the levelness is displayed as green when the error value is no greater than 1, as yellow when the error value is greater than 1 but not greater than 3, and as red when the error value is greater than 3.

[0095] In this embodiment, a data processing unit determines a first candidate calibration parameter based on real-time synchronous data. Using this first candidate calibration parameter, a first roadside normal vector for road surface fitting based on synchronous data is converted into a second roadside normal vector in the coordinate system corresponding to the 3D point cloud. The error between the second roadside normal vector and the knapsack normal vector for road surface fitting based on synchronous data is evaluated. From multiple sets of first candidate calibration parameters, a first calibration parameter is determined. This improves the accuracy of the first calibration parameter in real time.

[0096] In some implementations, step S102 may include steps S131 to S133:

[0097] Step S131: Connect the roadside camera using the communication unit.

[0098] Step S132: Using roadside cameras and visual interactive devices, real-time acquisition of matching points between images and 3D point clouds collected by multiple sets of roadside cameras.

[0099] In this method, a high-precision map of 3D point cloud and images captured by roadside cameras are displayed on the same screen using a visualization interactive device, and matching points are manually selected from the high-precision map of 3D point cloud and the images captured by roadside cameras.

[0100] In some implementations, each time a set of high-precision maps of 3D point clouds and images captured by roadside cameras are displayed through a visual interactive device, a set of matching points is manually selected. The operation of displaying maps and images through the visual interactive device and the point selection operation are repeatedly executed to complete the acquisition of multiple sets of matching points.

[0101] Step S133: Using the data processing unit, based on the matching points, the matching point conversion algorithm is used to determine the second calibration parameters; the matching point conversion algorithm is used to calculate the conversion relationship between three-dimensional spatial points and two-dimensional spatial points.

[0102] The data processing unit uses a matching point conversion algorithm in real time to convert the matching points from three-dimensional spatial points to two-dimensional spatial points, and calculates the second calibration parameters based on the two-dimensional spatial points. The matching point conversion algorithm may include, but is not limited to, the n-point perspective (PnP) algorithm.

[0103] In this embodiment, a communication unit connects to roadside cameras, and the roadside cameras and a visual interactive device acquire matching points of multiple sets of roadside camera images and 3D point clouds in real time. A data processing unit calculates the matching points using a matching point conversion algorithm to determine the second calibration parameter. This allows for portable real-time point selection using the visual interactive device, and the data processing unit automatically calculates the second calibration parameter based on the selected matching points.

[0104] In some implementations, step S133 may include step S141 or step S142:

[0105] Step S141: When the number of matching points is not less than the preset lower limit and less than the preset upper limit, the second calibration parameter is determined using the matching point conversion algorithm.

[0106] Step S142: When the number of matching points is not less than the preset upper limit, based on some of the matching points in each matching point, use the matching point conversion algorithm to determine the second candidate calibration parameter, based on the remaining matching points in each matching point excluding some of the matching points, determine the parameter error, and based on the parameter error, determine the second calibration parameter.

[0107] In step S142, the matching points used for calculation using the matching point conversion algorithm can be randomly selected automatically, and the number of matching points can be preset manually. As the number of matching points increases in real time, step S142 can be executed after step S141.

[0108] In this embodiment, when the number of matching points is not less than a preset lower limit and less than a preset upper limit, a matching point conversion algorithm is used to determine the second calibration parameter. When the number of matching points is not less than the preset upper limit, the matching point conversion algorithm is applied to a portion of the matching points to determine the second candidate calibration parameter. Based on the parameter error determined by the remaining matching points, the second calibration parameter is then determined. Thus, by setting the upper and lower limits for the number of matching points, the second calibration parameter can be updated in real time as the number of matching points increases, improving the accuracy and precision of the second calibration parameter.

[0109] In some implementations, determining the second calibration parameter based on parameter error in step S142 above may include the following step S151:

[0110] Step S151: Based on the parameter error and the parameter error threshold, determine the second calibration parameter; the second calibration parameter is the second candidate calibration parameter corresponding to the parameter error being the smallest and less than the parameter error threshold.

[0111] In some implementations, if the number of matching points is not less than a preset upper limit and the parameter error of the second candidate calibration parameter is less than the parameter error threshold, the second candidate calibration parameter corresponding to the minimum parameter error is output as the final second calibration parameter.

[0112] In some implementations, step S103 may include steps S161 to S162:

[0113] Step S161: Determine the first pixel point based on the first calibration parameter and the second calibration parameter; the first pixel point is the pixel point of the image of the roadside LiDAR's laser point cloud projected onto the roadside camera.

[0114] Specifically, the data processing unit uses the first calibration parameter to convert the laser point cloud of the roadside lidar in the coordinate system corresponding to the roadside lidar into point cloud data in the coordinate system corresponding to the three-dimensional point cloud. Then, using the second calibration parameter, the laser point cloud of the roadside lidar in the coordinate system corresponding to the three-dimensional point cloud is converted into pixels on the image projected onto the roadside camera.

[0115] Step S162: Determine the joint calibration parameters of the roadside sensing device based on the first pixel.

[0116] In some implementations, step S164 may include steps S171 to S173:

[0117] Step S171: Determine the pixel error based on the second pixel and the first pixel; the second pixel is the actual pixel selected from the image of the roadside camera by the laser point cloud of the corresponding roadside lidar.

[0118] Step S172: If the pixel error is less than the pixel error threshold, the candidate joint calibration parameters are determined as the joint calibration parameters of the roadside sensing device; the candidate joint calibration parameters are determined based on the first calibration parameter and the second calibration parameter at the current time.

[0119] Step S173: If the pixel error is not less than the pixel error threshold, redetermine the first calibration parameter and / or the second calibration parameter, and determine the joint calibration parameter of the roadside sensing device based on the redetermined first calibration parameter and / or second calibration parameter.

[0120] In some implementations, the distance between the first pixel and the second pixel can be compared to verify pixel errors.

[0121] In some implementations, the candidate joint calibration parameter can be a first calibration parameter and a second calibration parameter, or it can be a joint calibration parameter calculated based on the first calibration parameter and the second calibration parameter.

[0122] In some implementations, when the pixel error is less than the pixel error threshold, the candidate joint calibration parameters are uploaded to the roadside multi-source fusion sensing system as joint calibration parameters of the roadside sensing device, and the location information of the roadside sensing device is also uploaded to the roadside multi-source fusion sensing system.

[0123] In some implementations, if the pixel error is not less than a pixel error threshold, the candidate joint calibration parameters are considered unqualified. The first calibration parameters and / or the second calibration parameters are then redefined, and the pixel error is recalculated based on these redefined parameters until the pixel error is less than the pixel error threshold, thus determining the final joint calibration parameters for the roadside sensing device. Alternatively, the first or second calibration parameters can be redefined individually, or both can be redefined.

[0124] In this embodiment, the pixel error is determined based on the second pixel and the first pixel. If the pixel error is less than the pixel error threshold, the candidate joint calibration parameters at the current moment are determined as the joint calibration parameters of the roadside sensing device. If the pixel error is not less than the pixel error threshold, the candidate joint calibration parameters are re-determined. Thus, by introducing pixel error to determine the final joint calibration parameters and adjusting the joint calibration parameters in real time based on the pixel error, the accuracy of the joint calibration parameters can be improved, increasing the precision and stability of the roadside multi-source fusion sensing system using this embodiment.

[0125] Based on this, embodiments of this application provide a joint calibration system for roadside sensing devices based on high-precision maps. Figure 2A This application provides a schematic diagram of the implementation architecture of a joint calibration system for roadside sensing devices based on high-precision maps, as shown in the embodiments of this application. Figure 2A As shown, the roadside perception device joint calibration system 200 based on a high-precision map includes a roadside lidar 210, a roadside camera 220, and a backpack device 230. The roadside lidar 210 is used to collect radar data from the roadside; the roadside camera 220 is used to collect image data from the roadside; the backpack device 230 is used to construct a 3D point cloud high-precision map based on the data collected by the roadside lidar 210 and the roadside camera 220, calculate first calibration parameters, second calibration parameters, and joint calibration parameters of the roadside perception devices, acquire and visualize radar data and image data, and perform communication between modules, devices, or units inside and outside the backpack device.

[0126] Taking manual carrying of backpack equipment as an example, Figure 2B A schematic diagram illustrating the composition of a backpack device in a roadside sensing device joint calibration system based on a high-precision map, provided as an embodiment of this application, is shown below. Figure 2BAs shown, the backpack device 230 includes a SLAM lidar 231, a flat panel display 232, an integrated processing unit 233, shoulder straps 234, and a backpack connection portion 235. The integrated processing unit 233 includes a data processing unit, a communication unit, an RTK positioning terminal, and a balancer. The shoulder straps 234 are used to fit the backpack against the body, and the backpack connection portion 235 is used to connect the SLAM lidar 231 and the integrated processing unit 233.

[0127] Based on the roadside sensing device joint calibration system 200 and backpack device 230 based on high-precision maps provided in the above embodiments, this application provides a roadside sensing device joint calibration method based on high-precision maps.

[0128] Figure 2C A schematic diagram of the implementation process of a joint calibration method for roadside sensing devices based on high-precision maps provided in this application embodiment is shown in Figure 2. Figure 2C As shown, the method includes the following steps S201 to S212:

[0129] Step S201: Obtain raw perception data from roadside lidar and roadside cameras through the communication unit.

[0130] Among them, the joint calibration system for roadside perception devices based on high-precision maps connects to all roadside lidars and roadside cameras in real time through a communication unit to obtain the raw perception data of all roadside lidars and roadside cameras.

[0131] Step S202: Use a flat panel display to determine the roadside sensing area.

[0132] Among them, the roadside perception equipment joint calibration system based on high-precision maps visualizes the acquired raw perception data through a flat panel display, and the range, start point, end point and edge of the perception area on the road site are determined by the operator on the flat panel display.

[0133] Step S203: Determine the target movement path through the data processing unit.

[0134] Among them, the roadside perception device joint calibration system based on high-precision maps uses a data processing unit to indicate a reference path that runs through the entire roadside perception area, based on the perception range of the roadside perception area and the SLAM lidar, and uses a flat panel display as the target movement path for the backpack device.

[0135] Step S204: The backpack device moves along the target path and constructs a high-precision map of the three-dimensional point cloud of the roadside sensing area through the data processing unit, flat panel display and RTK positioning terminal.

[0136] In some implementations, a person carrying a backpack can move along the target path at a speed of 1 m / s without turning back. The person randomly selects four marker points at intervals. The data processing unit uses the SLAM algorithm and the principle of four points being coplanar to construct a high-precision three-dimensional laser point cloud map of the roadside perception area. This triggers the recording module of the flat panel display, causing it to switch from a waiting state to a recording state. After the RTK positioning terminal obtains the latitude, longitude, and elevation information of the marker points, it records the latitude, longitude, and elevation information of the marker points.

[0137] Step S205: Using a flat panel display, move the backpack device directly below the roadside lidar.

[0138] During the movement of the backpack device along the target path, the roadside perception device and calibration system based on the high-precision map displays the constructed 3D point cloud high-precision map in real time through a visualization interactive device, and indicates the location of the roadside lidar on the map. Based on this indication, the backpack device moves to directly below the roadside lidar, and uses a level to adjust the position of the backpack device horizontally, keeping the backpack device in a horizontal direction during the movement.

[0139] Step S206: Use the communication unit to acquire the synchronization data of the lidar.

[0140] During the movement of the backpack device along the target path, the roadside perception device joint calibration system based on the high-precision map acquires radar data collected by the roadside lidar and SLAM lidar in real time through the communication unit.

[0141] Step S207: Determine the first candidate calibration parameter through the data processing unit.

[0142] The roadside perception device joint calibration system based on high-precision maps uses a data processing unit to determine the conversion parameters for converting data from the coordinate system corresponding to the roadside lidar to the coordinate system corresponding to the 3D point cloud as the first candidate calibration parameters. The roadside lidar and SLAM lidar respectively perform road surface fitting on the radar data they collect to obtain multiple sets of first roadside normal vectors and knapsack normal vectors. Through the data processing unit, based on each set of first roadside normal vectors and knapsack normal vectors, a first candidate calibration parameter is determined. The multiple first roadside normal vectors are converted into multiple second roadside normal vectors using the multiple corresponding first candidate calibration parameters. The conversion process between the first roadside normal vector conversion and the second roadside normal vector conversion is described in formula (1) in the above embodiment.

[0143] Step S208: The data processing unit performs error assessment and determines the first calibration parameter.

[0144] In this error assessment, the error is the error between the second roadside normal vector and the knapsack normal vector. The roadside perception device joint calibration system based on high-precision maps determines an error based on the second roadside normal vector and the knapsack normal vector obtained from each set of synchronous data through the data processing unit. Then, based on multiple errors from multiple sets of synchronous data, the first candidate calibration parameter corresponding to the error with a qualified error assessment status is determined as the first calibration parameter in real time.

[0145] In some implementations, when the inverse cosine function is used to calculate the inverse cosine of the angle between the second side normal vector and the backpack normal vector as the error, during the horizontal adjustment process, the system can evaluate the level of the backpack device based on the magnitude of the error, classifying the level into three levels: red, yellow, and green, and displaying them on a flat panel display. The system automatically saves the first candidate calibration parameter corresponding to the smallest error in the green state as the first calibration parameter; wherein, the green state is considered qualified, and the red and yellow states are considered unqualified, green indicates an error of no more than 0.5, yellow indicates an error of more than 0.5 but no more than 0.8, and red indicates an error of more than 0.8.

[0146] Step S209: Using a flat panel display, select the matching points corresponding to the high-precision map of the 3D point cloud and the image of the roadside camera to determine the second candidate calibration parameters.

[0147] During the movement of the backpack device along the target path, a flat panel display is used to manually select matching points between a high-precision 3D point cloud map and images from roadside cameras in real time. Then, the data processing unit calculates based on the matching points to determine the second candidate calibration parameters.

[0148] In some implementations, a flat panel display can be used to visualize the images from the roadside camera and a high-precision map of the 3D point cloud on the left and right sides of the display, respectively, and a person can manually select matching points on the two sides.

[0149] In some implementations, when the number of matching points is less than 4 sets, the second candidate calibration parameters are not calculated. As the number of matching points increases to 4 sets, the roadside sensing device joint calibration system based on high-precision maps calculates the second candidate calibration parameters using the PnP algorithm through the data processing unit. As the number of matching points increases to 15 sets or more, the roadside sensing device joint calibration system based on high-precision maps automatically and randomly selects 4 / 5 sets through the data processing unit to calculate the second candidate calibration parameters using the PnP algorithm, and uses the remaining 1 / 5 sets of matching points to calculate parameter errors.

[0150] Step S210: The data processing unit performs error assessment and determines the second calibration parameters.

[0151] In some implementations, when the number of matching points is greater than 15 and there are second candidate calibration parameters within the parameter error threshold range, the roadside perception device joint calibration system based on the high-precision map automatically saves the second candidate calibration parameter corresponding to the minimum parameter error as the second calibration parameter output when the calculation of the second candidate calibration parameter is completed through the data processing unit.

[0152] Step S211: Determine the first pixel point through the data processing unit.

[0153] Among them, the roadside perception equipment joint calibration system based on high-precision maps uses a data processing unit to convert the laser point cloud of the roadside lidar from the point cloud data in the coordinate system corresponding to the roadside lidar to the point cloud data in the coordinate system corresponding to the three-dimensional point cloud using the first calibration parameter. Then, it uses the second calibration parameter to convert the laser point cloud of the roadside lidar in the coordinate system corresponding to the three-dimensional point cloud into pixels on the image projected onto the roadside camera.

[0154] Step S212: The data processing unit performs error evaluation based on the first pixel and the second pixel to determine the joint calibration parameters.

[0155] Among them, the roadside perception equipment joint calibration system based on high-precision maps uses a data processing unit to compare the distance between the first and second pixels in real time to verify the pixel error.

[0156] In some implementations, when the average pixel error is less than 1 pixel error, the first calibration parameter and the second calibration parameter at the current time are uploaded to the roadside multi-source fusion sensing system as joint calibration parameters of the roadside sensing device, and the location information of the roadside sensing device is also uploaded to the roadside multi-source fusion sensing system; when the average pixel error is not less than 1 pixel error, the first calibration parameter and the second calibration parameter at the current time are considered unqualified, and steps S205 to S212 are re-executed.

[0157] In this embodiment, the roadside sensing device joint calibration system based on a high-precision map constructs a high-precision 3D point cloud map of the road scene using roadside sensing devices and a backpack device. The backpack device is horizontally adjusted using a balancer. After real-time acquisition and processing of synchronous data from LiDAR, the error is evaluated by a data processing unit to determine the first calibration parameter. Then, matching points corresponding to the high-precision 3D point cloud map and the roadside camera image are selected in real-time. The error is calculated based on these matching points by the data processing unit, and a second calibration parameter is determined. Based on the first and second calibration parameters, a first pixel is determined, and error is evaluated based on the first and second pixels to determine the joint calibration parameters in real-time. Thus, based on an integrated joint calibration backpack tool, all functions are integrated, providing convenient, fast, and real-time parameter calculation capabilities. Furthermore, by acquiring a visualization platform with image matching point extraction accuracy superior to manual calculation, the system can calculate and evaluate the rationality of calibration parameters in real-time and save the optimal parameters. This ensures the adaptability of parameters during the use of the roadside sensing devices, guaranteeing the high stability and high accuracy of the roadside multi-source fusion sensing system, and achieving stable and continuous tracking of road traffic participants.

[0158] This application provides a joint calibration system for roadside sensing devices based on high-precision maps. Figure 3 This application provides a schematic diagram of the composition structure of a joint calibration system for roadside sensing devices based on high-precision maps, as shown in the embodiments of this application. Figure 3 As shown, the roadside sensing device joint calibration system 300 based on high-precision maps includes:

[0159] The system includes a roadside sensing device 310, a mobile SLAM lidar 320, a visual interaction device 330, and a data processing unit 340; among which,

[0160] The roadside sensing device 310 includes a roadside lidar 350 and a roadside camera 360;

[0161] The roadside lidar 350 is used to collect radar data of natural scenes;

[0162] The roadside camera 360 is used to collect image data of the natural scene;

[0163] The movable SLAM lidar 320 is used to collect radar data along the moving path of the SLAM lidar;

[0164] The visualization and interactive device 330 is used to visualize and display a high-precision map of three-dimensional point clouds, radar data collected by the roadside lidar, radar data collected by the mobile SLAM lidar, and image data collected by the roadside camera; and to provide visual instructions and interactive operations.

[0165] The data processing unit 340 is used to determine a first calibration parameter based on the high-precision map of the 3D point cloud, the radar data collected by the roadside lidar, and the radar data collected by the mobile SLAM lidar. The high-precision map of the 3D point cloud is constructed based on the 3D point cloud of the natural scene in the roadside sensing area, and the 3D point cloud is collected by the mobile SLAM lidar. The first calibration parameter is a transformation parameter between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the 3D point cloud. Based on the high-precision map of the 3D point cloud, the image data collected by the roadside camera, and the radar data collected by the mobile SLAM lidar, a second calibration parameter is determined. The second calibration parameter is a transformation parameter between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera. Based on the first calibration parameter and the second calibration parameter, a joint calibration parameter for the roadside sensing device is determined. The joint calibration parameter is a transformation parameter between the coordinate system corresponding to the roadside lidar and the pixel coordinate system corresponding to the image of the roadside camera.

[0166] This application is described with reference to flowchart illustrations and / or block diagrams of methods and systems according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0167] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0169] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0170] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0171] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0172] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0174] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0175] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0176] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0177] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0178] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A high-precision map-based joint calibration method for a roadside perception device, characterized in that, The roadside sensing device includes a roadside lidar and a roadside camera, and the method includes: A high-precision map based on 3D point clouds is constructed using the roadside lidar and a mobile real-time positioning and mapping system to determine the first calibration parameters. The high-precision map of 3D point clouds is constructed based on the 3D point clouds of the natural scene in the roadside sensing area. The 3D point clouds are collected by the mobile SLAM lidar. The first calibration parameters are the transformation parameters between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the 3D point clouds. Based on the high-precision map of the 3D point cloud, the second calibration parameter is determined using the roadside camera and the visualization interaction device; the second calibration parameter is the transformation parameter between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera. Based on the first calibration parameters and the second calibration parameters, the joint calibration parameters of the roadside sensing device are determined; the joint calibration parameters are the transformation parameters between the coordinate system corresponding to the roadside lidar and the pixel coordinate system corresponding to the image of the roadside camera.

2. The method according to claim 1, characterized in that, The portable SLAM lidar and the visual interactive device are integrated into the backpack device, which also includes a data processing unit, a communication unit, and a positioning terminal. Before determining the first calibration parameters using the roadside lidar and the mobile real-time positioning and mapping (SLAM) lidar constructed from the high-precision map based on 3D point clouds, the method further includes: Using the communication unit, connect to the roadside sensing device to obtain the raw sensing data from the roadside sensing device; Using the aforementioned visual interactive device, the roadside sensing area is determined based on the original sensing data; Using the data processing unit, the target's movement path is determined based on the roadside sensing area and the sensing capability range of the SLAM lidar; Using the positioning terminal, the positioning data of the SLAM lidar moving along the target's movement path is acquired; Using the data processing unit, a high-precision map of the three-dimensional point cloud is constructed based on the positioning data and the radar data collected by the SLAM lidar along the target's movement path.

3. The method according to claim 2, characterized in that, The backpack device also includes a balancing device; The high-precision map based on 3D point clouds utilizes the roadside lidar and a mobile real-time positioning and mapping system to construct a SLAM lidar system, determining the first calibration parameters, including: Based on the high-precision map of the three-dimensional point cloud, the backpack device is instructed to move to directly below the roadside lidar using the visualization and interactive device; The balancing device is used to keep the backpack in a horizontal position during movement; Using the communication unit, multiple sets of synchronized data collected during the movement of the backpack device are acquired in real time; wherein, each set of synchronized data includes radar data collected by the roadside lidar and the SLAM lidar respectively; Using the data processing unit, the first calibration parameter is determined based on the multiple sets of synchronized data.

4. The method according to claim 3, characterized in that, The step of determining the first calibration parameter based on the multiple sets of synchronized data includes: The conversion parameters between each set of synchronized data were determined as the first candidate calibration parameters; For each set of synchronized data, road surface fitting is performed on the set of synchronized data to obtain the first roadside normal vector and the knapsack normal vector of the road surface; the first roadside normal vector is the normal vector in the coordinate system corresponding to the roadside lidar, and the knapsack normal vector is the normal vector in the coordinate system corresponding to the three-dimensional point cloud. Based on the first candidate calibration parameters, the first roadside normal vector is converted into a second roadside normal vector in the coordinate system corresponding to the three-dimensional point cloud; Based on the error between the second roadside normal vector and the knapsack normal vector, the first calibration parameter is determined from multiple sets of first candidate calibration parameters.

5. The method according to claim 2, characterized in that, The high-precision map based on the 3D point cloud, using the roadside camera and the visualization interaction device, determines the second calibration parameters, including: The roadside camera is connected using the communication unit. Using the roadside camera and the visualization interaction device, multiple sets of images captured by the roadside camera and matching points of the 3D point cloud are acquired in real time; Using the data processing unit, based on the matching points, a matching point conversion algorithm is used to determine the second calibration parameters; the matching point conversion algorithm is used to calculate the conversion relationship between three-dimensional spatial points and two-dimensional spatial points.

6. The method according to claim 5, characterized in that, The step of using the data processing unit to determine the second calibration parameter based on the matching point and using a matching point conversion algorithm includes: When the number of matching points is not less than a preset lower limit and less than a preset upper limit, the matching point conversion algorithm is used to determine the second calibration parameter; If the number of matching points is not less than a preset upper limit, a second candidate calibration parameter is determined based on a portion of the matching points in each matching point using the matching point conversion algorithm. The parameter error is determined based on the remaining matching points in each matching point excluding the portion of the matching points, and the second calibration parameter is determined based on the parameter error.

7. The method according to claim 6, characterized in that, Determining the second calibration parameter based on the parameter error includes: Based on the parameter error and the parameter error threshold, the second calibration parameter is determined; the second calibration parameter is the second candidate calibration parameter corresponding to the parameter error being the smallest and less than the parameter error threshold.

8. The method according to any one of claims 1-7, characterized in that, The determination of the joint calibration parameters of the roadside sensing device based on the first calibration parameters and the second calibration parameters includes: Based on the first calibration parameters and the second calibration parameters, a first pixel is determined; the first pixel is a pixel in the image of the roadside LiDAR's laser point cloud projected onto the roadside camera. Based on the first pixel, the joint calibration parameters of the roadside sensing device are determined.

9. The method according to claim 8, characterized in that, The determination of the joint calibration parameters of the roadside sensing device based on the first pixel includes: The pixel error is determined based on the second pixel and the first pixel; the second pixel is the actual pixel selected from the image of the roadside camera corresponding to the laser point cloud of the roadside lidar. If the pixel error is less than the pixel error threshold, the candidate joint calibration parameters are determined as the joint calibration parameters of the roadside sensing device; the candidate joint calibration parameters are determined based on the first calibration parameters and the second calibration parameters at the current time. If the pixel error is not less than the pixel error threshold, the first calibration parameter and / or the second calibration parameter are redefined, and based on the redefined first calibration parameter and / or the second calibration parameter, the joint calibration parameter of the roadside sensing device is determined.

10. A joint calibration system for roadside sensing devices based on high-precision maps, characterized in that, include: The system includes roadside sensing devices, a mobile SLAM lidar, a visual interactive device, and a data processing unit; wherein the roadside sensing devices include a roadside lidar and a roadside camera. The roadside lidar is used to collect radar data of natural scenes; The roadside camera is used to collect image data of the natural scene; The mobile SLAM lidar is used to collect radar data along the SLAM lidar's moving path; The visualization and interactive device is used to visualize and display a high-precision map of 3D point clouds, radar data collected by the roadside lidar, radar data collected by the mobile SLAM lidar, and image data collected by the roadside camera; and to provide visual instructions and interactive operations. The data processing unit is used to determine a first calibration parameter based on the high-precision map of the 3D point cloud, the radar data collected by the roadside lidar, and the radar data collected by the mobile SLAM lidar. The high-precision map of the 3D point cloud is constructed based on the 3D point cloud of the natural scene in the roadside sensing area, and the 3D point cloud is collected by the mobile SLAM lidar. The first calibration parameter is a transformation parameter between the coordinate system corresponding to the roadside lidar and the coordinate system corresponding to the 3D point cloud. Based on the high-precision map of the 3D point cloud, the image data collected by the roadside camera, and the radar data collected by the mobile SLAM lidar, a second calibration parameter is determined. The second calibration parameter is a transformation parameter between the coordinate system corresponding to the 3D point cloud and the pixel coordinate system corresponding to the image of the roadside camera. Based on the first calibration parameter and the second calibration parameter, a joint calibration parameter for the roadside sensing device is determined. The joint calibration parameter is a transformation parameter between the coordinate system corresponding to the roadside lidar and the pixel coordinate system corresponding to the image of the roadside camera.

Images