A co-calibration method and system for multiple laser radars without common view

Abstract

The present application relates to a kind of non-common view multi-laser radar joint calibration method and system, the method includes: trajectory motion execution, the pose data of robot movement is acquired, and equivalent common view constraint is generated according to pose data;Multi-source data acquisition, the pose data and point cloud data of robot are acquired;Data preprocessing, by space-time synchronization and feature extraction, construct static environment benchmark model;Point cloud joint registration, utilize Gaussian mixture model to fuse multi-radar data, static environment benchmark model, solve the external parameter transformation matrix between radar, and output calibration result.The present application carries out data acquisition and preprocessing to multi-laser radar point cloud and odometry, then based on the probability characteristics of the above observed point cloud data corresponding Gaussian mixture model is established, the point cloud of the above multiple laser radars is registered using the model, finally realized the multi-laser radar of non-common view area without special calibration target, dynamic, high-precision joint calibration.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to a joint calibration method and system for multiple lidar systems without common view based on a Gaussian mixture model. Background Technology

[0002] As the manufacturing industry undergoes increasingly profound digital transformation and intelligent upgrading, the demand for and application of robotics technology are becoming more and more widespread. Among them, mobile robots equipped with multiple LiDAR sensors can perform functions such as autonomous navigation and obstacle avoidance, automatic material identification and loading / unloading, enabling them to play a core role in unmanned factories.

[0003] The robustness and accuracy of the aforementioned functions both depend on the joint calibration of multiple LiDARs before the robot leaves the factory. Because the robot needs to perceive surrounding information from all directions, multiple LiDARs are usually fixedly installed in different positions on the robot, resulting in no common field of view or only a very small common field of view between different LiDARs.

[0004] Currently, the automated calibration methods for multiple lidar systems without shared view are mostly non-probabilistic registration methods such as Iterative Closest Point (ICP) and its variants. Because these methods use a pairwise pairing of point clouds, they are quite sensitive to initial extrinsic parameters and iterative registration parameters, resulting in systematic deviations in the calibration results. This often leads to large point cloud registration deviations or even registration failures during the calibration process, ultimately requiring manual registration to achieve joint calibration.

[0005] For example, Patent 1 (authorization publication number CN113640778B) uses vehicle-mounted GNSS to achieve joint calibration of multiple lidars without common view. However, the positioning accuracy of GNSS is generally between 1.5 meters and 3 meters, and the positioning accuracy of RTK is between 0.1 meters and 0.2 meters, while the perception accuracy required by mobile robots is generally between 0.01 meters and 0.05 meters. In addition, GNSS has poor applicability in indoor environments. Therefore, the accuracy of this calibration method cannot meet the accuracy requirements of mobile robots operating indoors.

[0006] For example, Patent 2 (application publication number CN116007659) utilizes a calibration room containing multiple sensors and calibration targets to achieve joint calibration of multiple lidars without shared field of view. This method requires the prior setup of the calibration room and manual remote control of a robot to a designated location, which makes the calibration cost high and the process complex. Summary of the Invention

[0007] In view of the shortcomings of the prior art, the purpose of this invention is to provide a joint calibration method and system for multiple lidars without common view, so as to solve one or more problems in the prior art.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows: A joint calibration method for multiple lidar systems without common view includes the following steps: The trajectory motion is executed, the robot's pose data during movement is acquired, and an equivalent common-view constraint is generated based on the pose data. The validity of the equivalent common-view constraint is then verified. Multi-source data acquisition to obtain robot pose data and point cloud data; Data preprocessing involves spatiotemporal synchronization and feature extraction to construct a static environmental benchmark model; Point cloud joint registration utilizes a Gaussian mixture model to fuse multi-radar data and a static environment benchmark model, solves the extrinsic transformation matrix between radars, and filters dynamic points in the static environment benchmark model to output calibration results.

[0009] Furthermore, the calibration method also includes the following steps: During system initialization, timing control is used to ensure the reliability of the collected data.

[0010] Furthermore, the calibration method also includes the following steps: Visual verification determines whether the calibration results meet the requirements. If yes, the verification results are saved; otherwise, a warning is triggered. The parameters are saved, the calibration results are written to the corresponding parameter configuration file, and metadata is created based on the configuration file and archived to the server.

[0011] Furthermore, the system initialization includes the following steps: Start the robot; Hardware initialization control continues until the sensor is stable and ready. The robot moves to the starting point of the preset trajectory; The starting point positioning verification determines whether the overlap between the real-time acquired point cloud and the preset reference point cloud is greater than a set threshold. If yes, the calibration program is started to enter the trajectory motion execution step; otherwise, the positioning verification is performed again.

[0012] Furthermore, the hardware initialization control includes the following steps: The robot calculates the sensor warm-up time based on an adaptive waiting time model; Check whether the sensor has reached the aforementioned sensor preheating time; Furthermore, the trajectory motion execution includes the following steps: Real-time acquisition of pose data; the robot acquires pose data using active or passive modes based on a preset trajectory; A spatiotemporal mapping function is constructed based on the pose data to unify the scanning points of different radars at different times into the same global coordinate system; Generate equivalent common-view constraints; Determine whether the equivalent co-view constraint is valid. If yes, proceed to multi-source data acquisition; otherwise, adjust the trajectory and return to real-time pose data acquisition.

[0013] Furthermore, the multi-source data acquisition includes the following steps: Intelligently sets the location and number of parking points on a preset trajectory; Determine if the robot has moved to the parking spot; if yes, collect static data; otherwise, record dynamic data. Determine if the static data is valid. If yes, move to the next parking spot. If no, return to re-determine if the robot has moved to the designated parking spot.

[0014] Furthermore, the data preprocessing includes the following steps: Optimize data spatiotemporal synchronization and construct a reference coordinate system to achieve precise alignment of pose data and point cloud data at the parking point; An environmental baseline is established to transform the point cloud data of parking points at any given time into a global coordinate system; Identify whether the point cloud data in the global coordinate system has dynamic point clouds. If so, remove the dynamic point clouds; otherwise, output a static reference.

[0015] Furthermore, the point cloud joint registration includes the following steps: Construct a Gaussian mixture model and establish the correlation between the point cloud and the environmental benchmark model based on the model to obtain the correlation probability; The association probability is calculated using the EM algorithm, which calculates the association probability through the E-step. The extrinsic parameter transformation matrix is ​​optimized by iterating through M-step to find the optimal extrinsic parameters. After each iteration, a convergence condition is checked to determine whether another iteration is needed. If so, the extrinsic parameters obtained in this iteration are substituted into the EM algorithm to solve again; otherwise, the optimal extrinsic parameters are considered to have been found, and the calibration results are output.

[0016] Furthermore, the visual verification includes the following steps: A spatial error heatmap is generated based on the calibration results; Calculate the confidence interval of the parameters based on the spatial error heatmap, and then quantify the uncertainty of the external parameters. Furthermore, the metadata includes calibration time, calibration result, device number, raw data, and operator information.

[0017] Accordingly, the present invention also provides a robot, including processor; Memory used to store processor-executable instructions; The processor is configured to execute the aforementioned joint calibration method for multiple lidar systems without common view.

[0018] Compared with the prior art, the beneficial technical effects of the present invention are as follows: (i) This invention improves the accuracy of calibration data by collecting and preprocessing data from multiple lidar point clouds and odometers. Then, based on the probability characteristics of the observed point cloud data, a corresponding Gaussian mixture model is established. This model is then used to jointly register the point clouds of the multiple lidars. Finally, dynamic and high-precision joint calibration of multiple lidars without common view zones is achieved without the need for special calibration targets.

[0019] (II) Furthermore, compared with traditional ICP and NDT methods, the maximum initial pose error threshold for point cloud registration using the GMM-EM algorithm in this invention can be greater than 1 meter, while traditional methods basically fail when the initial pose error exceeds 30 centimeters. This invention uses the GMM algorithm to perform structured modeling of the observed point cloud, overcoming the problem of uneven density in the environmental point cloud, resulting in stronger robustness and faster convergence. Moreover, this invention uses the EM algorithm to solve the constructed Gaussian mixture model containing latent variables. Furthermore, this invention employs covariance adaptive scaling to accelerate the coarse-to-fine convergence process in solving the above model.

[0020] (III) Furthermore, this invention combines the structural attributes of environmental features with probabilistic optimization to construct a multi-resolution point cloud registration framework that progresses from coarse to fine. At the theoretical level, it introduces covariance adaptive scaling to achieve rapid global convergence of the Gaussian mixture model based on the EM algorithm, and proposes anisotropic covariance coding structural constraints. At the engineering level, it solves the problem that traditional point cloud registration methods cannot converge quickly or even fail when the initial pose error is large. Its millimeter-level accuracy can meet the positioning and perception accuracy requirements of robots operating in actual industrial scenarios. In terms of application, it enables the implementation of a high-precision point cloud registration method based on calibration-free targets in scenarios such as warehousing and logistics robots and unmanned vehicles. Compared with manual calibration or traditional automatic methods, the calibration method of this invention is more efficient, more accurate, and more robust. Attached Figure Description

[0021] Figure 1 The diagram shows a flowchart of a joint calibration method for multiple lidar systems without common view, according to an embodiment of the present invention.

[0022] Figure 2 The diagram illustrates a joint calibration method for multiple lidar systems without common view, and a flowchart of the system initialization steps in accordance with an embodiment of the present invention.

[0023] Figure 3The diagram shows a flowchart of the trajectory motion self-propagation step in a joint calibration method for multiple lidar systems without common view according to an embodiment of the present invention.

[0024] Figure 4 The diagram illustrates the multi-source data acquisition steps in a joint calibration method for multiple lidar systems without common view, according to an embodiment of the present invention.

[0025] Figure 5 The diagram shows a data preprocessing step in a joint calibration method for multiple lidar systems without common view, according to an embodiment of the present invention.

[0026] Figure 6 The diagram illustrates the point cloud joint registration step in a joint calibration method for multiple lidar systems without common view according to an embodiment of the present invention.

[0027] Figure 7 The diagram shows a flowchart of the visualization verification step in a joint calibration method for multiple lidar systems without common view according to an embodiment of the present invention.

[0028] Figure 8 The diagram shows a parameter saving step in a joint calibration method for multiple lidar systems without common view according to an embodiment of the present invention.

[0029] Figure 9 The diagram shows a top view and a side view of a stealthy logistics robot with dual radars in a non-common-viewing area, using a joint calibration method and system based on an embodiment of the present invention.

[0030] Figure 10 This invention illustrates a joint calibration method for multiple lidar systems without common view, along with a schematic diagram of the preset calibration trajectory and corresponding parking point in the system. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed description of a joint calibration method and system for multiple lidar systems without common view, in conjunction with the accompanying drawings and specific embodiments, will further illustrate the present invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the purpose of the embodiments of this invention. Please refer to the accompanying drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only used to complement the content disclosed in the claims, for those skilled in the art to understand and read, and are not intended to limit the implementation conditions of this invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportional relationships, or adjustments to the size, without affecting the effects and objectives achieved by this invention, should still fall within the scope of the technical content disclosed in this invention.

[0032] A joint calibration method for multiple lidar sensors without shared view is disclosed. This method is applied to a robot equipped with multiple lidar sensors. The invention employs a probabilistic registration method, establishing a Gaussian mixture model based on the probabilistic characteristics of observed point cloud data. This Gaussian mixture model is then used to jointly register the point clouds of multiple lidar sensors, achieving dynamic joint calibration of multiple lidar sensors without shared view without the need for a special calibration target. The method includes the following steps: Step A: System initialization. Timing control is used to ensure the reliability of the collected data and avoid the initial errors introduced by the sensors during the robot's cold start.

[0033] For details, please refer to Figure 1 and Figure 2 The system initialization process includes the following detailed steps: A1: Start the robot (i.e., power on the robot). Press the robot's start button to put the robot into operation.

[0034] A2: Hardware initialization control until the sensor is stable and ready. The detailed steps of the hardware initialization control are as follows: A200: The robot calculates the sensor warm-up time based on an adaptive waiting time model, wherein the adaptive waiting time model is:

[0035] in This indicates the sensor warm-up time during the robot's cold start. Scaling factor The zero bias standard deviation of the gyroscope. The minimum standard deviation threshold, This represents the minimum waiting time. Higher gyroscope noise leads to longer sensor warm-up times. The adaptive waiting time model described above can prevent initial data drift caused by sensor insecurity, ensuring the accuracy of the trajectory starting point during the trajectory motion execution steps. Specifically, the scaling factor mentioned above... Set to 2, minimum waiting time Set to 5 seconds, minimum standard deviation threshold Set to 0.05. Since the zero standard deviation varies among different gyroscopes, the specific value should be obtained from their product specifications.

[0036] A201: Sensor self-test complete. This means whether the sensor self-test has reached the sensor preheating time. If so, the sensor self-test is complete.

[0037] A202: The robot moves to the starting point of the preset trajectory.

[0038] A203: Starting point positioning verification. The robot's main LiDAR acquires a preset reference point cloud at the starting point location. It then determines whether the overlap between the real-time acquired point cloud and the preset reference point cloud at the starting point location exceeds a threshold. If yes, the starting point positioning verification is successful, and the calibration program is initiated to proceed to the trajectory motion execution step. If not, repositioning verification is performed, i.e., the real-time point cloud is reacquired. The purpose of reacquiring the real-time point cloud is to verify whether the robot has reached the designated starting point location; that is, verification is performed while acquiring data. Point cloud data from this step is not retained after successful starting point verification. Specifically, the determination is made using the following formula:

[0039] in, The point cloud acquired in real time by the main lidar. Preset a reference point cloud for the starting position. The threshold for successful localization is set when the overlap between the point cloud acquired in real-time by the main LiDAR and the preset reference point cloud at the starting point location exceeds the threshold. Upon successful localization, this step establishes a precise initial coordinate system for subsequent trajectory movement, reducing calibration errors. The aforementioned main LiDAR is the robot's front-facing LiDAR; this threshold... Set to 0.85. The point cloud mentioned above refers to all environmental features within the scanning range of a lidar, and records the coordinate position of each environmental feature in the lidar coordinate system.

[0040] Please refer to Figure 3 The above calibration method also includes step B: trajectory motion execution steps: After system initialization is complete, the trajectory motion execution step begins. This step controls the robot to move along a preset trajectory (either in active or passive mode, acquiring the robot's pose sequence along the entire preset trajectory) to obtain pose data during robot movement (the pose is obtained through the robot's odometry). Based on this pose data, an equivalent common-view constraint is generated, and the validity of this constraint is verified. Generating the equivalent common-view constraint establishes a common-view region among multiple lidars in the time dimension; that is, the same area scanned by each lidar at different times constitutes the common-view region. Step B aims to provide a spatiotemporal correlation basis for step E, satisfying the necessary constraints for subsequent point cloud registration.

[0041] For further details, please refer to... Figure 10 In this embodiment, the trajectory is preferably in the shape of an "8" (other trajectories besides this shape can also be used in other embodiments), and the total length of the trajectory is... Set to 40 meters, minimum parking distance Set to 8 meters, environmental complexity coefficient Setting it to 1, calculations show that 6 parking points need to be set. Based on the calibration scenario, these six points are specifically set as 2 intersections of a figure-eight and 4 vertices.

[0042] Please refer to Figure 9 and Figure 10 The robot includes a front-mounted LiDAR and a rear-mounted LiDAR. The front-mounted LiDAR is installed at the front of the robot, and its coordinate system is... The rear-mounted LiDAR is installed at the rear of the robot, coordinate system .

[0043] For details, please refer to Figure 3 The trajectory motion execution includes the following steps: B1: Real-time pose data acquisition; the robot acquires pose data in active or passive mode according to a preset trajectory; specifically, when performing a movement task, the robot acquires pose data at fixed time intervals. Sampling pose information:

[0044] in For the first The timestamp (in seconds) of the next sample. For rotation matrix, This is the translation vector (unit: meters). It is a zero vector. The total number of samples, Indicates in Time (the first) The robot's pose information (timestamp at the time of the next sampling).

[0045] Furthermore, the aforementioned active mode involves placing the robot on an immovable plane (such as the ground) and having the robot autonomously (or remotely controlled) move along a preset trajectory. The robot's pose sequence is calculated using the robot's odometry.

[0046] The passive mode described above involves placing the robot on a movable, rotatable guide rail platform, which then drives the robot to follow a preset path. Trajectory motion, robot pose sequence Data is acquired directly through sensors in mechanical devices.

[0047] Step B2: Construct a spatiotemporal mapping function based on the pose data to unify the scanning points of different radars at different times into the same global coordinate system.

[0048] Specifically, based on the above pose matrix Construct a spacetime mapping function, which is expressed as follows:

[0049] in, Let the coordinates of the point cloud be in the lidar coordinate system. This represents the mapping of the point cloud in the global coordinate system. This represents the continuous pose in the formula for the continuous spatiotemporal mapping function, while the above... In fact, the discrete pose is obtained by sampling at time intervals.

[0050] Step B3: Generate equivalent common-view constraints in the global coordinate system based on the point cloud described above. These constraints can be understood as determining whether the same location (or similar locations within a certain range) in the environment is scanned by the robot's front and rear LiDAR sensors. The point cloud satisfying these equivalent constraints is taken as the equivalent common-view point set. When determining this set, not only distance constraints but also time constraints are required to improve the accuracy of the common-view point set. For example, the predetermined path in Step B may appear twice or more at the same location, and the odometry pose will be different at different times. Therefore, to improve calibration accuracy, a time threshold needs to be set for judgment.

[0051] Specifically, an equivalent common-view constraint is considered to be established when the following conditions are met:

[0052] in, This is an equivalent set of co-viewing points (effective matching point pairs). The Euclidean distance threshold for point cloud matching. The time window threshold, Represents timestamp and timestamp The time interval between these intervals must be within a threshold to establish an equivalent formula constraint. Let these be the coordinates of the point cloud in the coordinate system of the front-end lidar. The coordinates of the point cloud in the coordinate system of the rear lidar are: This represents the mapping of the point cloud acquired by the front-end lidar in the global coordinate system. This represents the mapping of the point cloud acquired by the rear-mounted LiDAR in the global coordinate system. In this embodiment, the Euclidean distance threshold for point cloud matching is used. Set to 0.05m, time window threshold Set to 2s.

[0053] Step B4: Determine if the equivalent common-view constraint is valid. If yes, proceed to multi-source data acquisition; otherwise, adjust the trajectory and return to real-time pose data acquisition. The trajectory adjustment step includes steps B2 to B4, which includes the criteria for determining whether pose adjustment is needed and the requirements for trajectory adjustment. Specifically, the trajectory is adjusted so that the point cloud overlap rate between the two radars in different time and space conditions meets the minimum threshold.

[0054] Specifically, the criteria for determining whether the equivalent co-visibility constraint is valid are as follows:

[0055] in, To achieve the minimum overlap rate, , Let A and B be the point sets of their effective fields of view, respectively. To determine the number of point clouds within the effective field of view, the robot continues moving when the constraint is valid; when the constraint is invalid, the trajectory is adjusted and the robot returns to step B3 to regenerate the equivalent co-view constraint, and then step B4 is executed again for verification. Specifically, the aforementioned minimum overlap rate... In this embodiment, it is set to 0.15.

[0056] Please refer to Figure 4 The calibration method described above also includes step C: a multi-source data acquisition step, which is used to acquire the robot's pose data and point cloud data. This step is performed simultaneously with step B, that is, the data acquisition step is executed synchronously when the robot moves according to the preset trajectory.

[0057] The above multi-source data acquisition steps include the following specific steps: Step C1: Intelligently set the location and number of parking points on the preset trajectory to achieve parking at the preset trajectory parking point locations under each preset number.

[0058] Specifically, the number of parking spots is calculated using the following formula:

[0059] in, The total length of the trajectory (unit: meters). Minimum parking distance (unit: meters). Environmental complexity coefficient (levels 1-5), symbol Indicates rounding up. This indicates the number of parking spots.

[0060] Step C2: Determine if the robot has moved to the parking point. If yes, collect static data; if no, record dynamic data. Specifically, the criterion for determining whether the robot has moved to the parking spot is when the robot's speed... meters per second and angular velocity The duration of continuous data acquisition when collecting static point cloud data (data collected when the robot is stationary at the parking point) in radians per second. Adaptive adjustment based on the following formula:

[0061] in, This is the standard deviation of vibration acceleration, used to dynamically adjust the data acquisition duration. This represents the continuous acquisition time, and in this embodiment, the minimum stopping speed. Set to 0.005 meters per second, minimum stopping angular velocity. Set to 0.005 radians per second.

[0062] When the robot is stationary at the aforementioned parking point, it is necessary to simultaneously collect point clouds from multiple LiDAR sensors and odometry pose data; while during robot movement, only odometry data needs to be collected.

[0063] When the parking point is determined to have been reached, static data is collected, namely, point cloud data and pose data are recorded as follows: 1) Pose data:

[0064] 2) Point cloud data:

[0065] in, Indicates the first One parking spot, Represents the rotation matrix. It is a translation vector. Represents the zero vector. In the first The total number of point clouds collected from each parking spot. In the first A collection of point clouds collected from each parking spot.

[0066] In non-parking areas, the lidar point cloud acquisition is paused to avoid motion distortion. Simultaneously, a fixed frequency is used. A continuous sequence of poses is recorded, where the pose sequence is a set of all pose data, and its formula is as follows:

[0067] in It refers to The pose at each moment was collected in total. indivual, It is all the acquired pose sequences (i.e., pose set).

[0068] Step C4: Determine whether the collected static data is valid. If yes, move to the next parking point. If not, return to re-determine whether the robot has moved to the designated parking point.

[0069] Specifically, the criteria for judging the validity of static data are as follows:

[0070] in, Indicates the first The point cloud in the first Coordinates of time, Indicates the first The point cloud in the first Coordinates of time, This represents a point cloud data quality metric used to measure the effectiveness of static data.

[0071] If If two consecutive points are detected, then the current point will be re-collected; If so, adjust the trajectory parameters.

[0072] Please refer to Figure 5 The above calibration method also includes step D: Step D constructs a static environment benchmark model through spatiotemporal synchronization and feature extraction. Step D specifically includes the following steps: Step D1: Optimize data spatiotemporal synchronization and construct a reference coordinate system to achieve precise alignment of pose data and point cloud data at parking point time. Constructing a reference coordinate system means setting a global coordinate system for all data in advance and transforming all collected point cloud data and pose data from their respective coordinate systems to this coordinate system, which can also be called the world coordinate system or global coordinate system.

[0073] The reference coordinate system is constructed as follows:

[0074] in, and They are time points and time The captured pose. It is a moment The pose is based on the time. and time The calculated pose and For the closest The pose timestamp, and the order of the three is as follows: , It is a moment With time The time difference between them It is a moment With time The time difference between them Time weighting coefficient ( The construction of this reference coordinate system ensures pose error Angle, time error millisecond.

[0075] This reference coordinate system is a unified spatial reference system established through spatiotemporal synchronization, providing a common measurement benchmark for all LiDAR data. Its origin is the starting point of the robot trajectory. This reference coordinate system serves as the reference benchmark for raw data acquisition in step C, the target coordinate system for point cloud transformation in step D, and the unified reference system for extrinsic parameter optimization in step E.

[0076] Step D2: Construct an environmental baseline to transform the point cloud data at the parking moment to a global coordinate system. This environmental baseline is a collection of point clouds constructed in the baseline coordinate system, including the original static point cloud and dynamic point cloud. The static point cloud refers to the point cloud obtained by scanning stationary objects in the environment when the robot is stationary at the parking point. Since this point cloud has minimal distortion, its accuracy is high, and subsequent registration steps will be based on this static point cloud set. The dynamic point cloud refers to the point cloud data obtained when the robot is stationary at the parking point, but the objects have undergone pose movement and shape changes. Due to point cloud distortion, this will affect the accuracy of subsequent calibration; therefore, this dynamic point cloud set will be removed in subsequent preprocessing steps.

[0077] Specifically, the environmental baseline represents the raw point cloud data collected at different parking points at different times, which, after coordinate axis transformation, are unified to the aforementioned baseline coordinate system. The global coordinate system transformation is as follows:

[0078] in, This is the original point cloud of the parking spot. This is the point cloud in the global coordinate system. Using this formula, the overall point cloud output is obtained as follows: The aforementioned environmental baseline is generated by fusing all parking point clouds in the baseline coordinate system. It covers the entire environment through multi-angle scanning and only includes fixed structures such as walls, columns, and ground planes.

[0079] Step D3: Identify whether there is a dynamic point cloud in the global coordinate system. If yes, remove the dynamic point cloud; if no, output a static reference.

[0080] Among them, point Represented as the pose of a cloud at a point in the environmental baseline. This represents the entire point cloud set mentioned above. Midpoint The point that is closest in location is used to distinguish points. Is it a spatial reference anchor point belonging to a static environment? Indicates the current point It is the probability of a static point cloud. Indicates the current point It is the minimum probability threshold of a static point cloud. To match the tolerance, the filtering rules are as follows: This is then considered a dynamic point and removed. Here, the matching tolerance... Set to 0.05 meters, minimum dynamic point recognition threshold. Set to 0.3, when These are then considered dynamic points and discarded.

[0081] Furthermore, the environmental baseline after removing the dynamic point cloud is the purified version, also known as the baseline model. It is a highly reliable static structure after removing the dynamic point cloud. This baseline model is composed of all static structure point clouds and is used for subsequent registration.

[0082] After the above data preprocessing is completed, step E, point cloud joint registration, is initiated. It uses a Gaussian mixture model to fuse multi-radar data and a static environment benchmark model, solves the extrinsic parameter transformation matrix between radars, and filters dynamic points in the static environment benchmark model to output calibration results.

[0083] Furthermore, especially in multi-LiDAR calibration scenarios without targets, the following problems are faced: (i) Unstructured environment: It is impossible to pre-set feature points; (ii) Uneven point cloud density: Near-field points are dense, and far-field points are sparse; (iii) Large initial external parameter error: Traditional ICP algorithm fails when the initial error is >30cm.

[0084] For details, please refer to Figure 6 Step E, point cloud joint registration, includes the following steps: Step E1: Establish the association between the point cloud and the environmental baseline model based on the Gaussian mixture model, and obtain the association probability. Specifically, the following function uses the EM (Expectation Maximization) algorithm to solve the E-step of the GMM model to calculate the association probability.

[0085] in, For point clouds Belongs to characteristics The posterior probability (contribution). For feature index, This represents the number of all Gaussian components (features) in the mixture model. . is the average position of the baseline. As the baseline covariance matrix, Features The multivariate Gaussian distribution density function, Features The weights satisfy .

[0086] Step E2: Optimize the extrinsic parameter transformation matrix using the EM algorithm. The association probability is calculated via the E-step, and the optimal extrinsic parameters are obtained through M-step iteration. Each iteration requires a convergence condition check: whether another iteration is needed. If yes, the extrinsic parameters obtained in this iteration are substituted into the EM algorithm for resolving; otherwise, the optimal extrinsic parameters are considered to have been obtained, and the calibration results are output. Specifically, the formula for solving the optimal extrinsic parameter matrix is ​​as follows: in, To find the transformation matrix that maximizes the objective function , For point clouds Based on radar points transformation Transformation from local coordinate system to global coordinate system This represents the weighted contribution of different features. The goal of the above optimization is to obtain the extrinsic parameters between the two radars. After the convergence iteration is completed, the optimal extrinsic parameters are obtained, which is the final result required for radar calibration.

[0087] The above optimization problem is solved using the Levenberg-Marquardt algorithm, and the convergence condition is as follows:

[0088] in, This represents the translational change (L2 norm). To set a distance threshold, For rotational changes, To set an angle threshold. Distance threshold. Set to 0.005 meters, angle threshold. Set to 0.005 radians, the extrinsic parameters are optimized based on the M-step iterative optimization in the EM algorithm. .

[0089] Step E3: Covariance Adaptation: Accelerate convergence using the following exponential decay strategy:

[0090] in, For the number of iterations, The initial covariance matrix, For the first The feature covariance matrix of the next iteration The attenuation coefficient is... The decay step size coefficient represents the decay step size per iteration. The decay occurs once for each iteration. This strategy allows points with larger covariances in the early stages to be associated with features over a wider range, ensuring global convergence; in the later stages, smaller covariances force more precise matching, improving calibration accuracy.

[0091] Here, attenuation coefficient Set to 0.8, which is the decay step size. The coefficient is set to 5.

[0092] For details, please refer to Figure 7 After the point cloud joint registration is completed in step D, step E, visualization verification, is executed. Step E is used to determine whether the calibration results meet the requirements. If yes, the verification results are saved; otherwise, a warning is triggered. Warnings can be triggered in various forms, including but not limited to visual prompts, voice messages, and LED lights. The steps include the following: Step E1: Generate a spatial error heatmap. The formula for generating the spatial error heatmap is as follows:

[0093] in, Registration error (unit: meters) Maximum display error (unit: meters) The nearest reference point is used for calculation. The required variables, and This indicates that the calculated value will be converted to its corresponding color. This is achieved through color level mapping: when the error... Color levels That is, it changes from blue to red. Maximum display error. Set to 0.2 meters.

[0094] Step E2: Calculate the confidence interval for the parameters and quantify the uncertainty of the extrinsic parameters. The calculation of the confidence interval includes the following formula:

[0095] in, and These are the Fisher information matrices for the translation component and the rotation component, respectively. Let be the likelihood function in step E. The Fisher information matrix is ​​a commonly used mathematical concept and will not be elaborated further in this invention.

[0096] For details, please refer to Figure 8 After step E, visualization verification, is completed, step F, parameter saving, is executed. The calibration results are written to the corresponding parameter configuration file, and metadata is created based on the configuration file and archived to the server. The metadata includes calibration time, calibration results, device number, raw data, and operator information.

[0097] Accordingly, the present invention also provides a robot, the robot comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the aforementioned joint calibration method for multiple lidar systems without common view.

[0098] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the claims.

[0099] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A joint calibration method for multiple lidar systems without common view, characterized in that... The steps include the following: The trajectory motion execution controls the robot to move along a preset trajectory and acquires pose data output by the robot's odometry. A spatiotemporal mapping function is constructed based on this pose data to unify scan points collected by different radars at different times into the same global coordinate system. Based on this spatiotemporal mapping function, point clouds collected by different LiDARs at different times and transformed to the global coordinate system are matched with different radar point clouds according to Euclidean distance thresholds and time window thresholds to form an equivalent common-view point set associated with different radar data, thereby constructing an equivalent common-view constraint. The effectiveness of the equivalent common-view constraint is verified by determining whether the overlap rate of the point clouds corresponding to the equivalent common-view point set is greater than a preset minimum overlap rate threshold. If yes, multi-source data acquisition begins; otherwise, the trajectory is adjusted and the robot returns to real-time pose data acquisition. During the trajectory motion execution, when the robot moves to a preset parking point and is stationary, static point cloud data and corresponding pose data from each LiDAR are acquired. Data preprocessing involves synchronizing the collected multi-source data in time and space and constructing a static environmental benchmark model. Specifically, this includes: transforming and fusing the point cloud data collected from all parking points into the global coordinate system based on the corresponding pose data to form an environmental benchmark point cloud; and identifying and removing dynamic points from the environmental benchmark point cloud by comparing its stability with that of neighboring point clouds to obtain a static environmental benchmark model. Point cloud joint registration is performed by using a Gaussian mixture model to probabilistically correlate the point cloud data of the multiple lidars with the static environment benchmark model, and the external parameter transformation matrix between each lidar is solved by iterative optimization using the expectation-maximization algorithm, and the calibration results are output.

2. The joint calibration method for multiple lidar systems without common view as described in claim 1, characterized in that: The calibration method also includes the following steps: System initialization utilizes timing control to ensure the reliability of collected data.

3. The joint calibration method for multiple lidar systems without common view as described in claim 1, characterized in that: The calibration method also includes the following steps: Visual verification determines whether the calibration results meet the requirements. If yes, the verification results are saved; otherwise, a warning is triggered. The parameters are saved, the calibration results are written to the corresponding parameter configuration file, and metadata is created based on the configuration file and archived to the server.

4. The joint calibration method for multiple lidar systems without common view as described in claim 2, characterized in that: System initialization includes the following steps: Start the robot; Hardware initialization control continues until the sensor is stable and ready. The robot moves to the starting point of the preset trajectory; The starting point positioning verification determines whether the overlap between the real-time acquired point cloud and the preset reference point cloud is greater than a set threshold. If yes, the calibration program is started to enter the trajectory motion execution step; otherwise, the positioning verification is performed again.

5. The joint calibration method for multiple lidar systems without common view as described in claim 4, characterized in that: The hardware initialization control includes the following steps: The robot calculates the sensor warm-up time based on an adaptive waiting time model; Check whether the sensor has reached the above-mentioned sensor preheating time.

6. The joint calibration method for multiple lidar systems without common view as described in claim 1, characterized in that: The trajectory motion execution includes the following steps: Real-time acquisition of pose data; The robot acquires pose data using either active or passive modes based on a preset trajectory.

7. The joint calibration method for multiple lidar systems without common view as described in claim 1, characterized in that: The multi-source data acquisition includes the following steps: Intelligently sets the location and number of parking points on a preset trajectory; Determine if the robot has moved to the parking spot; if yes, collect static data; otherwise, record dynamic data. Determine if the static data is valid. If yes, move to the next parking spot. If no, return to re-determine if the robot has moved to the designated parking spot.

8. The joint calibration method for multiple lidar systems without common view as described in claim 1, characterized in that: The data preprocessing includes the following steps: Optimize data spatiotemporal synchronization and construct a reference coordinate system to achieve precise alignment of pose data and point cloud data at the parking point; An environmental baseline is established to transform the point cloud data of parking points at any given time into a global coordinate system; Identify whether the point cloud data in the global coordinate system has dynamic point clouds. If so, remove the dynamic point clouds; otherwise, output a static reference.

9. The joint calibration method for multiple lidar systems without common view as described in claim 1, characterized in that: The point cloud joint registration includes the following steps: The steps are as follows: Construct a Gaussian mixture model and establish the correlation between the point cloud and the environmental benchmark model based on the model to obtain the correlation probability; The association probability is calculated using the EM algorithm, which calculates the association probability through E-step. The extrinsic parameter transformation matrix is ​​optimized by iterating through M-step to obtain the optimal extrinsic parameters. After each iteration, a convergence condition is checked to determine whether another iteration is needed. If so, the extrinsic parameters obtained in this iteration are substituted into the EM algorithm for re-solving. If not, the optimal extrinsic parameters are considered to have been obtained, and the calibration result is output.

10. The joint calibration method for multiple lidar systems without common view as described in claim 3, characterized in that: The visual verification includes the following steps: Generate a spatial error heatmap based on the calibration results; The confidence interval of the parameters is calculated based on the spatial error heatmap, and the uncertainty of the external parameters is quantified.

11. The joint calibration method for multiple lidar systems without common view as described in claim 3, characterized in that: The metadata includes calibration time, calibration result, device number, raw data, and operator information.

12. A system, characterized in that: The system includes a processor; Memory used to store processor-executable instructions; The processor is configured to execute a joint calibration method for multiple lidar systems without common view, as described in any one of claims 1 to 11.

Images