A multi-sensor fusion method and system for intelligent driving vehicles

Abstract

The present invention belongs to the technical field of intelligent driving vehicles, and specifically relates to a multi-sensor fusion method and system for intelligent driving vehicles. The method comprises: constructing an extended target tracker based on a GM-PHD algorithm and a rectangular target model of a detection object; using the extended target tracker to process detection information of a millimeter-wave radar to obtain millimeter-wave radar track information of the detection object; using a constructed bounding box detector and a JPDA tracker configured with an IMM-UKF to process detection information of a lidar to obtain lidar track information of the detection object; using time-space conversion to process the millimeter-wave radar track information and the lidar track information to obtain a central fusion node; and using an IMF algorithm to process the central fusion node to obtain global track information. The present invention solves the combinatorial explosion introduced by a data association method and the timing problem caused by the misordering of local track information of different sensors.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent driving vehicles, and particularly relates to a multi-sensor fusion method and system for intelligent driving vehicles. Background Art

[0002] With the continuous evolution of automotive intelligence in recent years, intelligent driving vehicle technology has received increasing attention. Perception and cognition are key to intelligent driving vehicles. They provide the basis for decision-making during driving, and their performance is directly related to the overall vehicle control effect.

[0003] Common sensors include millimeter-wave radar and lidar. Millimeter-wave radar sensors suffer from large heading angle deviations, large lateral position deviations, and an inability to obtain target volume information. Lidar sensors experience significant degradation in sensing accuracy in rainy and snowy conditions. Furthermore, lidar cannot directly obtain target velocity information, making it less sensitive to changes in target velocity.

[0004] Because single-sensor intelligent vehicles are no longer able to cope with complex driving environments, multi-sensor configurations have become standard for intelligent vehicles to enhance their perception and cognitive capabilities. However, the increasing number of detected targets and the disparity in measurement information from different sensors present new challenges for multi-sensor fusion. On the one hand, with the advancement of sensor technology, extended targets can occupy multiple sensor resolution units, introducing a combinatorial explosion problem for data association methods. On the other hand, the data obtained from millimeter-wave radar and lidar measurements may be subject to processing time delays and communication delays during transmission, which in turn leads to timing issues caused by the misordering of local track information. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention proposes a multi-sensor fusion method and system for intelligent driving vehicles, which includes:

[0006] S1: An extended target tracker is constructed based on the GM-PHD algorithm and the rectangular target model of the detected object. The extended target tracker is used to process the 2D detection information of the millimeter-wave radar to obtain the millimeter-wave radar track information of the detected object.

[0007] S2: Construct a bounding box detector and a JPDA tracker with IMM-UKF. Use the bounding box detector and the JPDA tracker with IMM-UKF to process the 3D detection information of the lidar to obtain the lidar track information of the detected object.

[0008] S3: Use time-space conversion to process the millimeter-wave radar track information and the lidar track information to obtain a central fusion node; use the IMF algorithm to process the central fusion node to obtain global track information; and track the detected object based on the global track information.

[0009] Preferably, the process of building an extended target tracker includes:

[0010] Obtaining a rectangular extended target state of the detected object according to the rectangular target model of the detected object;

[0011] According to the rectangular extended target state of the detected object, the GM-PHD algorithm is used to calculate the k-time multi-target predicted PHD and the k-time multi-target posterior PHD to obtain the extended target tracker.

[0012] Furthermore, the rectangular expansion target state is represented as:

[0013] ξ=(γ,x,X)

[0014] Wherein, ξ represents the state vector of the extended target of the detected object, γ represents the measurement rate state of the extended target of the detected object, x represents the motion state of the extended target of the detected object, and X represents the extended state of the extended target of the detected object.

[0015] Preferably, the process of constructing a bounding box detector includes: preprocessing the lidar data using a RANSAC-based plane fitting algorithm to obtain a target point cloud; clustering the target point cloud using a Euclidean algorithm; constructing a state vector of the bounding box detector based on the clustered target point cloud, and then obtaining a bounding box detector.

[0016] Furthermore, the state vector of the bounding box detector is:

[0017]

[0018] Where x′ represents the state vector, x represents the horizontal coordinate of the detection target, y represents the vertical coordinate of the detection target, v represents the speed of the detection target, θ represents the direction angle of the detection target, ω represents the angular velocity of the detection target, and z represents the vertical coordinate of the detection target. Indicates the vertical velocity of the detected target, L indicates the length of the detected target, W indicates the width of the detected target, and H indicates the height of the detected target.

[0019] Preferably, the process of building a JPDA tracker configured with IMM-UKF includes:

[0020] The JPDA tracker configured with IMM-UKF consists of an input interaction module, a UKF filtering module, a probability update module, a JPDA data association module, and an output fusion module;

[0021] The input interaction module calculates and outputs a second state estimate and a second covariance matrix based on the first state estimate and the first covariance matrix of the UKF filter in the UKF filtering module at time k;

[0022] The UKF filter in the UKF filter module outputs the third state estimate and the third covariance matrix at time k+1 based on the output of the input interaction module and the effective observation vector at time k;

[0023] The probability update module calculates the conditional probability of the motion model at time k+1 based on the residual information of the UKF filter module;

[0024] The JPDA data association module calculates the second measurement information of the target at time k+1 under the motion model based on the third state estimate, the third covariance matrix and the first measurement information of the target under the motion model;

[0025] The output fusion module calculates the fused state estimate and covariance matrix according to the conditional probability of the motion model at time k+1, the second measurement information, the third state estimate and the third covariance matrix.

[0026] Preferably, the formula for processing the central fusion node using the IMF algorithm includes:

[0027] Update the covariance:

[0028]

[0029] Update the state estimate:

[0030]

[0031] Among them, P(k|k) represents the global covariance of the sensor from time 0 to k, P(k|k-1) represents the global covariance of the sensor from time 0 to k-1, and P i (k|k) represents the local covariance of the i-th sensor from time 0 to time k, P i (k|k-1) represents the local covariance of the i-th sensor from time 0 to k-1, N k Indicates the number of sensors, represents the global state estimation of the sensor from time 0 to k, represents the global state estimation of the sensor from time 0 to k-1, represents the local state estimation of the i-th sensor from time 0 to k, represents the local state estimate of the i-th sensor from time 0 to k-1.

[0032] A multi-sensor fusion system for intelligent driving vehicles, the system is used to execute a multi-sensor fusion method for intelligent driving vehicles, comprising: an extended target tracking module, a boundary detector module, a point target tracking module, and a track fusion module;

[0033] The extended target tracking module is used to process the 2D detection information of the millimeter wave radar according to the GM-PHD algorithm and the rectangular target model of the detected object to obtain the millimeter wave radar track information of the detected object;

[0034] The bounding box detector module is used to process the 3D detection information of the lidar based on the RANSAC plane fitting algorithm and the Euclidean algorithm to obtain the 3D information of the detected object;

[0035] The point target tracking module is used to process the 3D information of the detected object using a JPDA tracker configured with IMM-UKF to obtain the lidar track information of the detected object;

[0036] The track fusion module is used to fuse the millimeter wave radar track information of the detected object and the laser radar track information of the detected object to obtain global track information.

[0037] The beneficial effects of the present invention are as follows: for the 2D detection information of the millimeter-wave radar, the present invention adopts the GM-PHD algorithm to construct an extended target tracker to realize the tracking of the detected object track; for the 3D point cloud information of the lidar, the lidar data is preprocessed based on the RANSAC algorithm to remove redundant point clouds, and a bounding box detector is constructed based on the Euclidean distance clustering algorithm. Furthermore, a JPDA tracker configured with IMM-UKF is constructed to realize the tracking of the detected object track, solving the combinatorial explosion problem introduced by the data association method; a state space for track fusion is constructed based on the state space of the lidar, and the conversion from the millimeter-wave radar state space to the track state space is completed. Furthermore, the track fusion of the lidar and the millimeter-wave radar is realized based on the IMF algorithm to form a global track, solving the timing problem caused by the misordering of local track information of different sensors; compared with the existing technology, the present invention can improve the overall perception accuracy, reduce bandwidth requirements and computing costs, and when a sensor fails or fails, another sensor can be used to compensate, which is highly practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the multi-sensor fusion method for intelligent driving vehicles in the present invention;

[0039] Figure 2 Schematic diagram of a rectangular target model in the present invention;

[0040] Figure 3Schematic diagram of the JPDA tracker framework configured with IMM-UKF in the present invention;

[0041] Figure 4 Schematic diagram of the track fusion update process in the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] This paper proposes a multi-sensor fusion method for intelligent driving vehicles, such as Figure 1 As shown, the method includes the following contents:

[0044] S1: An extended target tracker is constructed based on the Gaussian mixture probability hypothesis density (GM-PHD) algorithm and a rectangular target model of the detected object. The extended target tracker is used to process the 2D detection information of the millimeter-wave radar to obtain the millimeter-wave radar track information of the detected object.

[0045] Millimeter-wave radar can easily misdetect a single target as multiple targets. To address this issue, the present invention constructs an extended target tracker based on the GM-PHD algorithm and a rectangular target model of the detected object to track the detected object, i.e., the intelligent driving vehicle. The process of constructing the extended target tracker includes:

[0046] Get the rectangular target model of the detected object, such as Figure 2 As shown, the rectangular extended target state of the detected object is obtained according to the rectangular target model of the detected object; the rectangular extended target state is expressed as:

[0047] ξ=(γ,x,X)

[0048] Where ξ represents the state vector of the extended target of the detected object, γ represents the measurement rate state of the extended target of the detected object, which is a scalar that obeys the gamma distribution; x represents the motion state of the extended target of the detected object, and the vector x can be modeled as x = [x, y, v, θ, ω, L, W] T , [x,y] Trepresents the two-dimensional coordinate position information of the detection target, v, θ, ω represent the speed, direction angle, and angular velocity of the detection target, respectively, and L and W represent the length and width of the detection target, respectively. X represents the extended state of the detection object, that is, the state after the expansion of a single measurement information. When the extended target measurements are randomly distributed in a certain space, a rectangle can usually be used to approximate the extended state of the extended target.

[0049] The GM-PHD filter is constructed based on the rectangular extended target state of the detected object. The GM-PHD filter approximates the multi-target probability hypothesis density through weighted Gaussian components. Assuming that the multi-target posterior PHD at time k-1 can be expressed in the form of a Gaussian mixture:

[0050]

[0051] Among them, J k-1 is the number of Gaussian components at time k-1, is the weight of the i-th Gaussian component, N(·; m, P) represents the probability density function of the Gaussian distribution, is the probability density function of the i-th Gaussian distribution at time k-1, with a mean of The covariance is

[0052] The k-time multi-target prediction PHD and k-time multi-target posterior PHD can also be expressed in the form of Gaussian mixture, which are:

[0053]

[0054]

[0055] Where x represents the motion state of the rectangular extended target state of the detection object, D k|k-1 (x) represents the multi-target prediction PHD at time k, D S,k|k-1 (x) represents the PHD of the surviving Gaussian component at time k, γ k (x) represents the new target at time k, which refers to the new observation point PHD obtained by the sensor. represents the weight of the i-th Gaussian component at time k; Indicates the mean The covariance is Gaussian component, J k|k-1 Denotes the number of Gaussian components predicted at time k, D k|k (x) represents the posterior multi-objective PHD at time k; p D,k It represents the target detection probability, which refers to the probability of making a correct judgment of "signal" when there is a signal at the radar input and the two judgment results of "signal" or "no signal" may be obtained due to noise. represents the weight of the jth update component, z k Represents the target measurement value at time k, which refers to the physical information value such as the target coordinates obtained by the radar; Indicates the mean The covariance is Gaussian component of .

[0056] Prediction of D by GM-PHD algorithm k|k-1 (x) and update D k|k (x) Complete the construction of the GM-PHD filter and obtain a 2-D local tracker suitable for millimeter-wave radar, namely the extended target tracker.

[0057] The extended target tracker is used to process the 2D detection information of the millimeter-wave radar to obtain the millimeter-wave radar track information of the detected object.

[0058] Traditional trackers, such as Joint Probabilistic Data Association (JPDA), generally assume that a sensor outputs at most one measurement per scan of a single target. Consequently, high-resolution sensors equipped with traditional trackers must perform clustering before outputting measurement data through data association. This increases computational cost and significantly impacts measurement accuracy. An extended target tracker, constructed using the GM-PHD algorithm and a rectangular target model, simultaneously handles clustering and data association, reducing computational overhead while maintaining measurement accuracy.

[0059] S2: Construct a bounding box detector and a JPDA tracker equipped with IMM-UKF; use the bounding box detector and the JPDA tracker equipped with IMM-UKF to process the 3D detection information of the lidar to obtain the lidar track information of the detected object.

[0060] LiDAR obtains far more measurement information per target than millimeter-wave radar, and it also obtains ground-based measurement information. The amount of data it outputs is not on the same order of magnitude as that of millimeter-wave radar. Directly using an extended target tracker for processing would result in a significant lag in the output signal due to excessive computational complexity. Therefore, the present invention constructs a bounding box detector to process the 3D detection information from the LiDAR. The process of constructing a bounding box detector includes:

[0061] The lidar data is preprocessed using a plane fitting algorithm based on random sample consensus (RANSAC) to remove redundant point cloud information such as the ground, thereby obtaining the target point cloud and reducing computational overhead. Specifically, three points are randomly selected from the initial point cloud and the sampling plane formed by the three points is calculated. Secondly, the distance from all point clouds to the sampling plane is calculated, a distance threshold is set, and the point clouds are divided into inliers (normal data) and outliers (abnormal data) according to the threshold. Then, the number of inliers and outliers is counted, and the above process is repeated to the maximum number of iterations to select the plane with the most inliers. Finally, based on the plane with the most inliers, the plane is refitted with all its inliers to obtain the final fitting plane equation. According to the final fitting plane equation, redundant point clouds such as ground points are removed to obtain the target point cloud.

[0062] After removing redundant point clouds, the point clouds belonging to each type of main target will appear in a floating and separated state in space. The object is detected using a rectangular bounding box detector. First, the target point cloud in this state is clustered using the Euclidean algorithm. Based on the clustering results, the state vector of the bounding box detector is obtained. The state vector of the bounding box detector is expressed as:

[0063]

[0064] Among them, x′ represents the state vector. Compared with the state vector of the rectangular model of the millimeter-wave radar, the state vector of the rectangular bounding box detector has more z, and H are three variables, z represents the vertical coordinate of the detection target, It represents the vertical velocity of the detected target, and H represents the height of the detected target.

[0065] like Figure 3 As shown in FIG, the process of constructing a joint probabilistic data association (JPDA) tracker configured with an interacting multiple model unscented Kalman filter (IMM-UKF), i.e., a point target tracker, includes: the JPDA tracker configured with IMM-UKF consists of an input interaction module, a UKF filtering module, a probability update module, a JPDA data association module, and an output fusion module, and is used to realize the track tracking of objects detected by the lidar;

[0066] The UKF filter obtains the first state estimate based on the state vector x′ of the bounding box detector at time k

[0067] The input interaction module estimates the first state of the UKF filter at time k in the UKF filter module and the first covariance matrix Calculate the second state estimate after multiple targets interact and the second covariance matrix And output, where j represents the motion model, which is a velocity model constructed according to the motion state of the target, i = 1, 2, ..., r, r is the number of UKF filters;

[0068] The UKF filter in the UKF filter module is based on the output of the input interaction module and the effective observation vector Z at time k. k , output the third state estimate at time k+1 and the third covariance matrix

[0069] The probability update module calculates the conditional probability of motion model j at time k+1 based on the residual information of the UKF filter module in is the residual information of the UKF filter;

[0070] The JPDA data association module is based on the first measurement information of the target under the motion model. And the associated probability (third state estimate and third covariance matrix), calculate the second measurement information of target k+1 under motion model j Among them, the first measurement information is measurement information such as target vehicle speed;

[0071] The output fusion module is based on the conditional probability of motion model j at time k+1 Second measurement information The third state estimate and the third covariance matrix Calculate the fused state estimate and covariance matrix, so that the UKF filter output with small estimation error becomes the fused tracking output, and then obtain the fused state estimate and the covariance matrix P(k+1).

[0072] After the above steps, a 3-D local tracker suitable for lidar can be obtained, namely the JPDA tracker configured with IMM-UKF.

[0073] By processing 3D lidar detection information using a bounding box detector and a JPDA tracker equipped with IMM-UKF, lidar tracks of detected objects can be obtained. This method reduces computational overhead without increasing hardware costs, effectively improving the lag in lidar track output.

[0074] S3: Use time-space conversion to process the millimeter-wave radar track information and the lidar track information to obtain a central fusion node; use the IMF algorithm to process the central fusion node to obtain global track information; and track the detected object based on the global track information.

[0075] The millimeter-wave radar and lidar measurement information is processed by a 2-D local tracker and a 3-D local tracker, respectively, to form two local tracks: the millimeter-wave radar track of the detected object and the millimeter-wave radar track of the detected object. Spatiotemporal transformation is used to unify the local track information into the same coordinate system and time node, and then a fusion center node is constructed to achieve data association between the different track information. Since the data measured by the millimeter-wave radar and lidar may be subject to delayed processing time and communication delays during transmission, this can cause timing issues caused by misordering of the local track information. To address this issue, the covariance between the two sensors must be calculated and then duplicate values must be subtracted. To address this issue, the fusion center node uses an information matrix fusion (IMF) algorithm to update the 2-D and 3-D local tracks.

[0076] The track update process is as follows Figure 4 As shown. At time k-2, the global track information only contains the information from 0 to k i -1 time 2-D local track information. At time k-1, the global track information is integrated with the existing information from 0 to k. j 3-D local track information at time t, in other words, global track information contains 0 to k j However, at time k, if the global track information is directly integrated with k i The 2-D local track information at the moment will cause repeated fusion of information. Because the above fused information is in addition to the k i -1 to k i In addition to the 2-D local track information at time, there are also repeated 0 to k i -1 2-D local track information.

[0077] The IMF algorithm can avoid repeated fusion of track information during the update process. It updates the fused track data by updating the covariance and state estimation, preventing old data from entering the new target and thus avoiding repeated fusion. The algorithm flow is as follows:

[0078] Update the covariance:

[0079]

[0080] Update the state estimate:

[0081]

[0082] Among them, P(k|k) represents the global covariance of the sensor from time 0 to k, corresponding to the global track information; P(k|k-1) represents the global covariance of the sensor from time 0 to k-1; P i (k|k) represents the local covariance of the i-th sensor from time 0 to time k, corresponding to the local track information; P i (k|k-1) represents the local covariance of the i-th sensor from time 0 to k-1, N k Indicates the number of sensors, represents the global state estimation of the sensor from time 0 to k, represents the global state estimation of the sensor from time 0 to k-1, represents the local state estimation of the i-th sensor from time 0 to k, represents the local state estimate of the i-th sensor from time 0 to k-1; where the sensor refers to millimeter wave radar or lidar.

[0083] The IMF algorithm is used to fuse two local tracks to obtain global track information. Users can track the intelligent driving vehicle based on the global track information.

[0084] The present invention also provides a multi-sensor fusion system for intelligent driving vehicles, which is used to execute a multi-sensor fusion method for intelligent driving vehicles, including: an extended target tracking module, a boundary detector module, a point target tracking module and a track fusion module;

[0085] The extended target tracking module is used to process the 2D detection information of the millimeter wave radar according to the GM-PHD algorithm and the rectangular target model of the detected object to obtain the millimeter wave radar track information of the detected object;

[0086] The bounding box detector module is used to process the 3D detection information of the lidar based on the RANSAC plane fitting algorithm and the Euclidean algorithm to obtain the 3D information of the detected object;

[0087] The point target tracking module is used to process the 3D information of the detected object using a JPDA tracker configured with IMM-UKF to obtain the lidar track information of the detected object;

[0088] The track fusion module is used to fuse the millimeter wave radar track information of the detected object and the laser radar track information of the detected object to obtain global track information.

[0089] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-sensor fusion method for intelligent driving vehicles, characterized in that: include: S1: Construct an extended target tracker based on the GM-PHD algorithm and the rectangular target model of the detected object; The extended target tracker is used to process the 2D detection information of the millimeter-wave radar to obtain the millimeter-wave radar track information of the detected object; The process of building an extended object tracker involves: Obtaining a rectangular extended target state of the detected object according to the rectangular target model of the detected object; According to the rectangular extended target state of the detected object, the GM-PHD algorithm is used to calculate the k-time multi-target predicted PHD and k-time multi-target posterior PHD to obtain the extended target tracker. S2: Construct a bounding box detector and a JPDA tracker with IMM-UKF. Use the bounding box detector and the JPDA tracker with IMM-UKF to process the 3D detection information of the lidar to obtain the lidar track information of the detected object. The process of constructing a bounding box detector includes: preprocessing the lidar data using a RANSAC-based plane fitting algorithm to obtain a target point cloud; clustering the target point cloud using the Euclidean algorithm; constructing the state vector of the bounding box detector based on the clustered target point cloud, and then obtaining the bounding box detector; The process of building a JPDA tracker configured with IMM-UKF includes: The JPDA tracker configured with IMM-UKF consists of an input interaction module, a UKF filtering module, a probability update module, a JPDA data association module, and an output fusion module; The input interaction module calculates and outputs a second state estimate and a second covariance matrix based on the first state estimate and the first covariance matrix of the UKF filter in the UKF filtering module at time k; The UKF filter in the UKF filter module outputs the third state estimate and the third covariance matrix at time k+1 based on the output of the input interaction module and the effective observation vector at time k; The probability update module calculates the conditional probability of the motion model at time k+1 based on the residual information of the UKF filter module; The JPDA data association module calculates the second measurement information of the target at time k+1 under the motion model based on the third state estimate, the third covariance matrix and the first measurement information of the target under the motion model; The output fusion module calculates the fused state estimate and covariance matrix based on the conditional probability of the motion model at time k+1, the second measurement information, the third state estimate and the third covariance matrix; S3: Use time-space conversion to process the millimeter-wave radar track information and the lidar track information to obtain a central fusion node; use the IMF algorithm to process the central fusion node to obtain global track information; and track the detected object based on the global track information.

2. The multi-sensor fusion method for intelligent driving vehicles according to claim 1, characterized in that: The rectangular extended target state is represented as: ξ=(γ,x,X) Wherein, ξ represents the state of the extended target of the detected object, γ represents the measurement rate state of the extended target of the detected object, x represents the motion state of the extended target of the detected object, and X represents the extended state of the extended target of the detected object.

3. The multi-sensor fusion method for intelligent driving vehicles according to claim 1, characterized in that: The state vector of the bounding box detector is: Where x′ represents the state vector, x represents the horizontal coordinate of the detection target, y represents the vertical coordinate of the detection target, v represents the speed of the detection target, θ represents the direction angle of the detection target, ω represents the angular velocity of the detection target, and z represents the vertical coordinate of the detection target. Indicates the vertical velocity of the detected target, L indicates the length of the detected target, W indicates the width of the detected target, and H indicates the height of the detected target.

4. The multi-sensor fusion method for intelligent driving vehicles according to claim 1, characterized in that: The formulas for processing the central fusion node using the IMF algorithm include: Update the covariance: Update the state estimate: Among them, P(k|k) represents the global covariance of the sensor from time 0 to k, P(k|k-1) represents the global covariance of the sensor from time 0 to k-1, and P i (k|k) represents the local covariance of the i-th sensor from time 0 to time k, P i (k|k-1) represents the local covariance of the i-th sensor from time 0 to k-1, N k Indicates the number of sensors, represents the global state estimation of the sensor from time 0 to k, represents the global state estimation of the sensor from time 0 to k-1, represents the local state estimation of the i-th sensor from time 0 to k, represents the local state estimate of the i-th sensor from time 0 to k-1.

5. A multi-sensor fusion system for intelligent driving vehicles, the system being used to execute the multi-sensor fusion method for intelligent driving vehicles according to any one of claims 1 to 4, characterized in that: include: Extended target tracking module, boundary detector module, point target tracking module and track fusion module; The extended target tracking module is used to process the 2D detection information of the millimeter wave radar according to the GM-PHD algorithm and the rectangular target model of the detected object to obtain the millimeter wave radar track information of the detected object; The bounding box detector module is used to process the 3D detection information of the lidar based on the RANSAC plane fitting algorithm and the Euclidean algorithm to obtain the 3D information of the detected object; The point target tracking module is used to process the 3D information of the detected object using a JPDA tracker configured with IMM-UKF to obtain the lidar track information of the detected object; The track fusion module is used to fuse the millimeter wave radar track information of the detected object and the laser radar track information of the detected object to obtain global track information.

Images