A vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm

Abstract

The present invention discloses a vehicle 3D multi-target tracking method based on a convolutional neural network combined with a greedy algorithm, including: pre-processing laser radar point cloud data on a highway; inputting the processed point cloud into a point cloud vehicle target detection neural network model to generate a vehicle target detection state value; using a convolutional neural network to input the vehicle target state and predict the predicted value of the next state; setting a Euclidean distance filter gate to screen effective prediction values; setting an association metric threshold, and using a greedy algorithm to perform data association for detection values ​​with interference in the association; using a kinematic model to associate with an initialization trajectory for detection values ​​with low detection confidence; updating the state of the associated vehicle trajectory, and using a trajectory management module to generate a new trajectory and an extinction trajectory. The present invention has a lightweight structure, reduces the amount of equipment calculation, has strong robustness, and is applicable to highway vehicle multi-target tracking based on laser point cloud data.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent traffic information technology, and specifically relates to a vehicle 3D multi-target tracking method based on a convolutional neural network combined with a greedy algorithm, which can be applied to highway vehicle multi-target tracking based on laser point cloud data. Background Art

[0002] With the continuous development of intelligent technology, visual sensors have become an important part of robotics. As a type of visual sensor technology, LiDAR point cloud technology plays an important role in multi-target tracking technology. Vehicle target tracking refers to "tracking" multiple car targets to capture information such as the target's moving direction and position, thereby providing users with accurate positioning and safety guarantees. Compared with traditional roadside monitoring equipment such as cameras and speed radars, LiDAR does not rely on ambient light sources, has strong robustness, and can obtain three-dimensional coordinate information of stationary and moving objects, as well as surface structure feature information of the surrounding environment in a fast and high-precision manner. Deploying multiple LiDARs on the roadside can obtain beyond-line-of-sight perception and accelerate the safe implementation of vehicle-road collaboration and autonomous driving.

[0003] By reviewing patents and papers related to vehicle 3D multi-target tracking, most existing methods use the route of detection first and then tracking, which leads to a high degree of coupling of the entire network and is dependent on the quality of the detection network, and cannot reflect the advanced nature of the tracking network. At the same time, due to the coupling of the entire network, point cloud computing is highly complex and computationally intensive, and it is difficult to ensure real-time performance when deployed on the roadside. At the same time, multi-target tracking often results in mutual occlusion. For the Kalman filter commonly used in 3D target tracking, when the vehicle is occluded, the state estimation confidence is low, and the vehicle tracking effect is insufficient.

[0004] Therefore, how to achieve a lightweight network and improve the tracking effect under occlusion conditions is of great significance for multi-target tracking of LiDAR. Summary of the invention

[0005] In view of this, the present invention provides a vehicle 3D multi-target tracking method based on a convolutional neural network combined with a greedy algorithm.

[0006] The objective of the present invention is achieved through the following technical solutions:

[0007] 1. A vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm, characterized by comprising:

[0008] Step 1: Collect vehicle point cloud data through highway road test laser radar and pre-process the point cloud data;

[0009] Step 2: Input the preprocessed point cloud data into the point cloud vehicle target detector to generate a vehicle target detection value, that is, the detection target state information. The vehicle target detection value includes the vehicle spatial position, geometric features and kinematic features. Generate a bounding box based on the detection value and save it. At any Test time, the vehicle target detection value is expressed as D = {s1, s2, ..., s N}, where s j =(x,y,z,l,w,h,θ,v x ,v y ,v z ,v θ ,T), j∈{1…N}, x, y, z represent the geometric center coordinates of the target; l, w, h represent the length, width and height of the target respectively; θ represents the orientation angle of the target; v x ,v y ,v z Respectively represent the components of the target's velocity on each coordinate axis; v θ represents the angular velocity of the target; T represents the target detection confidence; N represents the number of detected targets; j represents the target sequence number.

[0010] Step 3: Input the vehicle target detection value into the convolutional neural network to generate a prediction value for the next state.

[0011] Step 4: Set the Euclidean distance filter gate, calculate the Euclidean distance between the predicted value and the detected value, and filter the valid detected value;

[0012] Step 5: Set the association metric threshold, generate the Mahalanobis distance between the predicted value and the detected value, and generate matching pairs;

[0013] Step 6: For the matching pairs with high detection confidence, use the greedy algorithm to perform data association; for the matching pairs with low detection confidence, use the kinematic model to replace the low-quality detection with motion prediction and associate with the initialization trajectory;

[0014] Step 7: Update the status of the associated vehicle trajectories; use the trajectory management module to generate new trajectories and extinction trajectories.

[0015] Select the lightweight convolutional neural network MobilenetV2, take 4 consecutive frames of vehicle status information as input, extract the motion information in the continuous process, and add BatchNorm and Mish activation functions to each Bottleneck convolution layer:

[0016] f(X)=X*tanh(ln(1+e X )),

[0017] Where f(X) represents the Mish activation function, and X represents the state information of the input Mish activation function;

[0018] The convolutional neural network MobilenetV2 directly predicts the target state at the Test time and generates the predicted target state in, It represents the predicted state information of the i-th target at that moment, and M represents the number of predicted targets.

[0019] Furthermore, the step 4 specifically includes:

[0020] Step 4.1: At time k, calculate the Euclidean distance between the predicted target geometric center coordinates and the detected target geometric center coordinates Among them, s′ j (k) indicates that the j-th target obtained by the detector at time k only contains the state information of (x, y, z); Indicates that the predicted i-th target at time k only contains (x, y, z) state estimation information;

[0021] Step 4.2: Set the Euclidean distance filter gate, perform Euclidean distance filtering, screen the valid detection values, and discard the detection values ​​that do not meet the Euclidean distance gate, that is, screen out the detection values ​​that meet the following formula,

[0022] d e (i,j)<γ e ,

[0023] Among them, γ e Represents the Euclidean distance threshold.

[0024] Furthermore, the step 5 specifically includes:

[0025] Step 5.1: Calculate the Mahalanobis distance d between the predicted state after filtering by the Euclidean distance filter gate at time k and the detected state detected by the detector m (i,j) as the association metric,

[0026]

[0027] in,

[0028] s″ j (k) represents the j-th target removal angular velocity v obtained by the detector at time k θ The status information after

[0029] It represents the predicted angular velocity v of the i-th target removed at time k θ The status information after

[0030] S-1 (k) The corresponding covariance;

[0031] Step 5.2: Filter the predicted values ​​within the Mahalanobis distance threshold range, and retain the combination of detection values ​​and predicted values ​​within the Mahalanobis distance threshold range to generate matching pairs (i, j), that is, filter out the predicted values ​​that satisfy the following formula,

[0032] pairs(i,j)=d m (i,j)<γ m ,

[0033] Among them, γ m is the set Mahalanobis distance threshold.

[0034] Furthermore, the step 6 specifically includes:

[0035] Step 6.1: According to the matching pairs (m, n) obtained according to the Mahalanobis distance threshold in step 5, when the Mahalanobis distance threshold γ m When there is only one detection value within the range, the trajectory association is performed directly;

[0036] Step 6.2: When there are multiple predicted values ​​and detected values ​​that simultaneously meet the Mahalanobis distance threshold γ m When , the greedy algorithm is used to realize trajectory association, and the greedy algorithm is expressed by the following formula:

[0037] MatchedP=MatchedP∪{m},

[0038] MatchedD=MatchedD∪{n},

[0039] (m,n)∈{Value min [pairs(i,j)],...,Value max [pairs(i,j)]},

[0040] Among them, matchedP represents the set of predicted values ​​for successful trajectory association; matchedD represents the set of detected values ​​for successful trajectory association; Value min [pairs(i,j)] represents the m,n corresponding to the minimum Mahalanobis distance value; Value max [pairs(i,j)] represents m,n corresponding to the maximum Mahalanobis distance value; m,n represent the unmatched prediction target and detection target respectively;

[0041] Step 6.3: The detection confidence of the detection value in the matching pair is [T low ,T high], using the kinematic model to associate with the initial trajectory, using motion prediction to replace low-quality detection, and then using Kalman filtering to update,

[0042]

[0043] P(k|k-1)=AP(k-1|k-1)A T +Q,

[0044]

[0045] in, represents the state estimation at time k from time k-1; s(k-1) represents the state at time k-1, A is the state transfer matrix; W(k) is the process Gaussian noise at time k, and the covariance is Q; P(k) represents the covariance corresponding to s(k); P(k-1|k-1) represents the optimal state estimation covariance at time k-1; P(k|k-1) represents the prior state estimation covariance at time k; z(k) is the observation vector; H is the observation matrix; V(k) is the process Gaussian noise, and the covariance is R;

[0046]

[0047] s(k|k)=s(k|k-1)+K g (k)[z(k)-Hs(k|k-1)],

[0048] P(k|k)=[1-K g (k)H]P(k|k-1),

[0049] Among them, K g (k) represents the Kalman gain, s(k|k) represents the optimal state estimate at time k, and s(k|k-1) represents the prior state estimate at time k.

[0050] Furthermore, the step 7 specifically includes:

[0051] Step 7.1: Set up the track manager. If there are frames exceeding the threshold T born The unmatched detection value s j , define it as a new target, and its initial trajectory state is s j ;

[0052] Step 7.2: For unmatched predictions If the number of frames exceeds the threshold T dead , it is defined as leaving the target trajectory and deleted.

[0053] The beneficial effects of the present invention are:

[0054] The present invention uses the laser radar point cloud data on the highway and proposes a vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm. It uses MobilenetV2 convolutional neural network for state prediction and adopts Euclidean distance filtering to reduce the amount of associated operations to achieve 3D multi-target tracking. It avoids the defects of the existing method such as complex model, low real-time performance, and poor tracking effect in occlusion situations, and can be applied to highway road tests to achieve vehicle multi-target tracking.

[0055] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0057] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0058] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0059] like Figure 1 As shown, the present embodiment provides a vehicle 3D multi-target tracking method based on a convolutional neural network combined with a greedy algorithm, comprising the following steps:

[0060] Step 1: Collect vehicle point cloud data through highway road test laser radar and pre-process the point cloud data. The pre-processing may include cleaning the vehicle point cloud data (also called "raw point cloud data"), removing outliers, filtering out ground point clouds, and retaining valid point clouds.

[0061] Step 2: Input the preprocessed point cloud data into the point cloud vehicle target detector (also called "point cloud vehicle target detection neural network model") to generate a vehicle target detection value, that is, the detection target state information.

[0062] In some embodiments, the PV-RCNN point cloud target detection algorithm can be used to detect the input electric cloud data. After the point cloud data is input to the detector, the vehicle target detection value is output. The vehicle target detection value includes the vehicle spatial position, geometric features, kinematic features, and the confidence of the detection. According to the detection value, the bounding box of the detection target can also be generated and saved. At any Test time, the vehicle target detection value is expressed as D = {s1, s2, ..., s N}, where s j =(x,y,z,l,w,h,θ,v x ,v y ,v z ,v θ ,T), j∈{1…N}, x, y, z represent the geometric center coordinates of the target; l, w, h represent the length, width and height of the target respectively; θ represents the orientation angle of the target; v x ,v y ,v z Respectively represent the components of the target's velocity on each coordinate axis; v θ represents the angular velocity of the target; T represents the confidence level of target detection; N represents the number of detected targets; j represents the target sequence number;

[0063] Step 3: Input the vehicle target detection value into the convolutional neural network to generate a prediction value for the next state. In some embodiments, a lightweight convolutional neural network MobilenetV2 can be selected to take 4 consecutive frames of vehicle state information as input to extract motion information in the continuous process (for example, the data of frames 1-4 can be used to generate the prediction result of frame 5, and the data of frames 2-5 can be used to generate the prediction result of frame 6, and so on). In some embodiments, a total of 16 groups of shuffle channels can be designed to improve prediction performance. Add BatchNorm and Mish activation functions to each Bottleneck convolution layer:

[0064] f(X)=X*tanh(ln(1+e X )),

[0065] Among them, f(X) represents the Mish activation function, and X represents the state information of the input Mish activation function.

[0066] In some embodiments, the network may be trained using a mean square error loss function,

[0067]

[0068] Where n represents the training sample capacity.

[0069] The results of each Bottleneck convolutional layer are shown in Table 1.

[0070] Table 1

[0071]

[0072] Where C represents the dimension of the state vector.

[0073] In some embodiments, the convolutional neural network MobilenetV2 can be used to directly predict the target state at the Test time and generate a predicted target state in, Represents the state information of the i-th target at the predicted moment, where each element represents the result of the prediction, and its specific physical meaning is similar to that in step 2; M represents the number of predicted targets.

[0074] Step 4: Set the Euclidean distance filter gate, calculate the Euclidean distance between the predicted value and the detected value, and filter the valid detected value. In some embodiments, step 4 may specifically include:

[0075] Step 4.1: At time k, calculate the Euclidean distance between the predicted target geometric center coordinates (the prediction result obtained in step 3) and the detected target geometric center coordinates (the detection result obtained in step 2) Among them, s′ j (k) indicates that the j-th target obtained by the detector at time k only contains the state information of (x, y, z); Indicates that the predicted i-th target at time k only contains (x, y, z) state estimation information (i.e. predicted state information);

[0076] Step 4.2: Set the Euclidean distance filter gate, perform Euclidean distance filtering, screen the valid detection values, and discard the detection values ​​that do not meet the Euclidean distance gate, that is, screen out the detection values ​​that meet the following formula,

[0077] d e (i,j)<γ e ,

[0078] Among them, γ e Represents the Euclidean distance threshold.

[0079] Step 5: Setting an association metric threshold, generating a Mahalanobis distance between a predicted value and a detected value, and generating a matching pair. In some embodiments, step 5 may specifically include:

[0080] Step 5.1: Calculate the Mahalanobis distance d between the predicted state after filtering by the Euclidean distance filter gate at time k and the detected state detected by the detector m (i,j) as the association metric,

[0081]

[0082] in,

[0083] s″ j (k) represents the j-th target removal angular velocity v obtained by the detector at time k θ The status information after

[0084] It represents the predicted angular velocity v of the i-th target removed at time k θ The status information after

[0085] S- 1 (k) The corresponding covariance;

[0086] Step 5.2: Filter the predicted values ​​within the Mahalanobis distance threshold range, and retain the combination of detection values ​​and predicted values ​​within the Mahalanobis distance threshold range to generate matching pairs (i, j), that is, filter out the predicted values ​​that satisfy the following formula,

[0087] pairs(i,j)=d m (i,j)<γ m ,

[0088] Among them, γ m is the set Mahalanobis distance threshold.

[0089] Step 6: adopt two-stage association, for the matching pairs with high detection confidence of the detection value in the matching pairs, use the greedy algorithm to perform data association; for the matching pairs with low detection confidence of the detection value in the matching pairs, use the kinematic model, use motion prediction to replace the low-quality detection, and associate with the initialization trajectory. In some embodiments, step 6 may specifically include:

[0090] Step 6.1: According to the matching pairs (m, n) obtained according to the Mahalanobis distance threshold in step 5, when the Mahalanobis distance threshold γ m When there is only one detection value within the range, trajectory association is performed directly;

[0091] Step 6.2: When there are multiple predicted values ​​and detected values ​​that simultaneously meet the Mahalanobis distance threshold γ m When , the greedy algorithm is used to realize trajectory association. The greedy algorithm is expressed by the following formula:

[0092] MatchedP=MatchedP∪{m},

[0093] MatchedD=MatchedD∪{n},

[0094] (m,n)∈{Value min[pairs(i,j)],...,Value max [pairs(i,j)]},

[0095] Among them, matchedP represents the set of predicted values ​​for successful trajectory association; matchedD represents the set of detected values ​​for successful trajectory association; Value min [pairs(i,j)] represents the m,n corresponding to the minimum Mahalanobis distance value; Value max [pairs(i,j)] represents m,n corresponding to the maximum Mahalanobis distance value; m,n represent the unmatched prediction target and detection target respectively;

[0096] Step 6.3: The detection confidence of the detection value in the matching pair is [T low ,T high ], using the kinematic model to associate with the initial trajectory, using motion prediction instead of low-quality detection:

[0097]

[0098] Then the Kalman filter is used to update.

[0099]

[0100] P(k|k-1)=AP(k-1|k-1)A T +Q,

[0101]

[0102] in, represents the state estimation at time k from time k-1; s(k-1) represents the state at time k-1, A is the state transfer matrix; W(k) is the process Gaussian noise at time k, and the covariance is Q; P(k) represents the covariance corresponding to s(k); P(k-1|k-1) represents the optimal state estimation covariance at time k-1; P(k|k-1) represents the prior state estimation covariance at time k; z(k) is the observation vector; H is the observation matrix; V(k) is the process Gaussian noise, and the covariance is R;

[0103]

[0104] s(k|k)=s(k|k-1)+K g (k)[z(k)-Hs(k|k-1)],

[0105] P(k|k)=[1-K g (k)H]P(k|k-1),

[0106] Among them, Kg (k) represents the Kalman gain, s(k|k) represents the optimal state estimate at time k, and s(k|k-1) represents the prior state estimate at time k.

[0107] Step 7: Update the status of the associated vehicle trajectory; use the trajectory management module to generate new trajectories and extinction trajectories. In some embodiments, step 7 may specifically include:

[0108] Step 7.1: Set up the track manager. If the number of frames exceeds the set threshold T born The unmatched detection value s j , and s j ∈UnmatchedD, which is defined as a new target, and its initial state is s j , UnmatchedD represents the set of detection values ​​that have not been successfully matched;

[0109] Step 7.2: For unmatched predictions Right now If the number of frames exceeds the set threshold T dead , it is defined as leaving the target trajectory and deleted, where UnmatchedP represents the set of prediction values ​​that have not been successfully matched.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm, characterized in that: include: Step 1: Collect vehicle point cloud data through highway road test laser radar and pre-process the point cloud data; Step 2: Input the preprocessed point cloud data into the point cloud vehicle target detector to generate a vehicle target detection value, that is, the detection target state information. The vehicle target detection value includes the vehicle spatial position, geometric features and kinematic features. Generate a bounding box based on the detection value and save it. At any Test time, the vehicle target detection value is expressed as D = {s1, s2, ..., s N }, where s j =(x,y,z,l,w,h,θ,v x ,v y ,v z ,v θ ,T), j∈{1…N}, x, y, z represent the geometric center coordinates of the target; l, w, h represent the length, width and height of the target respectively; θ represents the orientation angle of the target; v x ,v y ,v z Respectively represent the components of the target's velocity on each coordinate axis; v θ represents the angular velocity of the target; T represents the target detection confidence; N represents the number of detected targets; j represents the target sequence number; Step 3: Input the vehicle target detection value into the convolutional neural network to generate a prediction value for the next state; Step 4: Set the Euclidean distance filter gate, calculate the Euclidean distance between the predicted value and the detected value, and filter the valid detected value; Step 5: Set the association metric threshold, generate the Mahalanobis distance between the predicted value and the detected value, and generate matching pairs; Step 6: For matching pairs with high detection confidence, use the greedy algorithm to associate data; for matching pairs with low detection confidence, use the kinematic model to replace low-quality detection with motion prediction and associate with the initialization trajectory. The detection confidence of the detection value is high, that is, the detection value satisfies the Mahalanobis distance threshold γ m of matching pairs; The matching pairs with lower detection confidence of the detection value, that is, the detection confidence of the detection value is between [T low ,T high ]; Step 7: Update the status of the associated vehicle trajectories; use the trajectory management module to generate new trajectories and extinction trajectories.

2. The vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm according to claim 1 is characterized in that: The step 1 includes cleaning the vehicle point cloud data, filtering out the ground point cloud, and retaining the valid point cloud.

3. The vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm according to claim 1 is characterized in that: The step 3 specifically includes: Select the lightweight convolutional neural network MobilenetV2, take 4 consecutive frames of vehicle status information as input, extract the motion information in the continuous process, and add BatchNorm and Mish activation functions to each Bottleneck convolution layer: f(X)=X*tanh(ln(1+e X )), Where f(X) represents the Mish activation function, and X represents the state information of the input Mish activation function; The convolutional neural network MobilenetV2 directly predicts the target state at the Test time and generates the predicted target state in, It represents the predicted state information of the i-th target at that moment, and M represents the number of predicted targets.

4. The vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm according to claim 3 is characterized in that: The step 4 specifically includes: Step 4.1: At time k, calculate the Euclidean distance between the predicted target geometric center coordinates and the detected target geometric center coordinates Among them, s j ′(k) means that the jth target obtained by the detector at time k only contains the state information of (x, y, z); Indicates that the predicted i-th target at time k only contains (x, y, z) state estimation information; Step 4.2: Set the Euclidean distance filter gate, perform Euclidean distance filtering, screen the valid detection values, and discard the detection values ​​that do not meet the Euclidean distance gate, that is, screen out the detection values ​​that meet the following formula, d e (i,j)<γ e , Among them, γ e Represents the Euclidean distance threshold.

5. The vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm according to claim 4 is characterized in that: The step 5 specifically includes: Step 5.1: Calculate the Mahalanobis distance d between the predicted state after filtering by the Euclidean distance filter gate at time k and the detected state detected by the detector m (i,j) as the association metric, in, s j ″(k) represents the j-th target removal angular velocity v obtained by the detector at time k θ The status information after It represents the predicted angular velocity v of the i-th target removed at time k θ The status information after S -1 (k) The corresponding covariance; Step 5.2: Filter the predicted values ​​within the Mahalanobis distance threshold range, and retain the combination of detection values ​​and predicted values ​​within the Mahalanobis distance threshold range to generate matching pairs (i, j), that is, filter out the predicted values ​​that satisfy the following formula, pairs(i,j)=d m (i,j)<γ m , Among them, γ m is the set Mahalanobis distance threshold.

6. The vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm according to claim 5 is characterized in that: The step 6 specifically includes: Step 6.1: According to the matching pairs (m, n) obtained according to the Mahalanobis distance threshold in step 5, when the Mahalanobis distance threshold γ m When there is only one detection value within the range, the trajectory association is performed directly; Step 6.2: When there are multiple predicted values ​​and detected values ​​that simultaneously meet the Mahalanobis distance threshold γ m When , the greedy algorithm is used to realize trajectory association, and the greedy algorithm is expressed by the following formula: MatchedP=MatchedP∪{m}, MatchedD=MatchedD∪{n}, (m,n)∈{Value min [pairs(i,j)],...,Value max [pairs(i,j)]}, Among them, matchedP represents the set of predicted values ​​for successful trajectory association; matchedD represents the set of detected values ​​for successful trajectory association; Value min [pairs(i,j)] represents the m,n corresponding to the minimum Mahalanobis distance value; Value max [pairs(i,j)] represents m,n corresponding to the maximum Mahalanobis distance value; m,n represent the unmatched predicted target and detected target respectively; Step 6.3: The detection confidence of the detection value in the matching pair is [T low ,T high ], using the kinematic model to associate with the initial trajectory, using motion prediction to replace low-quality detection, and then using Kalman filtering to update, P(k|k-1)=AP(k-1|k-1)A T +Q, in, represents the state estimation at time k from time k-1; s(k-1) represents the state at time k-1, A is the state transfer matrix; W(k) is the process Gaussian noise at time k, and the covariance is Q; P(k) represents the covariance corresponding to s(k); P(k-1|k-1) represents the optimal state estimation covariance at time k-1; P(k|k-1) represents the prior state estimation covariance at time k; z(k) is the observation vector; H is the observation matrix; V(k) is the process Gaussian noise, and the covariance is R; s(k|k)=s(k|k-1)+K g (k)[z(k)-Hs(k|k-1)], P(k|k)=[1-K g (k)H]P(k|k-1), Among them, K g (k) represents the Kalman gain, s(k|k) represents the optimal state estimate at time k, and s(k|k-1) represents the prior state estimate at time k.

7. The vehicle 3D multi-target tracking method based on convolutional neural network combined with greedy algorithm according to claim 1 is characterized in that: The step 7 specifically includes: Step 7.1: Set up the track manager. If there are frames exceeding the threshold T born The unmatched detection value s j , define it as a new target, and its initial trajectory state is s j ; Step 7.2: For unmatched predictions If the number of frames exceeds the threshold T dead , it is defined as leaving the target trajectory and deleted.

Images