A multi-target tracking method for intelligent driving

Abstract

The application discloses a kind of multi-target tracking methods for intelligent driving, including detection network, multi-scale re-identification network, hierarchical matching mechanism and trajectory optimization network, the detection network of the present application is based on YOLOv5, attention mechanism is integrated into its backbone network, and four times down-sampling tiny target prediction head is added in its head network, the ability of important feature positioning and tiny target detection is improved, for the problem that similar target features are difficult to distinguish due to high similarity, the bottleneck layer of multi-scale re-identification network adopts multi-flow structure, and uses attention mechanism to aggregate multi-flow features, realize dynamic fusion of multi-scale features, improve the expression ability of target appearance feature, in order to reduce the situation that target confidence is reduced when occlusion, causing missed detection, hierarchical matching mechanism is designed in data association. Finally, two plug-and-play lightweight models are also applied for trajectory global linking and smoothing completion.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a multi-target tracking method for intelligent driving. BACKGROUND

[0002] Multi-target tracking, as a key computer vision task, has a wide range of applications in intelligent driving, video monitoring, people counting, etc. The main purpose of multi-target tracking is to assign the same identity ID to the same target in each frame of the input video sequence to achieve data association and generate target trajectories.

[0003] Multi-target tracking is an important part of visual scene understanding. The challenges of difficult target detection, serious mutual occlusion and high similarity between targets seriously affect the precision and accuracy of multi-target tracking models. SUMMARY

[0004] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, the purpose of the present application is to provide a multi-target tracking method for intelligent driving.

[0005] The purpose of the present application is achieved by the following technical solutions:

[0006] A multi-target tracking method for intelligent driving, comprising:

[0007] A detection network for identifying people and vehicles in the input video sequence and generating bounding boxes and foreground confidence of the bounding boxes, the detection network comprising an input end, a backbone network, a neck network and a head network, the backbone network incorporating a coordinate attention mechanism;

[0008] A multi-scale re-identification network for multi-scale feature extraction of the bounding boxes generated by the detection network and the bounding boxes of the historical trajectories, generating an appearance feature vector; the multi-scale re-identification network comprising a bottleneck network composed of multiple convolution feature streams with different receptive fields;

[0009] The hierarchical matching mechanism: first, the detection frame generated by the detection network is divided into high-score detection frame and low-score detection frame according to the foreground confidence, the historical trajectory is predicted by Kalman filtering algorithm to generate a prediction frame, and the Mahalanobis distance is calculated with the high-score detection frame to obtain the motion feature correlation measure, then the feature vector of the high-score detection frame and the feature vector of the surrounding box of the last N frames of the historical trajectory are extracted by the multi-scale re-identification network, and the minimum cosine distance between the high-score detection frame and the surrounding box is calculated to obtain the appearance feature correlation measure, then the motion feature and the appearance feature correlation measure are linearly weighted to obtain the cost matrix, and the Hungarian algorithm is used for cascade matching, then the intersection over union matching is performed on the historical trajectory and the high-score detection frame which fails to match, and finally the intersection over union matching is performed again on the low-score detection frame and the trajectory which fails to match, the historical trajectory which successfully matches in any one time is updated by Kalman filtering, and the historical trajectory which fails to successfully match is deleted;

[0010] The trajectory optimization network: including an appearance-free linking model and a Gaussian smoothing interpolation model, the appearance-free linking model globally correlates all trajectories, and the Gaussian smoothing interpolation model smoothes and completes the blank trajectory, so that the obtained trajectory is closer to the real trajectory.

[0011] Further, the down-sampling rate of the head network is 4 times down-sampling branch, 8 times down-sampling branch, 16 times down-sampling branch and 32 times down-sampling branch.

[0012] Further, the feature scale concerned by each stream in the bottleneck network is determined by an index, a CBAM attention mechanism is used as a unified aggregation gate of each feature stream to generate a channel weight for dynamic fusion, and finally the original input and the output after fusion of all streams are connected in residual to obtain the final output.

[0013] Further, the multi-scale re-identification network further includes a lightweight component composed of a lightweight convolution, batch normalization and an activation function.

[0014] Further, the lightweight convolution is a LiteCONV module realized by a depth separable convolution, and the depth separable convolution includes a point convolution kernel and a depth convolution kernel.

[0015] Further, the hierarchical matching mechanism is used to solve the problem that the target cannot be matched successfully with the historical trajectory of the target after being occluded for a long time, and the specific steps are as follows:

[0016] The motion state of the target is defined as an 8-dimensional vector (u, v, g, h, u', v', g', h') of normal distribution, which is the center point coordinates, width-height ratio, height of the target detection frame and their derivatives with respect to time in the respective coordinate system, then the Kalman filtering algorithm is used to predict the existing trajectory to generate a predicted frame of the current frame, at this time the Mahalanobis distance is used to measure the similarity of the motion state of the predicted frame and the detection frame;

[0017] All the target frames of the last N frames of the trajectory are extracted using a multi-scale re-identification network, and multi-scale features are extracted from all the detection frames of the current frame, and the minimum cosine distance between each detection frame and the multi-scale features extracted from the target frames of the last N frames of each trajectory is calculated, which is used to measure the appearance information similarity between the detection frame and the trajectory;

[0018] The motion state similarity and appearance information similarity obtained by the above steps are linearly weighted to obtain a cost matrix of the detection frame of the current frame and the historical trajectory;

[0019] Meanwhile, thresholds are designed for the Mahalanobis distance and the minimum cosine distance, and pairs below the Mahalanobis distance threshold or above the minimum cosine distance threshold are set to infinity in the cost matrix;

[0020] According to the update state of the predicted frame, that is, the frame number difference between the predicted frame and the last successful matching detection frame is sorted from small to large, the smaller the frame number difference, the more preferentially the matching is performed according to the cost matrix through the Hungarian algorithm, through the matching result, the matched set, the unmatched trajectory set and the unmatched detection frame set are divided, the matched set is added to the historical trajectory, the unmatched set and the trajectory in the unconfirmed state are matched again through the intersection-over-union method, the detection frame that is still unmatched is created as a new trajectory in the unconfirmed state, and the trajectory that is still unmatched is confirmed as a new trajectory for three times, then it is converted to the confirmed state, and the trajectory that is still unmatched is deleted if it is in the unconfirmed state or in the confirmed state but the frame number difference from the last successful matching exceeds the maximum limit.

[0021] Further, the Gaussian smoothing interpolation model is a lightweight interpolation algorithm Gaussian smoothing interpolation, which uses Gaussian process regression to simulate nonlinear motion.

[0022] Further, the N is 100 frames.

[0023] Further, the appearance-free linking model specifically uses two trajectories as input, one of which is composed of the position and frame number of the last 30 frames, and if the frame number of the current trajectory is less than 30, 0 is used for padding.

[0024] Further, the appearance-free linking model is specifically: a time module extracts features along the time dimension using a 7x1 convolution kernel, then a fusion module uses a 1x3 convolution kernel to integrate all dimension information, and the generated two feature maps are respectively pooled and compressed into feature vectors and then connected, and finally a multi-layer perception model is used to predict the confidence score of the correlation.

[0025] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0026] (1) The detection network of the present application is based on YOLOv5, and the attention mechanism is integrated into the backbone network, and a four-fold downsampling tiny target prediction head is added to the head network, so as to improve the positioning of important features and the ability of tiny target detection.

[0027] (2) For the problem that similar target features are difficult to distinguish, the bottleneck layer of the re-identification network adopts a multi-stream structure, and the attention mechanism is used to aggregate the multi-stream features, so as to realize dynamic fusion of multi-scale features and improve the expression ability of target appearance features. In order to reduce the situation that the target confidence is reduced due to occlusion, thereby causing missed detection, a hierarchical matching mechanism is designed in data association. Finally, two plug-and-play lightweight models are applied for trajectory global linking and smoothing completion. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a detection network structure diagram of an embodiment of the present application;

[0029] Figure 2 is a bottleneck module structure diagram of a multi-scale re-identification network of an embodiment of the present application;

[0030] Figure 3 is a hierarchical matching mechanism flow chart of an embodiment of the present application;

[0031] Figure 4 is a principle diagram of an appearance-free linking model of an embodiment of the present application;

[0032] Figure 5 is an effect diagram of Gaussian smoothing interpolation of an embodiment of the present application;

[0033] Figure 6 is a work flow chart of the present application. DETAILED DESCRIPTION

[0034] The present application will be further described in detail below in conjunction with embodiments, but the embodiments of the present application are not limited thereto.

[0035] EMBODIMENT

[0036] As Figure 6As shown, a multi-target tracking method for intelligent driving is mainly aimed at multi-target tracking of pedestrians and vehicles in the process of intelligent driving, and adopts a detection-based tracking framework. Target tracking is to associate and match the detection boxes of the same target in each frame, and the historical trajectory is the motion trajectory before the present. Specifically, it includes:

[0037] The detection network detects and identifies pedestrians and vehicles in the input video sequence, generates a bounding box, includes image coordinates of the center point of the bounding box, width and aspect ratio of the bounding box, and generates a foreground confidence of the bounding box. The bounding box is a detection box.

[0038] As shown in Figure 1 The detection network includes an input end, a backbone network, a neck network, and a head network.

[0039] The backbone network (main network) introduces the C3 module in YOLOv5 into the CA coordinate attention mechanism to form a C3CA module. The CA attention mechanism module aims to enhance the expression ability of the mobile network to learn features. It can transform and change any intermediate feature tensor in the network and output a tensor of the same size.

[0040] The head network of the prior art provides three scale prediction heads to better adapt to different sizes of target objects, and their down-sampling rates are 8 times, 16 times, and 32 times, respectively: the large-scale prediction head has a large resolution and is used to predict small-size targets; the small-scale prediction head has a large receptive field and is used to predict large-size targets. However, in the intelligent driving scene, pedestrians and vehicles far from the camera often occupy very small pixels, and a smaller receptive field is needed to reduce the missed detection situation. Therefore, a 4 times down-sampling branch is added in the detection network: after the original picture is extracted by the backbone network, the features are fused in the neck network using a merging method, and finally a 160x160x128 dimensional feature map is obtained after being processed by the head network H2. The resolution of the feature map output by the H2 prediction head is high, has more accurate position information, and is more accurate for small target detection, thereby improving the ability of positioning important features and micro-target detection.

[0041] As shown in Figure 2 The multi-scale re-identification network is used for multi-scale feature extraction of the bounding boxes generated by the detection network and the bounding boxes of the historical trajectory, and generates an appearance feature vector. The multi-scale re-identification network includes a bottleneck network composed of multiple convolution feature streams with different receptive fields.

[0042] Since the traditional re-identification network for target tracking mostly borrows the architecture designed for general target classification problem, it is difficult to distinguish the feature differences between similar targets. The bottleneck module of the multi-scale re-identification network adopts a multi-flow design, which can be used to extract the multi-scale features of pedestrians, enhance the distinguishing ability of similar features of pedestrians and vehicles in the tracking process, and reduce the number of trajectory identity jumps.

[0043] The bottleneck module of the multi-scale re-identification network is composed of multiple convolution feature flows with different receptive fields. The feature scale concerned by each flow is determined by an index, a CBAM attention mechanism is used as a unified aggregation gate of each feature flow to generate channel weights for dynamic fusion, and finally the original input and the output after fusion of all flows are connected in residual to obtain the final output.

[0044] Further, the multi-scale re-identification network further comprises a lightweight component composed of a lightweight convolution, batch normalization and an activation function.

[0045] The lightweight convolution is a LiteCONV module, which is implemented by a depth separable convolution including a point convolution kernel and a depth convolution kernel. A standard convolution kernel 4D tensor (k, k, c1, c2) is decomposed into a point convolution kernel 4D tensor (1, 1, c1, c2) and a depth convolution kernel 4D tensor (k, k, 1, c2). If the width and height of the output feature map are w and h respectively, the calculation amount is reduced from h x w x k 2 x c1 x c2 to h x w x c2 x (k 2 +c1) by depth separable convolution, and the parameter amount is reduced from k 2 x c1 x c2 to (k 2 +c1) x c2. Therefore, replacing the standard convolution block with the lightweight convolution block composed of depth separable convolution blocks can greatly reduce the calculation amount and parameter amount of the network.

[0046] Hierarchical matching mechanism: the hierarchical matching mechanism of the present application is used to solve the problem that after a target is occluded for a long time, the reappearance of the target cannot be successfully matched with the historical trajectory of the target.

[0047] That is, when two targets occlude each other, the historical trajectory of the occluded target cannot be matched with the target in the current frame, and the target disappears temporarily. When the occluded target reappears, the ID of the target should be kept as much as possible to reduce the number of ID switching. The specific matching process is shown in the accompanying Figure 3

[0048] ​(1) Define the motion state of the target as a normal distribution 8-dimensional vector (u, v, γ, h, u', v', γ', h'), which is the center point coordinates, width-height ratio, height of the target frame, and their derivatives with respect to time in their respective coordinate systems, respectively. Then use the Kalman filter algorithm to predict the existing trajectory and generate the predicted frame of the current frame. At this time, the Mahalanobis distance is used to measure the similarity of the motion state of the predicted frame and the detection frame.

[0049] (2) Use a multi-scale re-identification network to extract multi-scale features from the target frame of the last 100 frames of all trajectories and all detection frames of the current frame. Calculate the minimum cosine distance between the multi-scale features extracted from each detection frame and the last 100 frames of each trajectory, which is used to measure the appearance information similarity between the detection frame and the trajectory.

[0050] (3) Linearly weight the motion state similarity and appearance information similarity obtained in the above steps to obtain the cost matrix of the current frame detection frame and the historical trajectory. At the same time, in order to reduce the number of matches, pairs with large differences in motion state or appearance information similarity need to be filtered out, so thresholds are designed for Mahalanobis distance and minimum cosine distance. Pairs with a Mahalanobis distance below the threshold or a minimum cosine distance above the threshold are set to infinity in the cost matrix to facilitate the calculation of the Hungarian algorithm and improve efficiency.

[0051] (4) According to the update state of the predicted frame, that is, the frame number difference between the predicted frame and the last successful detection frame matching, sort from small to large, the smaller the frame number difference, the more priority to match according to the cost matrix through the Hungarian algorithm. Through the matching result, divide the matched set, the unmatched trajectory set and the unmatched detection frame set. The matched set will be added to the historical trajectory, the unmatched set and the unconfirmed trajectory will be matched again through the intersection-over-union method, and the still unmatched detection frame will create a new trajectory in the unconfirmed state. If it is confirmed as a new trajectory for three consecutive times, it will be converted to a confirmed state, and if it is still unmatched, it will be deleted if it is in the unconfirmed state or the confirmed state but the frame number difference from the last successful match exceeds the maximum limit.

[0052] In step (4), the unmatched trajectory and detection frame will be matched multiple times to compensate for the impact of short-term matching failure caused by target occlusion and other factors. However, if the target is occluded for a long time and the frame number difference from the last successful match exceeds the maximum limit, the trajectory will be deleted. If it appears again, since there is no corresponding historical trajectory, it will be considered as a new target and a new ID will be assigned, causing identity jump. Therefore, based on this, the present application proposes a hierarchical matching mechanism to further improve the problem of identity jump.

[0053] The tracking algorithm based on detection depends on the performance of the detector, and when the target is blocked, the confidence of the detector is low, and most tracking algorithms will delete the low-confidence detection box to exclude the background mistaken for the target. However, when the target is blocked or motion blurred, the confidence of the target is also low, and directly deleting the low-confidence detection box will also cause the target to be lost and the track to be interrupted during tracking.

[0054] Further, other frames can also be selected in the hierarchical matching mechanism, and the number of frames is not limited to 100, and the more the number of frames, the better the effect, but the speed will decrease accordingly, and 100 frames are selected in the method.

[0055] In the application, the detection box is divided into high-score detection box and low-score detection box according to the confidence. Firstly, the high-score detection box still performs the cascade matching in the above method, and after the intersection-over-union matching is completed, the track that is not matched successfully and the low-score detection box are matched again, and the low-score detection box that is still not matched is deleted as background, and the track that is still not matched successfully and the track matched successfully this time are processed in the same way as the first matching.

[0056] In order to pursue high-accuracy association, the application performs global association on the track. Since it depends on appearance features, a large amount of calculation resources are needed, and it is easily affected by noise such as motion blur, so the application uses a non-appearance linking model. Figure 4 Its framework is shown. It takes two tracks Ti and Tj as input, where T* is composed of the position of the last 30 frames and the frame number, and if the frame number of the current track is less than 30, 0 is used for padding. The time module in the figure uses a 7x1 convolution kernel to extract features along the time dimension, and then the fusion module uses a 1x3 convolution kernel to integrate all the dimensional information, and then the generated two feature maps are respectively pooled and compressed into feature vectors and connected, and finally a multi-layer perception model is used to predict the confidence score of the association. At this time, the global connection of the track is converted into a linear assignment problem for solving.

[0057] For the blank track caused by detection loss, the commonly used method is linear interpolation. However, due to the missing motion information, its accuracy is limited. Although there are related strategies to improve the accuracy, such as introducing single-target tracking, Kalman filtering and other algorithms, it also brings additional time-consuming. The application is a lightweight interpolation algorithm Gaussian smoothing interpolation, which uses Gaussian process regression to simulate nonlinear motion.

[0058] Firstly, the Gaussian smoothing interpolation model for the i-th track is as follows:

[0059] p t =f (i) (t)+ε

[0060] Where t F is the frame number, pt ∈P is the bounding box information of the trajectory at frame number t, and ε is Gaussian noise.

[0061] Given the tracked and linearly interpolated trajectory, and assuming it obeys a Gaussian process f(i) ∈ GP(0, k(·, ·)), where is the radial basis function kernel. Based on the properties of Gaussian processes, let the new set of trajectory frame numbers be F*, and its smoothed position P* can be predicted by:

[0062] P * = K(F * ,F)(K(F,F)+σ 2 I) -1 P

[0063] where K(·, ·) is the covariance function based on k(·, ·), and in addition, the hyperparameter λ is used to control the smoothness of the trajectory, which is related to the length of the trajectory l. Generally, λ is selected by the following formula, where τ takes 10.

[0064] λ = τ * log(τ 3 / l)

[0065] The Figure 5 The difference between Gaussian smoothing interpolation and linear interpolation is illustrated. The original tracking results usually include noise vibration and trajectory blank, and linear interpolation directly fills the blank with a straight line without considering the motion information, while Gaussian smoothing interpolation smooths the entire trajectory with adaptive smoothness and solves both problems.

[0066] In order to ensure the tracking effect, the application designs a tracker with a separated structure. In order to solve the problem of difficult target detection, the detection module is based on the YOLOv5 network model, the coordinate attention mechanism is integrated into the C3 module in the backbone network, and a micro target prediction head is added in the head network; in order to more effectively distinguish similar features between the same type of targets, a re-identification network capable of obtaining multi-scale features is designed; in order to reduce the identity jump of the trajectory caused by mutual occlusion, a hierarchical matching mechanism is established by using Kalman filtering and the Hungarian algorithm, supplemented by an appearance-free linking model and a Gaussian smoothing interpolation model for trajectory optimization, so that the tracking trajectory is closer to the real trajectory.

[0067] The above embodiment is the preferred embodiment of the present application, but the embodiments of the present application are not limited by the above-mentioned embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement modes, and all shall be included in the protection scope of the present application.

Claims

1. A method for multi-target tracking for intelligent driving, characterized in that, The application relates to a detection network for identifying people and vehicles in an input video sequence and generating a bounding box and a foreground confidence of the bounding box, wherein the detection network comprises an input end, a backbone network, a neck network and a head network, and the backbone network is integrated with a coordinate attention mechanism. The application relates to a multi-scale re-identification network for performing multi-scale feature extraction on the bounding box generated by the detection network and the bounding box of a historical track to generate an appearance feature vector, wherein the multi-scale re-identification network comprises a bottleneck network composed of multiple convolution feature flows with different receptive fields. The application relates to a hierarchical matching mechanism, which comprises the following steps: firstly, dividing the detection box generated by the detection network into high-score detection boxes and low-score detection boxes according to the foreground confidence, performing Kalman filtering algorithm prediction on the historical track to generate a prediction box, and calculating the Mahalanobis distance between the high-score detection box and the prediction box to obtain a motion feature correlation measure; secondly, extracting the feature vector of the high-score detection box and the feature vector of the bounding box of the last N frames of the historical track by the multi-scale re-identification network, and calculating the minimum cosine distance between the N frames of the bounding box and the high-score detection box to obtain an appearance feature correlation measure; thirdly, linearly weighting the motion feature correlation measure and the appearance feature correlation measure to obtain a cost matrix, performing cascade matching by using the Hungarian algorithm, and then performing intersection over union matching on the historical track and the high-score detection box which fail to be matched; and finally, performing intersection over union matching again on the low-score detection box and the track which fails in the intersection over union matching, updating the historical track which is successfully matched in any one of the matchings by using the Kalman filtering algorithm, and deleting the historical track which fails in all the matchings. The application relates to a track optimization network, which comprises a non-appearance linking model and a Gaussian smoothing interpolation model, the non-appearance linking model globally correlates all the tracks, and the Gaussian smoothing interpolation model smoothes and completes the track blank, so that the obtained track is closer to a real track. The downsampling rates of the head network are 4 times, 8 times, 16 times and 32 times, respectively.

2. The multi-target tracking method of claim 1, wherein, The feature scales concerned by each flow in the bottleneck network are determined by an index, a CBAM attention mechanism is used as a unified aggregation gate of each feature flow to generate channel weights for dynamic fusion, and finally, residual connection is performed on the original input and the output after fusion of all the flows to obtain a final output.

3. The multi-target tracking method of claim 1, wherein, The multi-scale re-identification network further comprises a lightweight component composed of a lightweight convolution, batch normalization and an activation function.

4. The multi-target tracking method of claim 1, wherein, The lightweight convolution is a LiteCONV module, which is implemented by using a depth separable convolution, and the depth separable convolution comprises a point convolution kernel and a depth convolution kernel.

5. The multi-target tracking method of claim 4, wherein, The hierarchical matching mechanism is used to solve the problem that a target cannot be matched with a historical track of the target after the target is hidden for a long time and then appears again, and the specific steps are as follows: the motion state of the target is defined as an 8-dimensional normal distribution vector (u, v, gamma, h, u', v', gamma', h'), which respectively represents the center point coordinates, the height-width ratio, the height of the target detection box and the derivatives of the center point coordinates, the height-width ratio and the height with respect to time, then a Kalman filtering algorithm is used to predict the existing track to generate a prediction box of the current frame, and the Mahalanobis distance is used to measure the motion state similarity between the prediction box and the detection box.

6. The multi-target tracking method according to any one of claims 1-5, characterized in that, ​ ​ The multi-scale re-identification network is used to extract multi-scale features of the target frame of the last N frames of all trajectories and all detection frames of the current frame, and the minimum cosine distance between the multi-scale features of each detection frame and the target frame of each trajectory is calculated to measure the appearance information similarity between the detection frame and the trajectory; The motion state similarity and the appearance information similarity obtained in the above step are linearly weighted to obtain a cost matrix of the detection frame of the current frame and the historical trajectory; Meanwhile, thresholds are designed for the Mahalanobis distance and the minimum cosine distance, and the pairs below the Mahalanobis distance threshold or above the minimum cosine distance threshold are set to infinity in the cost matrix. According to the update state of the prediction frame, that is, the frame number difference between the prediction frame and the last successful matching detection frame is sorted from small to large, the smaller the frame number difference is, the more preferentially the prediction frame is matched according to the cost matrix through the Hungarian algorithm, and through the matching result, a matched set, an unmatched trajectory set and an unmatched detection frame set are divided, the matched set is added to the historical trajectory, the unmatched set and the trajectory in the unconfirmed state are matched again through the intersection-over-union method, the detection frame that is still unmatched is created into a new trajectory in the unconfirmed state, the trajectory that is still unmatched is confirmed as a new trajectory after being matched for three times, and then the trajectory is converted into the confirmed state, and the trajectory that is still unmatched is deleted if the trajectory is in the unconfirmed state or the frame number difference between the trajectory and the last successful matching frame exceeds the maximum limit.

7. The multi-target tracking method of claim 1, wherein, The Gaussian smoothing interpolation model is a lightweight interpolation algorithm, and Gaussian process regression is used to simulate nonlinear motion.

8. The multi-target tracking method of claim 1, wherein, The N is 100 frames.

9. The multi-target tracking method of claim 1, wherein, The appearance-free linking model is specifically two trajectories as input, one of which is composed of the position and frame number of the last 30 frames, and if the frame number of the current trajectory is less than 30, 0 is used for padding.

10. The multi-target tracking method of claim 9, wherein, The appearance-free linking model is specifically that a time module uses a 7x1 convolution kernel to extract features along the time dimension, then a fusion module uses a 1x3 convolution kernel to integrate all dimension information, and the generated two feature maps are connected after being pooled and compressed into feature vectors, and finally a multilayer perception model is used to predict the confidence score of the correlation. The Gaussian smoothing interpolation model is a lightweight interpolation algorithm, and Gaussian process regression is used to simulate nonlinear motion. The N is 100 frames. The appearance-free linking model is specifically two trajectories as input, one of which is composed of the position and frame number of the last 30 frames, and if the frame number of the current trajectory is less than 30, 0 is used for padding. The appearance-free linking model is specifically that a time module uses a 7x1 convolution kernel to extract features along the time dimension, then a fusion module uses a 1x3 convolution kernel to integrate all dimension information, and the generated two feature maps are connected after being pooled and compressed into feature vectors, and finally a multilayer perception model is used to predict the confidence score of the correlation.

Images