Improved vehicle and pedestrian tracking method based on Deep-sort

Abstract

The invention discloses a vehicle and pedestrian tracking algorithm recommendation method based on Deep-sort improvement. Relates to the field of recommendation methods, an improved Deep-Sort serves as a tracker, due to the fact that a tracking object of a Deep-Sort algorithm is a person, a research object of the invention is a vehicle and an appearance model is not applicable, the improved Deep-Sort improves a feature extraction network of the Deep-Sort in three aspects: the feature extraction network is rebuilt based on ResNet-34, so that the feature extraction network is more suitable for extraction of vehicle features; a center loss function and a triple loss function are introduced and are combined with a Softmax loss function for use, so that intra-class and inter-class distances of the network are better optimized; and a cosine annealing learning rate attenuation method is used, so that the final convergence effect of the network training model is better. The improved feature extraction network is trained, and experimental results show that the vehicle appearance model obtained through training has good classification capability, and the recognition capability of vehicle features is improved compared with an original model. In a multi-moving vehicle tracking experiment, the D-MMVT algorithm is tested by using an OTB-100 data set and a daily monitoring road video, the accuracy rate MOTA, the precision rate MOTP and the ID switching frequency are used as evaluation standards, a comparison experiment is performed with other tracking algorithms, and an experiment result shows that the D-MMVT algorithm has a better tracking effect on a severely shielded target and a small target; and the tracking accuracy and precision are improved.

Description

Technical Field

[0001] The present invention relates to the field of recommendation methods, and in particular to a vehicle and pedestrian tracking method based on Deep-sort improvement. Background Art

[0002] In computer vision, target detection refers to finding targets of interest in images and videos, or simply classifying and locating targets in a scene. Since targets vary in shape, appearance, and posture, and are affected by interference factors such as lighting, shadows, and occlusions during the imaging process, target detection has become one of the most challenging research directions in computer vision. Target tracking refers to establishing the position relationship of the target to be tracked in a continuous video, obtaining the target's complete motion trajectory, and predicting the target's exact position in the next frame by obtaining the target's position information in the first frame of the video. During the motion process, the target may have changes in shape, posture, scale, lighting, or occlusion, and the research on moving target tracking algorithms also focuses on solving these changes and specific applications. Target detection and tracking are widely used in multimedia imaging, video surveillance, and other aspects.

[0003] Target detection and tracking are generally divided into two steps: the first step is target detection, that is, identifying all targets in the initial frame of the video; the second step is target tracking, that is, tracking the identified targets in the subsequent video frames. Technically, there are single-target tracking and multi-target tracking. The latter technology is relatively complex and more difficult. Multi-vehicle detection and tracking, as one of the research branches of multi-moving target detection and tracking, has a wide range of applications in intelligent transportation systems. Intelligent transportation systems refer to the use of advanced artificial intelligence and other technologies to organically combine people, vehicles, and roads to form an orderly intelligent system, which can make full use of transportation infrastructure, reduce traffic load, improve traffic efficiency, and ensure traffic safety. Multi-vehicle detection and tracking technology helps intelligent vehicles perceive the surrounding environment and actively analyze the possibility of vehicle collisions to reduce accidents and ensure the safety of drivers, passengers, and pedestrians. However, there are complex changes in moving vehicles, such as vehicle scale, posture changes, and partial or full occlusion of vehicles in complex road scenes such as intersections. Therefore, there are still many difficulties in multi-vehicle detection and tracking technology. How to detect and track moving vehicles in real time and reliably in complex road scenes has important practical significance and is the basis and key to realizing intelligent transportation systems. Based on this, a recommended method for detecting and tracking vehicles and pedestrians in dense places is invented. Summary of the invention

[0004] The purpose of the present invention is to propose a detection and tracking method for vehicles and pedestrians in dense scenes, aiming at improving the accuracy of detection and tracking, in which the effective detection and real-time tracking of vehicles is the prerequisite for analyzing and judging the behavior of vehicles in intelligent transportation systems, which is of great significance to intelligent transportation.

[0005] In order to achieve the above-mentioned object of the invention, the present invention adopts a method for recommending detection and tracking methods of vehicles and pedestrians in dense scenes. Optionally, the method includes the following steps:

[0006] Step 1: First, improve the network for extracting features in Deep-Sort. After the input image passes through the feature extraction network, a feature vector is obtained. Finally, the feature is mapped to the unit sphere using normalization, and a 1×128 vector is output.

[0007] Step 2: Introduce the triplet loss function and the center loss function at the same time, and use the Softmax loss, center loss and triplet loss function for joint training;

[0008] Step 3: Use the cosine annealing learning rate decay method to first preheat the model, gradually increase the learning rate to the set initial learning rate at a very small learning rate, and then decrease it according to the cosine curve;

[0009] Step 4: Retrain the vehicle appearance model through the improvements of steps 1, 2 and 3. Input the current frame of the vehicle video into the detector to obtain the detection frame. Use the detection result as the input of the tracker. Use Kalman filtering to predict the state of the tracking frame in the next frame. Based on the appearance features and motion information of the vehicle, match the detection frame with the tracking frame through the Hungarian algorithm, and then use Kalman filtering to update the tracking frame of the current frame.

[0010] Optionally, the improved deep feature extraction network obtained in step 1:

[0011] The solid line connection part indicates that F(x) and x channels are the same and can be added directly, so the calculation method used is:

[0012] H(x)=F(x)+x (1)

[0013] The dotted line connection part indicates that F(x) is different from the x channel, and the calculation method used is:

[0014] H(x)=F(x)+wx (2)

[0015] Where W represents the convolution operation, which is used to adjust the dimension of the x channel.

[0016] Optionally, step 2 introduces a triplet loss function and a center loss function, and the joint training using the Softmax loss, center loss and triplet loss function includes the following steps:

[0017] Step 1: Calculate the feature y learned by the i-th sample in the deep feature extraction network structure i , the formula for the corresponding Softmax loss function is as follows:

[0018]

[0019] Among them, N represents the number of samples, M represents the number of vehicle IDs in the data set, k represents the kth category, and Wk corresponds to the weight of the category. represents the output of the fully connected layer, where W zi represents the weight of the i-th sample, Z i represents the actual label of the i-th sample, y i Represents the characteristics of the i-th sample.

[0020] Step 2: The center loss function is mainly used to reduce intra-class differences and can directly constrain the distance between sample features. In other words, the distance between the feature and the average feature of the same category should be small enough, which requires that the features of the same category should be close to their center points. The calculation formula is:

[0021]

[0022] where y i represents the extracted features of the i-th sample, C yi It represents the average feature of all sample features in the category corresponding to sample i, that is, the center point of the sample features of the same category, and N represents the number of samples. This loss function is suitable for classification tasks with high inter-class similarity. The purpose is to make the distance between the feature of each sample in a batch and the feature center as small as possible, that is, the intra-class distance should be as small as possible.

[0023] The loss function adjusts the coefficients to reduce the distance between the positive sample image and the target image, and increases the distance between the negative sample image and the target image. Finally, images with the same ID are clustered in the feature space to complete the re-identification task. The formula is as follows:

[0024]

[0025] Among them, (a, p, n) corresponds to the target image, positive sample image and negative sample image of the triplet respectively, and m is a predefined distance constant. Finally, the loss function expression of the improved Deep-Sort algorithm in this paper is as follows:

[0026] L Total =L s +L c +L T (7)

[0027] Optionally, in step 3, Deep-Sort uses a fixed-step learning rate decay method when training the feature extraction network. This method has the problem of slow model convergence and needs to modify the learning rate method. The cosine annealing learning rate decay method is used to first preheat the model, gradually increase the learning rate to the set initial learning rate at a very small learning rate, and then decrease it according to the cosine curve. The formula is as follows:

[0028]

[0029] Among them, n i min and n i max Respectively represent the minimum and maximum values ​​of the learning rate, T cur Indicates how many epochs are currently executed, T i Indicates the total number of epochs to be executed before the i-th warm-up model.

[0030] Optionally, in step 4, the state prediction needs to predict the position information of the target vehicle and the uncertainty of the position information, using the mean to represent the position information of the target and using the covariance to represent the uncertainty of the target position information;

[0031] The Kalman filter algorithm is a common optimal prediction and estimation algorithm. The principle of this algorithm is to estimate the state of the next measurement through dynamic measurements containing noise interference, and then fuse the predicted value with the measured value. The Kalman filter is used to predict the tracking frame at the current time t based on the tracking frame at time t-1. The formula is as follows:

[0032] X t =FX t-1 (9)

[0033] P t =FP t-1 F T +Q (10)

[0034] Among them, Xt represents the predicted tracking box at time t, Pt represents the covariance matrix at time t, Q represents the noise matrix of the current filter, F is the state transfer matrix, and dt is the difference between the frame at time t and the frame at time t-1:

[0035] Optionally, step 4 uses the Hungarian algorithm for matching, which gives the same priority to each object and solves the Hungarian algorithm on this basis, which means that the motion model and the appearance model must be accurately predicted, or a higher threshold must be set.

[0036] The Hungarian algorithm is an algorithm that seeks maximum matching. Through this algorithm, the detection frame and the tracking frame can be matched. If they can be matched correctly, it means that the tracking is stable and the target vehicle is not lost; if the match fails, the target vehicle corresponding to the tracking frame is considered lost in the current frame, and the target vehicle corresponding to the detection frame is considered newly added in the current frame. At this time, the result obtained by the Hungarian algorithm can be passed back to the Kalman filter algorithm to update the tracking target and tracking frame. The specific calculation steps of this algorithm are:

[0037] Step 1: Create a target matrix G based on the results of the fusion metric formula and the requirements of the target allocation problem A×B ;

[0038] Step 2: Find G A×B The minimum value of each row in the matrix, then G A×B Subtract the minimum value of each row from each element in each row, and the resulting matrix is ​​recorded as G1;

[0039] Step 3: Similar to step 2 in claim 5, find the minimum value of each column in the G1 matrix, then subtract the minimum value of each column from each element in each column of G1, and the resulting matrix is ​​recorded as G2;

[0040] Step 4: Find the minimum number of rows and columns required to cover all zeros in the current matrix, and determine whether the minimum number of rows and columns found is equal to the dimension A of the matrix. If they are equal, the Hungarian algorithm ends, and the matrix obtained at this time is the optimal solution, and the match is successful; if it is less than A, continue with the following steps.

[0041] Step 5: Find the minimum value among all the elements not covered in step 4 of claim 5, subtract the minimum value from each element not covered, and add the minimum value to the elements covered twice, to obtain a new matrix G3;

[0042] Step 6: Continue to loop through steps 4 and 5 until the minimum number of rows and columns required to cover the matrix is ​​equal to A. The resulting matrix is ​​the optimal solution.

[0043] The improved Deepsort algorithm first uses the appearance and motion models to calculate the similarity matrix to obtain the cost matrix and the threshold matrix. The threshold matrix is ​​used to limit the excessive values ​​in the cost matrix. The cost matrix is ​​then used as the input of the Hungarian algorithm for matching. If the match is successful, the output is the detection result. If it fails, the unmatched tracking frame and the unmatched detection frame are matched using IoU matching, and then the Hungarian algorithm is used for matching again. The matching process is an iterative process, starting from the 0th frame and ending when the frame number threshold is reached. The tracking frame and the detection frame of each frame are matched, and the matching result is returned. The optimal match is performed from short to long based on the tracking frame loss time. Through cascade matching processing, the occluded target vehicle can be re-tracked, and the ID switching during tracking is reduced. Finally, matching pairs, unmatched tracking frames, and unmatched detection frames can be obtained.

[0044] Optionally, in step 4, the tracking frame of the current frame is updated using Kalman filtering. The update is based on the detection frame of the current frame at time t, and the state of the tracking frame that successfully matches it is corrected to obtain a more accurate tracking result. t is the mean vector of the detection box, set Z t =[cx, cy, r, h], where H is the measurement matrix, which can transform the mean vector X of the tracking box t-1 Mapped to the detection space, the mean error between the detection box and the tracking box can be expressed as:

[0045] η=Z t -HX t-1 (10)

[0046] Take R as the noise matrix of the detector, set it to a 4×4 diagonal matrix, and set the values ​​on the diagonal to the two coordinates of the center point and the noise of width and height. The noise of width and height should be greater than the noise of the center point. t-1 Mapped to the detection space, plus the noise matrix R, the Kalman filter preprocessed data D is obtained, which can be shown by the following formula:

[0047] D=HP t-1 H T +R (11)

[0048] The Kalman gain K is used to estimate the importance of the error. The calculation formula is:

[0049] K=P t-1 H T D -1 (12)

[0050] Finally, the mean vector Xt and covariance matrix Pt of the tracking frame are updated according to the obtained K. The calculation formula is:

[0051] X t =Xt-1 +Kη (13)

[0052] P t =(I-KH)P t (14)

[0053] The beneficial effects of the present invention are as follows: This chapter designs a vehicle and pedestrian tracking algorithm based on the improved Deep-sort, using the trained model as the detector and the improved Deep-Sort as the tracker. Since the tracking object of the Deep-Sort algorithm is a person, and the research object of this article is a vehicle, the appearance model is not applicable. The improved Deep-Sort has made three improvements to the feature extraction network of Deep-Sort: the feature extraction network is rebuilt based on ResNet-34 to make it more suitable for the extraction of vehicle features; the center loss function and the triplet loss function are introduced and used in conjunction with the Softmax loss function to better optimize the intra-class and inter-class distances of the network; the cosine annealing learning rate decay method is used to make the final convergence effect of the network training model better. The improved feature extraction network is trained, and the experimental results show that the vehicle appearance model obtained by training has good classification ability and improves the recognition ability of vehicle features compared with the original model. In the multi-motion vehicle tracking experiment, the OTB-100 dataset and daily surveillance road videos were used to test the D-MMVT algorithm. The accuracy MOTA, precision MOTP and the number of ID switching were used as evaluation criteria, and compared with other tracking algorithms. The experimental results show that the improved algorithm has better tracking effects on severely occluded targets and small targets, and improves the accuracy and precision of tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of vehicle and pedestrian tracking based on Deep-sort improvement described in the present invention. DETAILED DESCRIPTION

[0055] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0056] Embodiment 1

[0057] Given a new dataset, we will perform the following operations on it according to the recommended method for improving vehicle and pedestrian tracking algorithms based on Deep-sort, including:

[0058] Step 1: Calculate the feature y learned by the i-th sample in the deep feature extraction network structure i , the formula for the corresponding Softmax loss function is as follows:

[0059]

[0060] Among them, N represents the number of samples, M represents the number of vehicle IDs in the data set, k represents the kth category, and Wk corresponds to the weight of the category. represents the output of the fully connected layer, where W zi represents the weight of the i-th sample, Z i represents the actual label of the i-th sample, y i Represents the characteristics of the i-th sample.

[0061] Step 2: The center loss function is mainly used to reduce intra-class differences and can directly constrain the distance between sample features. In other words, the distance between the feature and the average feature of the same category should be small enough, which requires that the features of the same category should be close to their center points. The calculation formula is:

[0062]

[0063] where y i represents the extracted features of the i-th sample, C yi It represents the average feature of all sample features in the category corresponding to sample i, that is, the center point of the sample features of the same category, and N represents the number of samples. This loss function is suitable for classification tasks with high inter-class similarity. The purpose is to make the distance between the feature of each sample in a batch and the feature center as small as possible, that is, the intra-class distance should be as small as possible.

[0064] The loss function adjusts the coefficients to reduce the distance between the positive sample image and the target image, and increases the distance between the negative sample image and the target image. Finally, images with the same ID are clustered in the feature space to complete the re-identification task. The formula is as follows:

[0065]

[0066] Among them, (a, p, n) corresponds to the target image, positive sample image and negative sample image of the triplet respectively, and m is a predefined distance constant. Finally, the loss function expression of the improved Deep-Sort algorithm in this paper is as follows:

[0067] L Total =L s +L c +L T

[0068] Step 3: Deep-Sort uses a fixed-step learning rate decay method when training the feature extraction network. This method has the problem of slow model convergence and needs to modify the learning rate method. The cosine annealing learning rate decay method is used to first preheat the model, gradually increase the learning rate to the set initial learning rate at a very small learning rate, and then decrease it according to the cosine curve. The formula is as follows:

[0069]

[0070] Among them, n i min and n i max Respectively represent the minimum and maximum values ​​of the learning rate, T cur Indicates how many epochs are currently executed, T i Indicates the total number of epochs to be executed before the i-th warm-up model.

[0071] Step 4: State prediction requires predicting the location information of the target vehicle and the uncertainty of the location information. The mean is used to represent the location information of the target, and the covariance is used to represent the uncertainty of the target location information.

[0072] Use Kalman filtering to predict the tracking frame at the current time t based on the tracking frame at time t-1. The formula is as follows:

[0073] X t =FX t-1

[0074] P t =FP t-1 F T +Q

[0075] Among them, Xt represents the predicted tracking box at time t, Pt represents the covariance matrix at time t, Q represents the noise matrix of the current filter, F is the state transfer matrix, and dt is the difference between the frame at time t and the frame at time t-1:

[0076] Step 4 uses the Hungarian algorithm to match.

[0077] The specific calculation steps of this algorithm are:

[0078] Step 1: Create a target matrix G based on the results of the fusion metric formula and the requirements of the target allocation problem A×B ;

[0079] Step 2: Find G A×B The minimum value of each row in the matrix, then G A×B Subtract the minimum value of each row from each element in each row, and the resulting matrix is ​​recorded as G1;

[0080] Step 3: Similar to step 2, find the minimum value of each column in the G1 matrix, then subtract the minimum value of each column from each element in each column of G1, and the resulting matrix is ​​recorded as G2;

[0081] Step 4: Find the minimum number of rows and columns required to cover all zeros in the current matrix, and determine whether the minimum number of rows and columns found is equal to the dimension A of the matrix. If they are equal, the Hungarian algorithm ends, and the matrix obtained at this time is the optimal solution, and the match is successful; if it is less than A, continue with the following steps.

[0082] Step 5: Find the minimum value among all the elements not covered in step 4, subtract this minimum value from each uncovered element, and add this minimum value to the elements that have been covered twice to obtain the new matrix G3;

[0083] Step 6: Continue to loop through steps 4 and 5 until the minimum number of rows and columns required to cover the matrix is ​​equal to A. The resulting matrix is ​​the optimal solution.

[0084] Step 4: Use Kalman filtering to update the tracking frame of the current frame. The update is based on the detection frame of the current frame at time t, and corrects the state of the tracking frame that successfully matches it to obtain a more accurate tracking result. t is the mean vector of the detection box, set Z t =[cx, cy, r, h], where H is the measurement matrix, which can transform the mean vector X of the tracking box t-1 Mapped to the detection space, the mean error between the detection box and the tracking box can be expressed as:

[0085] η=Z t -HX t-1 (10)

[0086] Take R as the noise matrix of the detector, set it to a 4×4 diagonal matrix, and set the values ​​on the diagonal to the two coordinates of the center point and the noise of width and height. The noise of width and height should be greater than the noise of the center point. t-1 Mapped to the detection space, plus the noise matrix R, the Kalman filter preprocessed data D is obtained, which can be shown by the following formula:

[0087] D=HP t-1 H T +R (11)

[0088] The Kalman gain K is used to estimate the importance of the error. The calculation formula is:

[0089] K=P t-1 H T D -1 (12)

[0090] Finally, the mean vector Xt and covariance matrix Pt of the tracking frame are updated according to the obtained K. The calculation formula is:

[0091] X t =X t-1 +Kη (13)

[0092] P t =(I-KH)P t (14)

[0093] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in the relevant technical field, are also included in the patent protection scope of the present invention.

Claims

1. The vehicle and pedestrian tracking method based on Deep-sort is characterized by: The method comprises the following steps Step A: First, improve the network for extracting features in Deep-Sort. After the input image passes through the feature extraction network, a feature vector is obtained. Finally, the feature is mapped to the unit sphere using normalization, and a 1×128 vector is output. Step B: Introduce the triplet loss function and the center loss function at the same time, and use the Softmax loss, center loss and triplet loss function for joint training; Step C: Use the cosine annealing learning rate decay method to first preheat the model, gradually increase the learning rate to the set initial learning rate at a very small learning rate, and then decrease it according to the cosine curve; Step D: Retrain the vehicle appearance model through the improvements of step A, step B and step C. Input the current frame of the vehicle video into the detector to obtain the detection frame. Use the detection result as the input of the tracker. Use Kalman filtering to predict the state of the tracking frame in the next frame. Step E: Based on the appearance features and motion information of the vehicle, the detection frame and the tracking frame are matched by the Hungarian algorithm, and then the tracking frame of the current frame is updated by the Kalman filter.

2. The vehicle and pedestrian tracking algorithm based on Deep-sort improvement according to claim 1 is characterized in that: The step A obtains the dimension of the improved channel, and the improved channel dimension is obtained by performing the following steps A1 to A3: Step A1: The solid line connection part indicates that F(x) and x channels are the same and can be directly added, so the calculation method used is: H(x)=F(x)+x Step A2: The dotted line connection indicates that F(x) and x channels are different, and the calculation method used is: H(x)=F(x)+wx Step A3: Where W represents a convolution operation to adjust the dimension of the x channel.

3. The vehicle and pedestrian tracking algorithm based on Deep-sort improvement according to claim 1 is characterized in that: Step B introduces the triplet loss function and the center loss function, and uses the Softmax loss, center loss and triplet loss function for joint training to obtain the loss function of the improved Deep-Sort algorithm, including the following steps B1 to B4: Step B1: For the feature yi learned by the i-th sample in the deep feature extraction network structure proposed in step B of claim 1, the formula of the corresponding Softmax loss function is as follows: Among them, N represents the number of samples, M represents the number of vehicle IDs in the data set, k represents the kth category, and Wk corresponds to the weight of the category. represents the output of the fully connected layer, where Wzi represents the weight of the i-th sample, Zi represents the actual label of the i-th sample, and yi represents the feature of the i-th sample. Step B2: The center loss function is mainly used to reduce intra-class differences and can directly constrain the distance between sample features. In other words, the distance between the feature and the average feature of the same category should be small enough, which requires that the features of the same category should be close to their center points. The calculation formula is: where y i represents the extracted features of the i-th sample, C yi It represents the average feature of all sample features in the category corresponding to sample i, that is, the center point of the sample features of the same category, and N represents the number of samples. This loss function is suitable for classification tasks with high inter-class similarity. The purpose is to make the distance between the feature of each sample in a batch and the feature center as small as possible, that is, the intra-class distance should be as small as possible. Step B3: The loss function adjusts the coefficients to reduce the distance between the positive sample image and the target image, and increases the distance between the negative sample image and the target image. Finally, images with the same ID are clustered in the feature space to complete the re-identification task. The formula is as follows: Step B4: (a, p, n) corresponds to the target image, positive sample image and negative sample image of the triplet respectively, and m is a predefined distance constant. Finally, the loss function expression of the improved Deep-Sort algorithm in this paper is as follows: L Total =L s +L c +L T 4. The vehicle and pedestrian tracking algorithm based on Deep-sort improvement according to claim 1 is characterized in that: Step C state prediction needs to predict the position information of the target vehicle and the uncertainty of the position information, using the mean to represent the position information of the target, and using the covariance to represent the uncertainty of the target position information, including the following steps C1 to C2; Step C1: Kalman filter algorithm is a common optimal prediction and estimation algorithm. The principle of this algorithm is to estimate the state of the next measurement through dynamic measurement containing noise interference, and then fuse the predicted value with the measured value. Use Kalman filter to predict the tracking frame at the current time t based on the tracking frame at time t-1. The formula is as follows: X t =FX t-1 Step C2: Xt represents the predicted tracking box at time t, Pt represents the covariance matrix at time t, Q represents the noise matrix of the current filter, F is the state transfer matrix, and dt is the difference between the frame at time t and the frame at time t-1: P t =FP t-1 F T +Q 5. The vehicle and pedestrian tracking algorithm based on Deep-sort improvement according to claim 1 is characterized in that: Step D uses the Hungarian algorithm for matching, which gives each object the same priority and solves the Hungarian algorithm on this basis, which means that the motion model and appearance model must be accurately predicted or a higher threshold must be set. The Hungarian algorithm is an algorithm that seeks maximum matching. Through this algorithm, the detection frame and the tracking frame can be matched. If they can be matched correctly, it means that the tracking is stable and the target vehicle is not lost; if the match fails, the target vehicle corresponding to the tracking frame is considered lost in the current frame, and the target vehicle corresponding to the detection frame is considered newly added in the current frame. At this time, the result obtained by the Hungarian algorithm can be passed back to the Kalman filter algorithm to update the tracking target and tracking frame. The specific calculation steps of this algorithm are D1 to D6: Step D1: Create a target matrix G based on the results of the fusion metric formula and the requirements of the target assignment problem A×B ; Step D2: Find G A×B The minimum value of each row in the matrix, then G A×B Subtract the minimum value of each row from each element in each row, and the resulting matrix is ​​recorded as G1; step D3: Similar to step D2 in claim 5, find the minimum value of each column in the G1 matrix, then subtract the minimum value of each column from each element of each column of G1, and the resulting matrix is ​​recorded as G2; Step D4: Find the minimum number of rows and columns required to cover all zeros in the current matrix, and determine whether the minimum number of rows and columns found is equal to the dimension A of the matrix. If they are equal, the Hungarian algorithm ends, and the matrix obtained at this time is the optimal solution, and the matching is successful; if it is less than A, continue with the following steps; Step D5: Find the minimum value among all the elements not covered in step D4 of claim 5, subtract the minimum value from each element not covered, and add the minimum value to the elements covered twice, to obtain a new matrix G3; Step D6: Continue to loop step D4 and step D5 in claim 5 until the minimum number of rows and columns required in the covering matrix is ​​equal to A, and the obtained matrix is ​​the optimal solution. The improved Deepsort algorithm first uses the appearance and motion models to calculate the similarity matrix to obtain the cost matrix and the threshold matrix. The threshold matrix is ​​used to limit the excessive values ​​in the cost matrix. The cost matrix is ​​then used as the input of the Hungarian algorithm for matching. If the match is successful, the output is the detection result. If it fails, the unmatched tracking frame and the unmatched detection frame are matched using IoU matching, and then the Hungarian algorithm is used for matching again. The matching process is an iterative process, starting from the 0th frame and ending when the frame number threshold is reached. The tracking frame and the detection frame of each frame are matched, and the matching result is returned. The optimal match is performed from short to long based on the tracking frame loss time. Through cascade matching processing, the occluded target vehicle can be re-tracked, and the ID switching during tracking is reduced. Finally, matching pairs, unmatched tracking frames, and unmatched detection frames can be obtained.

6. The vehicle and pedestrian tracking algorithm based on Deep-sort improvement according to claim 1 is characterized in that: Step E uses Kalman filtering to update the tracking frame of the current frame. The update is based on the detection frame at time t of the current frame, and corrects the state of the tracking frame that successfully matches it to obtain a more accurate tracking result. Steps E1 to E4 are: Step E1: Take Z t is the mean vector of the detection box, set Z t =[cx, cy, r, h], where H is the measurement matrix, which can transform the mean vector X of the tracking box t-1 Mapped to the detection space, the mean error between the detection box and the tracking box can be expressed as: η=Z t -HX t-1 Step E2: Take R as the noise matrix of the detector, set it to a 4×4 diagonal matrix, and set the values ​​on the diagonal to the two coordinates of the center point and the noise of width and height. The noise of width and height should be greater than the noise of the center point. t-1 Mapped to the detection space, plus the noise matrix R, the Kalman filter preprocessed data D is obtained, which can be shown by the following formula: D=HP t-1 H T +R Step E3: Use the Kalman gain K to estimate the importance of the error. The calculation formula is: K=P t-1 H T D -1 Step E4: Finally, the mean vector Xt and covariance matrix Pt of the tracking frame are updated according to the obtained K. The calculation formula is: X t =X t-1 +Kη P t =(I-KH)P t

Images