A pedestrian multi-target tracking method, device and computer readable storage medium

Abstract

The application discloses a pedestrian multi-target tracking method and device and a computer readable storage medium. The specific scheme comprises the following steps: obtaining a pedestrian detection result and an appearance feature vector by using a DLA34 network, setting high and low confidence threshold values to classify the detection result, and giving priority to high confidence detection, and after double matching of the appearance feature and the position information is completed for the high confidence detection, double supplementary matching is performed on the low confidence detection result, so that the data association task is better realized. According to the application, the low confidence detection result is included in the range to be matched through double supplementary matching, the track multi-frame interruption problem caused by occlusion is reduced, the completeness of the tracking track is improved, and the tracking accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of pedestrian tracking, and in particular to a pedestrian multi-target tracking method, device and computer readable storage medium. BACKGROUND

[0002] Multi-target tracking technology is a hot research direction in the field of computer vision, aiming to locate multiple targets in the input video, maintain their identity, and give their motion trajectory. The main tracking targets of multi-target tracking can be divided into pedestrians, drones, vehicles, etc., and compared with drone and vehicle targets, pedestrian targets have characteristics such as multiple postures, variable appearance features, and flexible action trajectories, making multi-target tracking for pedestrian targets more challenging. Existing pedestrian multi-target tracking technologies can be divided into two stages: pedestrian detection and data association.

[0003] Most of the traditional multi-target tracking algorithms use manual features of images to complete the pedestrian detection task. Manual feature calculation is simple, but the robustness is low, and it is difficult to handle target deformation and occlusion problems. Convolutional neural networks can obtain deep features of images, which have deeper semantic information and stronger discrimination ability, but the calculation amount in the training process is large. With the continuous improvement of CPU, GPU and integrated circuit computing power, the huge calculation amount demand of using convolutional neural networks to process problems can be met, and the traditional multi-target tracking technology is gradually replaced by deep learning methods, which become the mainstream technology in the field of multi-target tracking.

[0004] The data association stage of the current pedestrian multi-target tracking technology often filters out low-confidence detection results by setting a threshold, and only implements data association on high-confidence detection results. Since there are often occluded pedestrians in low-confidence detection results, this way is easy to miss some targets occluded by other pedestrians or buildings, leading to the problem of multi-frame interruption of the trajectory. SUMMARY

[0005] To solve the above problems, the present application provides a pedestrian multi-target tracking method, device and computer readable storage medium, which classifies the detection results by setting high and low confidence double thresholds, and adopts a targeted matching scheme for the two types of detection results, in order to reduce the multi-frame interruption of the trajectory caused by occlusion, better complete the data association task, and further improve the tracking accuracy.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a pedestrian multi-target tracking method, which comprises the following steps:

[0008] S1, using a pre-trained model, training on a pedestrian multi-target tracking dataset as a target detector;

[0009] S2, obtaining the bounding box information and appearance features of the pedestrian target for each frame image of the video by using a target detector;

[0010] S3, setting a confidence threshold to divide the pedestrian detection results into a high-confidence detection set and a low-confidence detection set;

[0011] S4, preferentially performing data association on the high-confidence detection set and the candidate trajectories based on the principle of first appearance similarity and then position similarity, and obtaining candidate trajectories and detection results that are not successfully matched, updating the appearance features of the candidate trajectories that are successfully associated, initializing the detection results that are not successfully associated, and obtaining new trajectories;

[0012] S5, based on the principle of first position similarity and then appearance similarity, performing data association on the low-confidence detection set and the candidate trajectories that are not successfully matched, and defining a lost target as a candidate trajectory that is not successfully matched for consecutive multiple frames.

[0013] Further, the S1 specifically comprises:

[0014] A pedestrian multi-target tracking data set MOT17 is obtained, which contains 14 video sequences, of which 7 video sequences are used for training the model, and the remaining 7 are used for algorithm testing;

[0015] A pre-trained DLA34 network model is used as a target detector, a loss function is constructed, and the model parameters are trained and fine-tuned using the training set of the pedestrian multi-target tracking data set.

[0016] Further, the S2 specifically comprises:

[0017] The target detector outputs the center point offset, the bounding box size, the image heat map, and the appearance features for each frame image of the video, and the center point coordinates of the pedestrian target are obtained by transforming the regression back through the heat map Gaussian function;

[0018] The appearance features include a 128-dimensional vector, which is used to calculate the appearance feature similarity in the subsequent data association stage.

[0019] Further, the S3 specifically comprises:

[0020] High and low confidence thresholds δ1 and δ2 are set, and the classification based on the confidence is as follows

[0021]

[0022] The detection results with a confidence greater than δ1 are defined as high-confidence detections, and the detection results with a confidence in the interval [δ1, δ2] are defined as low-confidence detections, which are used in the subsequent classification and matching process.

[0023] Further, the S4 specifically comprises:

[0024] The appearance feature similarity between the high-confidence detection result and the candidate trajectory is calculated to obtain a similarity matrix, a matching cost matrix is calculated according to the similarity matrix, and a data association scheme I is obtained by using the Hungarian matching algorithm, and the trajectory successfully matched to the detection result in the data association scheme I is removed from the candidate trajectory;

[0025] The state estimation value of the pedestrian, i.e., the predicted position, is calculated according to the state vector of the trajectory based on the Kalman filter;

[0026] The intersection over union IOU between the high-confidence detection result and the predicted position of the candidate trajectory is calculated to obtain a position similarity matrix, a matching cost matrix is calculated according to the similarity matrix, and a data association scheme II is obtained by using the Hungarian matching algorithm, and the trajectory successfully matched to the detection result in the data association scheme II is removed from the candidate trajectory;

[0027] The high-confidence detection result successfully associated in the data association schemes I and II is added to the corresponding candidate trajectory, and the pedestrian appearance feature thereof is updated according to a certain weight;

[0028] The high-confidence detection result not successfully associated with the candidate trajectory is defined as a new target, and is initialized to obtain a new trajectory.

[0029] Further, the S5 specifically comprises:

[0030] The state estimation value of the pedestrian, i.e., the predicted position, is calculated according to the state vector of the trajectory based on the Kalman filter;

[0031] The intersection over union IOU between the low-confidence detection result and the predicted position of the candidate trajectory is calculated to obtain a position similarity matrix, a matching cost matrix is calculated according to the similarity matrix, and a data association scheme III is obtained by using the Hungarian matching algorithm, and the trajectory successfully matched to the detection result in the data association scheme III is removed from the candidate trajectory;

[0032] The cosine distance between the low-confidence detection result and the pedestrian appearance feature of the candidate trajectory is calculated to obtain an appearance similarity matrix, a matching cost matrix is calculated according to the similarity matrix, and a data association scheme IV is obtained by using the Hungarian matching algorithm;

[0033] The low-confidence detection result successfully associated in the data association schemes III and IV is added to the corresponding candidate trajectory;

[0034] If there is a candidate trajectory not successfully matched for a plurality of continuous frames, the candidate trajectory is defined as a lost target, and is not given the matching right in the subsequent frame tracking process.

[0035] In a second aspect, the present application provides a pedestrian multi-target tracking device, which comprises:

[0036] A first acquisition module is configured to acquire first video stream data to be tracked.

[0037] A first detection module is configured to detect pedestrian target information in the first video stream.

[0038] A first classification module is configured to classify the detection results into high-confidence detection results and low-confidence detection results based on confidence.

[0039] A first data association module is configured to associate the high-confidence detection results with candidate tracks one by one.

[0040] A second data association module is configured to associate the low-confidence detection results with candidate tracks one by one.

[0041] A first determination module is configured to determine new targets, lost targets, and candidate tracks for subsequent tracking.

[0042] The new target is a high-confidence detection result of failed data association, which needs to be initialized to obtain a new track.

[0043] The lost target is a candidate track of failed association with high-confidence and low-confidence detection results in consecutive multiple frames, and the candidate track is defined as a lost target, i.e., no longer data association is performed.

[0044] The candidate track contains other candidate tracks in previous frames except the lost target and new tracks in the current frame, which are used for tracking in subsequent frames.

[0045] In a third aspect, the present application provides a computer-readable storage medium comprising computer instructions, which, when executed on a pedestrian multi-target tracking device, cause the device to perform any one of the methods of the first and second aspects.

[0046] Compared with the prior art, the present application has the beneficial effects that the present application alleviates the problem of multi-frame interruption of tracks caused by frequent occlusion in a pedestrian multi-target tracking scene, through double matching, setting a confidence threshold to classify the detection results, and adopting a targeted matching strategy, the occluded pedestrian detection results are included in the matching range in the data association stage to a certain extent, the completeness of the tracking track is improved, and thus the accuracy of tracking is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the embodiments are briefly introduced as follows.

[0048] Figure 1 It is a schematic diagram of the pedestrian multi-target tracking method of the present application.

[0049] Figure 2 A high-confidence detection result matching flowchart of an embodiment of the present application is shown in the figure.

[0050] Figure 3 A low-confidence detection result matching flowchart of an embodiment of the present application is shown in the figure.

[0051] Figure 4 A pedestrian multi-target tracking device diagram of an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0052] In order to more clearly illustrate the technical solutions of the present application, the present application will be further described below in combination with the drawings and embodiments.

[0053] The data association stage is an important part of the pedestrian multi-target tracking technology. At present, the data association stage of the pedestrian multi-target tracking technology is mostly focused on matching high-confidence detection results, which is easy to miss some pedestrian targets with low confidence due to being blocked by other pedestrians or buildings, thereby causing the problem of multi-frame interruption of the trajectory.

[0054] Based on this, the present application provides a pedestrian multi-target tracking method, device and computer readable storage medium, which classifies the detection results by setting high and low confidence double thresholds, adopts a targeted matching scheme for the two types of detection results, reduces the multi-frame interruption of the trajectory caused by blocking, better completes the data association task, and further improves the tracking accuracy.

[0055] Please refer to Figure 1 , Figure 1 A flowchart of a pedestrian multi-target tracking method provided by the present application is shown in the figure. Figure 1 As shown in the figure, the pedestrian multi-target tracking method provided by the present application includes the following steps:

[0056] S1, using a pre-trained model, training on a pedestrian multi-target tracking dataset as a target detector, the process is completed by the following steps:

[0057] S11, obtaining a pedestrian multi-target tracking dataset and pre-processing it.

[0058] The pedestrian multi-target tracking dataset is the MOT17 dataset provided by the MOT Challenge official website, which contains 14 video sequences in total, of which 7 video sequences are used for training the model, and the remaining 7 are used for algorithm testing. It covers a variety of challenging tracking situations, including sequences from static and dynamic cameras, sequences from high and low angles, and all video sequences have large and small targets, moving targets, stationary targets, and difficult samples such as blocked targets.

[0059] S12, determine the deep network model and its input and output structure.

[0060] The deep network model is a DLA34 deep feature fusion network, which is based on ResNet and iteratively stacked between multiple residual modules, expands the hierarchical depth aggregation based on linear aggregation, considers more network levels, and fuses the spatial features of the shallow level and the semantic features of the deep level, thereby improving the deep feature representation capability.

[0061] It should be noted that the output structure of the deep network used in the application can be divided into a detection branch and an appearance feature extraction branch.

[0062] Further, the detection branch thereof is composed of three parallel heads, which are respectively used for estimating the heat map of the target center position, the size of the bounding box and the offset of the target center.

[0063] Further, the appearance feature extraction branch thereof is used to output a 128-dimensional appearance feature vector.

[0064] S13, determine the multi-task learning loss function.

[0065] It should be noted that the model training process of the pedestrian multi-target tracking algorithm provided by the application follows a multi-task learning mode, which needs to complete the pedestrian target detection task to determine the position information of all pedestrians, and also needs to extract the appearance feature vector of the pedestrian, which is used for similarity calculation in the subsequent data association stage.

[0066] The multi-task learning loss function is composed of three parts of heat map regression, bounding box regression and appearance feature extraction.

[0067] Here, the heat map is used to determine the position coordinates of the target center point, and its loss function is calculated as follows

[0068]

[0069] Where N is the total number of pedestrian targets, is the heat map estimation result, M is the heat map label, and α, β are predefined parameters for balancing the training weight of positive and negative samples.

[0070] Here, the bounding box regression is used to determine the size of the bounding box and the offset of the target center point, and the loss function adopts the L1loss method, and the calculation method is as follows

[0071]

[0072] Where and respectively represent the prediction results of the estimated i-th target bounding box center point offset and bounding box size, and λ1, λ2 represent the weight values of the center offset regression task and the bounding box size prediction task respectively.

[0073] Here, the appearance feature extraction branch aims to generate features that can distinguish different targets, so that the similarity between different targets is less than that between the same targets.

[0074] For example, the appearance feature extraction branch extracts the center position of each target in the heat map The corresponding depth feature is mapped into a category vector P according to P={p(k), k∈[1, K]}, where K represents the total number of categories. In the appearance feature extraction task, one category corresponds to the identity of a pedestrian. L i (k) is the classification result of the i-th target in the manually labeled label. The loss function of the appearance feature extraction task is represented as follows

[0075]

[0076] Further, the joint loss function of the algorithm is represented as follows

[0077]

[0078] where w1 and w2 are learnable parameters that balance the detection branch and the appearance feature extraction branch.

[0079] S2, the bounding box information and the appearance feature vector of the pedestrian target are obtained by using the target detector for each frame image of the video. This process is completed by the following steps:

[0080] S21, input the frame image of the video into the depth model, and each output head in the model processes the depth feature output by the network to obtain the heat map, the center point offset of the bounding box, the bounding box size and the appearance feature vector.

[0081] The processing mode is to first complete the secondary transformation of the depth feature through a 3x3 convolution layer, and then complete the dimension reduction through a 1x1 convolution layer.

[0082] S22, the center point coordinate information is obtained by Gaussian function transformation regression on the heat map.

[0083] It should be noted that the response value of each point in the heat map is between 0 and 1, and the closer the response value is to 1, the greater the probability of predicting that the point is the center point of the target.

[0084] S23, combine the center point coordinate, offset, and bounding box size to obtain the pedestrian target detection result.

[0085] S3, set a confidence threshold, divide the pedestrian detection results into a high confidence detection set and a low confidence detection set, and the classification method is as follows:

[0086] S31, set high and low confidence thresholds δ1 and δ2, and the classification method based on confidence is as follows

[0087]

[0088] The high and low confidence thresholds can be obtained through a limited number of experiments.

[0089] S32, define the detection result with a confidence greater than δ1 as high confidence detection, and define the detection result with a confidence in the interval [δ1, δ2] as low confidence detection, which is used for subsequent classification matching process.

[0090] S4, based on the principle of first appearance similarity and then position similarity, preferentially perform data association on the high confidence detection set and the candidate trajectory, and obtain the candidate trajectory and the detection result that are not matched successfully, update the pedestrian appearance feature of the candidate trajectory that is successfully associated, and initialize the detection result that is not successfully associated to obtain a new trajectory.

[0091] Please refer to Figure 2 , Figure 2 The high confidence detection result matching process of the embodiment of the application is shown in the figure. The high confidence detection result and the candidate trajectory are matched through the following steps:

[0092] S41, calculate the pedestrian appearance feature similarity between the high confidence detection result and the candidate trajectory, obtain a similarity matrix, and calculate a cost matrix.

[0093] The calculation and measurement method of the appearance feature similarity is cosine distance. Cosine distance focuses on the difference between two feature vectors in the direction of the vector space. The smaller the cosine distance between the detection result and the candidate trajectory, the smaller the angle between the vectors, and the more similar the appearance features of the two, and the greater the possibility that they belong to the same pedestrian. The cosine distance calculation formula is as follows

[0094] dist(D, T) = 1-cos(D, T)

[0095] When there are N1 detection results and N2 candidate trajectories, an N1xN2 cost matrix needs to be defined, and each element in the matrix represents the cost of associating the candidate detection corresponding to the row coordinate with the to-be-matched trajectory corresponding to the column coordinate. The cost function needs to satisfy the inverse relationship with the similarity of the two, that is, the more similar the candidate detection and the to-be-matched trajectory, the smaller the association cost, and the greater the difference between the candidate detection and the to-be-matched trajectory, the greater the association cost. The cost matrix calculation formula is as follows

[0096] costij = 1-dist(D i , T j )

[0097] S42, obtaining a data association scheme one based on the Hungarian matching algorithm according to the cost matrix, and removing the track successfully matched to the detection result from the candidate track in the data association scheme one.

[0098] The Hungarian matching algorithm is a classic algorithm for solving the optimal scheme of task allocation, commonly used in task scheduling, resource allocation and other fields. The algorithm first subtracts the minimum value of each row of the cost matrix from all elements of the row, then subtracts the minimum value of each column of the cost matrix from all elements of the column, and finally removes all 0 elements in the matrix with the least horizontal or vertical lines, and this process is repeated until the number of lines used is equal to the minimum value of N1 and N2, that is, the association matrix

[0099] It should be noted that the association matrix A needs to satisfy that each to-be-matched track is associated with at most one candidate detection, and each candidate detection is associated with at most one to-be-matched track, which can be expressed as

[0100]

[0101]

[0102] S43, obtaining the state estimation value of the pedestrian, i.e. the predicted position, based on the Kalman filter according to the state vector of the track.

[0103] The Kalman filter first uses an 8-dimensional vector to represent the state of the candidate track, the first 4 dimensions represent the horizontal and vertical coordinates of the center point of the target bounding box, the width-height ratio and the height, and the last 4 dimensions represent the horizontal and vertical coordinate change rate of the center point of the track, the width-height ratio change rate and the height change rate, and then constructs a motion model of the pedestrian target based on the Kalman filter according to the state vector of the track, and calculates the state estimation value of the target.

[0104] S44, calculating the intersection over union (IOU) between the high-confidence detection result and the predicted position of the candidate track to obtain a position similarity matrix, and obtaining a data association scheme two by using the Hungarian matching algorithm, and removing the track successfully matched to the detection result from the candidate track in the data association scheme two.

[0105] The IOU refers to the quotient value obtained by dividing the overlapping area of two regions by the set area. The larger the IOU is, the larger the overlap ratio of the predicted box and the detection box is, and the higher the position similarity is, and vice versa. The calculation formula is as follows

[0106]

[0107] Here, the cost matrix calculation formula for Hungarian matching is

[0108] cost ij =1-IOU(Det i ,Pre j )

[0109] S45, the high confidence detection result associated successfully in the data association scheme one and two is added to the corresponding candidate trajectory, and the pedestrian appearance feature is updated according to a certain weight.

[0110] The feature updating method is to update according to a certain weight, and the calculation formula is as follows

[0111]

[0112] Wherein, alpha is a predefined parameter, used to balance the weight of the re-identification feature of i t In the application, alpha is set to 0.1.

[0113] S46, the high confidence detection result not associated successfully with the candidate trajectory is defined as a new target, the detection result is initialized, and a new trajectory is obtained.

[0114] S5, based on the principle of first position similarity and then appearance similarity, the data association of low confidence detection set and unmatching successful candidate trajectory is implemented, and the continuous multiple frames of unmatching successful candidate trajectory is defined as a lost target.

[0115] Please refer to Figure 3 , Figure 3 It is a low confidence detection result matching flowchart of the embodiment of the application. The low confidence detection result and the candidate trajectory are matched through the following steps:

[0116] S51, based on the Kalman filter, the state estimation value of the pedestrian, i.e. the predicted position, is calculated according to the state vector of the trajectory.

[0117] S52, the intersection over union IOU between the low confidence detection result and the predicted position of the to-be-associated trajectory is calculated, the position similarity matrix is obtained, the data association scheme three is obtained by using the Hungarian matching algorithm, and the trajectory successfully matched to the detection result in the data association scheme three is removed from the candidate trajectory.

[0118] S53, finally, the cosine distance between the low confidence detection result and the pedestrian appearance feature of the to-be-associated trajectory is calculated, the appearance similarity matrix is obtained, and the data association scheme four is obtained by using the Hungarian matching algorithm.

[0119] S54, the low confidence detection result associated successfully in the data association scheme three, four is added to the corresponding candidate trajectory.

[0120] S55, if there are continuous multiple frames of candidate trajectories that are not matched successfully, the candidate trajectories are defined as lost targets, and the matching right is not given to the candidate trajectories in the subsequent frame tracking process.

[0121] The state estimation process, the similarity calculation measurement method and the matching algorithm are the same as those in S3.

[0122] The application also provides a pedestrian multi-target tracking device. Figure 4 , Figure 4 A schematic diagram of the pedestrian multi-target tracking device according to an embodiment of the application is shown in FIG. 1. Figure 4 As shown in FIG. 1, the device comprises a first acquisition module, a first detection module, a first classification module, a first data association module, a second data association module and a first determination module.

[0123] The first acquisition module is configured to acquire first video stream data to be tracked.

[0124] The first detection module is configured to detect pedestrian target information in the first video stream, including pedestrian bounding box position information, corresponding confidence score and appearance feature vector.

[0125] The bounding box position information is used for marking the position similarity calculation in the subsequent data association module.

[0126] The confidence score is used for classifying the detection result.

[0127] The appearance feature vector is used for appearance feature similarity calculation in the subsequent data association module.

[0128] The first classification module is configured to set a confidence threshold to divide the detection result into high confidence detection result and low confidence detection result.

[0129] The first data association module is configured to associate the high confidence detection result with the candidate trajectory one by one.

[0130] The association method first matches the high confidence detection result with the candidate trajectory based on the appearance feature similarity, and then matches the high confidence detection result that is not matched successfully with the candidate trajectory based on the position similarity.

[0131] The second data association module is configured to associate the low confidence detection result with the candidate trajectory one by one.

[0132] The association method matches high-confidence detection results with candidate tracks based on position similarity first, and then matches low-confidence detection results with candidate tracks based on appearance feature similarity.

[0133] The matching process is based on a Hungarian matching algorithm.

[0134] The first determination module is configured to determine new targets, lost targets and candidate tracks for subsequent tracking.

[0135] The new target is a high-confidence detection result of data association failure, which needs to be initialized to obtain a new track.

[0136] The lost target is a candidate track of continuous multi-frame and high / low-confidence detection result association failure, which is defined as a lost target and no longer performs data association.

[0137] The candidate track contains other candidate tracks in previous frames except the lost target and new tracks in the current frame, which are used for subsequent frame tracking.

[0138] The application also provides a computer readable storage medium comprising computer instructions, which, when executed on a pedestrian multi-target tracking device, enable the device to perform each step in the above embodiments.

Claims

1. A pedestrian multi-target tracking method, characterized by, The method comprises the following steps: S1, using a pre-trained model, training on a pedestrian multi-target tracking dataset as a target detector; S2, using the target detector to obtain the bounding box information and appearance features of the pedestrian target for each frame of image of the video; S3, setting a confidence threshold, and dividing the pedestrian detection results into a high-confidence detection set and a low-confidence detection set; S4, based on the principle of first appearance similarity and then position similarity, preferentially performing data association on the high-confidence detection set and the candidate track, and obtaining the candidate track and the detection result that are not matched successfully, updating the pedestrian appearance features of the candidate track that is successfully associated, initializing the detection result that is not successfully associated, and obtaining a new track; S5, based on the principle of first position similarity and then appearance similarity, performing data association on the low-confidence detection set and the candidate track that is not matched successfully, and defining the candidate track that is not matched successfully for consecutive multiple frames as a lost target.

2. The method of claim 1, wherein, The S1 specifically comprises: obtaining a pedestrian multi-target tracking dataset MOT17, which contains 14 video sequences in total, 7 of which are used for training the model, and the remaining 7 are used for algorithm testing; using a pre-trained DLA34 network model as the target detector, constructing a loss function, training and fine-tuning the model parameters by using the training set of the pedestrian multi-target tracking dataset.

3. The method of claim 1, wherein, The S2 specifically comprises: using the target detector to output the center point offset, the bounding box size, the image heat map and the appearance features for each frame of image of the video, and transforming back the center point coordinates of the pedestrian target by using the heat map Gaussian function regression. The appearance features include a 128-dimensional vector, which is used to calculate the appearance feature similarity in the subsequent data association stage.

4. The method of claim 1, wherein, The S3 specifically comprises: setting high and low confidence thresholds δ1 and δ2, and the classification based on the confidence is as follows the detection results with the confidence greater than δ1 are defined as high-confidence detection, and the detection results with the confidence in the interval [δ1, δ2] are defined as low-confidence detection, which are used in the subsequent classification matching process.

5. The method of claim 1, wherein, The S4 specifically comprises: calculating the appearance feature similarity between the high-confidence detection results and the candidate tracks, obtaining a similarity matrix, calculating a matching cost matrix according to the similarity matrix, and obtaining a data association scheme one by using the Hungarian matching algorithm, and removing the tracks that are successfully matched to the detection results from the candidate tracks in the data association scheme one; based on the Kalman filter, calculating the state estimation value of the pedestrian according to the state vector of the track, i.e. the predicted position; calculating the intersection over union (IOU) between the high-confidence detection results and the predicted positions of the candidate tracks, obtaining a position similarity matrix, calculating a matching cost matrix according to the similarity matrix, and obtaining a data association scheme two by using the Hungarian matching algorithm, and removing the tracks that are successfully matched to the detection results from the candidate tracks in the data association scheme two; adding the high-confidence detection results that are successfully associated in the data association schemes one and two to the corresponding candidate tracks, and updating the pedestrian appearance features according to a certain weight; defining the high-confidence detection results that are not successfully associated with the candidate tracks as new targets, initializing them, and obtaining new tracks.

6. The method of claim 1, wherein, The S5 specifically comprises: Based on the Kalman filter, the state estimation value of the pedestrian, i.e. the predicted position, is calculated according to the state vector of the trajectory; The intersection over union (IOU) between the low-confidence detection result and the predicted position of the candidate trajectory is calculated to obtain a position similarity matrix, a matching cost matrix is calculated according to the similarity matrix, and a data association scheme three is obtained by using the Hungarian matching algorithm, and the trajectory successfully matched to the detection result in the data association scheme three is removed from the candidate trajectory; The cosine distance between the low-confidence detection result and the pedestrian appearance feature of the candidate trajectory is calculated to obtain an appearance similarity matrix, a matching cost matrix is calculated according to the similarity matrix, and a data association scheme four is obtained by using the Hungarian matching algorithm; The low-confidence detection result successfully associated in the data association schemes three and four is added to the corresponding candidate trajectory; If there is a candidate trajectory that is not successfully matched for continuous multiple frames, the candidate trajectory is defined as a lost target, and the candidate trajectory is not given the matching right in the subsequent frame tracking process.

7. A pedestrian multi-target tracking apparatus characterized by comprising: The device comprises: a first acquisition module configured to acquire first video stream data to be tracked; a first detection module configured to detect pedestrian target information in the first video stream; a first classification module configured to classify detection results into high-confidence detection results and low-confidence detection results based on confidence; a first data association module configured to one-to-one associate the high-confidence detection results with candidate trajectories; a second data association module configured to one-to-one associate the low-confidence detection results with candidate trajectories; a first determination module configured to determine new targets, lost targets, and candidate trajectories; the new target is a high-confidence detection result that fails in data association, and needs to be initialized to obtain a new trajectory; the lost target is a candidate trajectory that fails to be associated with high-confidence detection results and low-confidence detection results for continuous multiple frames, and when the candidate trajectory is defined as a lost target, the data association is no longer performed on the candidate trajectory; the candidate trajectory contains other candidate trajectories in previous frames except the lost target and new trajectories in the current frame, and is used for tracking in subsequent frames.

8. A computer-readable storage medium, characterized in that, The computer program product comprises computer instructions, and when the computer instructions run on the pedestrian multi-target tracking device, the device performs the steps of the method in any one of claims 1-6. The computer program product comprises computer instructions, and when the computer instructions run on the pedestrian multi-target tracking device, the device performs the steps of the method in any one of claims 1-6.

Images