A multi-target tracking method based on point-surface matching association

Abstract

The application discloses a multi-target tracking method based on point-face matching association, comprising the following steps: acquiring a video to be detected and performing serialization, constructing a detector according to a feature extraction network, sequentially performing target detection on video frames in the video frame sequence according to the detector, obtaining a tth video frame and a t+1th video frame from the video frame sequence, dividing targets in the t+1th video frame into high-confidence targets and low-confidence targets, performing first association matching on the high-confidence targets according to a target frame similarity perception algorithm, adding matched targets into target tracks of the tth video frame, initializing target tracks of the targets which fail to match, respectively, performing second association matching on the low-confidence targets, adding matched targets into target tracks of the tth video frame, discarding the targets which fail to match, and sequentially traversing all the video frames until all the target tracks are obtained. The detection confidence of the occluded target is improved, and the accuracy of multi-target track association is improved.

Description

Technical Field

[0001] This invention relates to the field of multi-target tracking, and more particularly to a multi-target tracking method based on point-area matching association. Background Technology

[0002] Visual object tracking is a widely applied computer vision task. Unlike object classification and object detection, which have clear definitions, object tracking can be divided into two tasks based on different scenarios: 1) Single-object tracking (VOT) and 2) Multi-object tracking (MOT). Single-object tracking involves continuously labeling a target in a video sequence based on an arbitrary target category given in the first frame. Multi-object tracking, on the other hand, involves detecting and tracking targets that may appear or disappear in a video sequence with known target categories, and requires labeling the tracked targets with IDs.

[0003] Early tracking schemes often tracked points of interest in space and time. These trackers were typically simple and fast, performing well and stably in simple scenarios with slow linear motion and few targets. However, their tracking performance was very poor when encountering low-level cues such as image edges. With the development of computer vision, many detection-based target tracking algorithms have become popular. These methods rely on target detection models to detect targets and require data association between targets in consecutive frames to connect their motion trajectories. This two-stage tracking strategy of detection followed by tracking has become the mainstream multi-target tracking paradigm. While detection-based tracking paradigms largely utilize the high performance of deep learning-based target detectors, the data association algorithms for the trackers remain slow and complex. Summary of the Invention

[0004] This invention provides a multi-target tracking method based on point-area matching association to overcome the above-mentioned technical problems.

[0005] A multi-target tracking method based on point-area matching association includes,

[0006] Step 1: Obtain the video to be detected and perform serialization processing on the video to obtain a video frame sequence.

[0007] Step 2: Construct a detector based on the feature extraction network, and then use the detector to perform target detection on each video frame in the video frame sequence and label all detected targets.

[0008] Step 3: Let t = 1, initialize the target trajectory based on the target detected in the first video frame, and initialize the buffer pool, which is used to store the target trajectory set.

[0009] Step 4: Obtain the t-th and t+1-th video frames from the video frame sequence. Divide the targets in the t+1-th video frame into high-confidence targets and low-confidence targets. For high-confidence targets, perform a first association matching based on the target bounding box similarity perception algorithm. Add successfully matched targets to the target trajectory of the t-th video frame, and initialize the target trajectories of unmatched targets. Store the target trajectories in a cache pool. For low-confidence targets, perform a second association matching. Add successfully matched targets to the target trajectory of the t-th video frame, and discard unmatched targets. The second association matching includes obtaining low-confidence targets and representing them as targets to be matched, and performing a two-dimensional Gaussian distribution on the targets to be matched and the targets in the t-th video frame respectively. The modeling process involves obtaining the two-dimensional Gaussian distribution probability of each target after modeling. A distance matrix is ​​constructed based on the two-dimensional Gaussian distribution probabilities of all targets in the t-th and t+1-th video frames. After normalizing the distance matrix, a similarity matrix for the (t+1-th)-th video frame is obtained. Based on the similarity matrix, similarity matching is performed on low-confidence targets in the (t+1-th)-th video frame. If a low-confidence target in the (t+1-th)-th video frame meets the similarity matching condition, it is added to the target trajectory of the t-th video frame, and the updated target trajectory is stored in a cache pool. If a low-confidence target in the (t+1-th)-th video frame does not meet the matching condition, it is discarded.

[0010] Step 5: Determine whether the t-th video frame is the last video frame in the video frame sequence. If yes, obtain all target trajectories. If not, set t = t + 1 and repeat step 4.

[0011] Preferably, the step of constructing a detector based on the feature extraction network includes modifying the feature extraction network. The modification includes using a deep aggregation network as the backbone network of the feature extraction network, employing a deformable convolutional network for feature extraction, using a deep aggregation upsampling module for feature fusion, and adding a global context information extraction module to the feature extraction network. The modified feature extraction network is then used as the detector.

[0012] Preferably, the modeling of the target to be matched and the target in the t-th video frame using a two-dimensional Gaussian distribution includes modeling according to formulas (1) and (2).

[0013]

[0014]

[0015] Where (a,b) represents the center coordinates of the target, w and h represent the semi-axis of the target center along the x-axis and y-axis, respectively, and x, μ, and Σ represent the coordinates (x,y), mean vector, and covariance matrix of the Gaussian distribution, respectively. Let the mean vector μ = [a,b] and the covariance matrix Σ = diag[w 2 / 4,h 2 / 4], the target follows a two-dimensional Gaussian distribution N(μ,Σ), denoted as B~N(μ,Σ).

[0016] Preferably, the step of performing similarity matching on low-confidence targets in the (t+1)th video frame based on the similarity matrix includes setting a similarity threshold and obtaining the similarity matrix S. nm , n represents the number of low-confidence targets in the (t+1)th video frame, m represents the number of target trajectories in the tth video frame, when S ij If the similarity threshold is not met, it means that the i-th target and the j-th target trajectory are not related, and S is set to... ij Set to 0, otherwise do not modify S. ij The values ​​of are given, where i represents the i-th target in the (t+1)-th video frame, j represents the j-th target trajectory in the t-th video frame, i≦n, j≦m, and the updated S nm Input the data into a greedy algorithm to obtain the j-th target trajectory that is most closely related to the i-th target.

[0017] Preferably, dividing the targets in the (t+1)th video frame into high-confidence targets and low-confidence targets includes setting a tracking threshold, acquiring all targets in the (t+1)th video frame, calculating the confidence level based on the pixel values ​​of the targets, and classifying the target as a high-confidence target when the confidence level of the target exceeds the tracking threshold, and vice versa.

[0018] This invention provides a multi-target tracking method based on point-to-surface matching association. By measuring the normalized Gaussian similarity distance between targets and their trajectories in two adjacent video frames, the method predicts the target trajectory, improving the accuracy of multi-target trajectory association. A detector is constructed that extracts global contextual information from low-level features before the feature extraction network, enhancing the expressive power of key texture information in low-level features and thus improving the detection confidence of occluded targets. A secondary association matching and caching strategy for multi-target trajectories is implemented. The PS association algorithm is used to perform secondary matching for occluded or unmatched targets, and a caching mechanism is applied to broken trajectories to form long-term stable multi-target trajectory associations. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of the method of the present invention;

[0021] Figure 2 This is a schematic diagram of the first-stage point-to-surface matching association of the present invention;

[0022] Figure 3 This is a flowchart illustrating the implementation framework of the present invention;

[0023] Figure 4 This is a diagram showing the multi-target tracking results in an embodiment of the present invention;

[0024] Figure 5(a) shows the value analysis of MOTA and IDF1 indices under the NWS similarity constant C of this invention;

[0025] Figure 5(b) shows the value analysis of the IDs index under the NWS similarity constant C of the present invention;

[0026] Figure 6(a) is a comparative test of the MOTA index under the value N of the cached frame number of the present invention;

[0027] Figure 6(b) is a comparative test of the IDF1 index under the cache frame number value N of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Figure 1 This is a flowchart of the method of the present invention, as shown below. Figure 1 As shown, the method in this embodiment may include:

[0030] Step 1: Obtain the video to be detected and perform serialization processing on the video to obtain a video frame sequence.

[0031] Step 2: Construct a detector based on the feature extraction network, and then use the detector to perform target detection on each video frame in the video frame sequence and label all detected targets.

[0032] Step 3: Let t = 1, initialize the target trajectory based on the target detected in the first video frame, and initialize the buffer pool, which is used to store the target trajectory set.

[0033] Step 4: Obtain the t-th and t+1-th video frames from the video frame sequence. Divide the targets in the t+1-th video frame into high-confidence targets and low-confidence targets. For high-confidence targets, perform a first association matching based on the target bounding box similarity perception algorithm. Add successfully matched targets to the target trajectory of the t-th video frame, and initialize the target trajectories of unmatched targets. Store the target trajectories in a cache pool. For low-confidence targets, perform a second association matching. Add successfully matched targets to the target trajectory of the t-th video frame, and discard unmatched targets. The second association matching includes obtaining low-confidence targets and representing them as targets to be matched, and performing a two-dimensional Gaussian distribution on the targets to be matched and the targets in the t-th video frame respectively. The modeling process involves obtaining the two-dimensional Gaussian distribution probability of each target after modeling. A distance matrix is ​​constructed based on the two-dimensional Gaussian distribution probabilities of all targets in the t-th and t+1-th video frames. After normalizing the distance matrix, a similarity matrix for the (t+1-th)-th video frame is obtained. Based on the similarity matrix, similarity matching is performed on low-confidence targets in the (t+1-th)-th video frame. If a low-confidence target in the (t+1-th)-th video frame meets the similarity matching condition, it is added to the target trajectory of the t-th video frame, and the updated target trajectory is stored in a cache pool. If a low-confidence target in the (t+1-th)-th video frame does not meet the matching condition, it is discarded.

[0034] Step 5: Determine whether the t-th video frame is the last video frame in the video frame sequence. If yes, obtain all target trajectories. If not, set t = t + 1 and repeat step 4.

[0035] Based on the above scheme, the trajectory of the target is predicted by measuring the normalized Gaussian-Wasserstein similarity distance between the target and the target trajectory in two adjacent video frames, thereby improving the accuracy of multi-target trajectory association. A detector is constructed to extract global contextual information from low-level features before the feature extraction network, enhancing the expressive power of key texture information in low-level features, thereby improving the detection confidence of occluded targets. A secondary association matching and caching strategy for multi-target trajectories is implemented. The PS association algorithm is used to perform secondary matching for occluded or unmatched targets, and a caching mechanism is applied to broken trajectories to form long-term stable multi-target trajectory association.

[0036] Step 1: Obtain the video to be detected and perform serialization processing on the video to obtain a video frame sequence.

[0037] Step 2: Construct a detector based on the feature extraction network. Global attention information for low-level features is extracted before the feature extraction network. Rich texture and channel information are used to optimize the features of interest. Constructing the detector based on the feature extraction network involves modifying the feature extraction network. This modification includes using a deep aggregation network as the backbone of the feature extraction network, employing a deformable convolutional network for feature extraction, using a deep aggregation upsampling module for feature fusion, and adding a Global Context Information Extraction (GCE) module to the feature extraction network. The modified feature extraction network is then used as the detector.

[0038] The feature extraction network uses a deep aggregation network (DLA) as its backbone to extract multi-level features. The DLA backbone adds more skip connections similar to the feature pyramid network (FPN) between low-level and high-level features. In the fusion part, an iterative deep aggregation upsampling module (IDAUp) is used, which replaces the ordinary convolutional network in IDAUp with a deformable convolutional network (DCN) that can dynamically adjust the receptive field according to the target size to fuse the multi-scale features extracted by DLA.

[0039] Input: The feature extraction network allows input images of size 3×H×W, and the total downsampling rate of the DLA backbone network is 32.

[0040] Output: After passing through the IDAUp fusion upsampling network, the final output is a 4x downsampled feature map with a size of 64×H / 4×W / 4.

[0041] The attention mechanism employed by GCE consists of two parts: spatial attention and channel attention. By simultaneously focusing on points of interest through spatial attention and channel attention, the feature extraction network is allowed to focus on more important features in the input feature map F, helping the network's base layer to better process the spatial and channel information of the input data and extract relevant features.

[0042] The formula for calculating GCE is as follows: as shown in formula (1):

[0043]

[0044] Where MP and AP are the max pooling layer and average pooling layer, respectively, W is the convolutional layer weight, σ is the sigmoid activation function, GMP and GAP are the global max pooling layer and global average pooling layer, respectively, and MLP is a multilayer perceptron. A represents element-wise product. s Representing spatial attention, A c This indicates channel attention.

[0045] The detector sequentially performs target detection on video frames in the video frame sequence and marks all detected targets. The detector detects the target bounding box information Size=(a,b,w,h) of all targets in the t-th frame of the video, where (a,b) are the center coordinates of the target and (w,h) are the width and height of the target bounding box.

[0046] Step 3: Let t = 1. Initialize the target trajectory based on the target detected in the first video frame. Initializing the target trajectory involves storing the target's position information and video frame information in a set with the same target number. Specifically, this includes the target number, target center coordinates, target length, target width, buffer duration, and a set of frame numbers. The buffer duration is the duration the target appears in the video, and the set of frame numbers is the set of sequence numbers for all video frames in which the target appears. Initialize a buffer pool to store the target trajectory set.

[0047] Step 4: Obtain the t-th and t+1-th video frames from the video frame sequence. Divide the targets in the t+1-th video frame into high-confidence targets and low-confidence targets. This division includes setting a tracking threshold, obtaining all targets in the t+1-th video frame, calculating the confidence level based on the target's pixel values, and classifying the target as a high-confidence target if its confidence level exceeds the tracking threshold, and otherwise as a low-confidence target.

[0048] For high-confidence targets, the first association matching is performed according to the target box similarity perception algorithm. The successfully matched targets are added to the target trajectory of the t-th video frame, and the target trajectories of the unmatched targets are initialized respectively and stored in the cache pool. The first association matching according to the target box similarity perception algorithm includes calculating the normalized Gaussian Wasserstein similarity between all high-confidence targets and all target trajectory boxes of the previous frame, matching them one by one according to the maximum similarity score between them, assigning the same target number to the matched high-confidence targets, and assigning new target numbers to the unmatched high-confidence targets.

[0049] For low-confidence targets, a second association matching is performed. Successfully matched targets are added to the target trajectory of the t-th video frame, while unmatched targets are discarded. This second association matching includes acquiring low-confidence targets and representing them as targets to be matched, and modeling the targets to be matched and the targets in the t-th video frame using a two-dimensional Gaussian distribution.

[0050] The step of modeling the target to be matched and the target in the t-th video frame using a two-dimensional Gaussian distribution includes modeling according to formulas (2) and (3).

[0051]

[0052]

[0053] Where (a,b) represents the center coordinates of the target, w and h represent the semi-axis of the target center along the x-axis and y-axis, respectively, and x, μ, and Σ represent the coordinates (x,y), mean vector, and covariance matrix of the Gaussian distribution, respectively. Let the mean vector μ = [a,b] and the covariance matrix Σ = diag[w 2 / 4,h 2 / 4] The target follows a two-dimensional Gaussian distribution N(μ,Σ), denoted as B~N(μ,Σ). The two-dimensional Gaussian distribution is used to remodel the detected target to obtain the probability density of the two-dimensional Gaussian distribution of each target. The distribution weight (weight refers to the probability of the two-dimensional Gaussian distribution) of the foreground pixels from the center of the target to the edge of the target box gradually decreases.

[0054] Obtain the two-dimensional Gaussian distribution probability of each target after modeling, and construct a distance matrix based on the two-dimensional Gaussian distribution probabilities of all targets in the t-th and t+1-th video frames.

[0055] The construction of the distance matrix specifically includes,

[0056] Let the two-dimensional Gaussian distribution of target 1 in the current frame be N1(μ1,Σ1), and the two-dimensional Gaussian distribution of target 2 in the previous frame be N2(μ2,Σ2). The Wasserstein distance between the two targets is described by formula (4).

[0057]

[0058] Simplifying, we get formula (5):

[0059]

[0060] Where ||·||2 represents 2-normal form, ||·|| p This represents the p-normative form. According to the above formula, substituting the target box B1 = (a1, b1, w1, h1), μ1 = [a1, b1], Σ1 = diag[w1], we get... 2 / 4,h1 2 / 4], target box B2=(a2,b2,w2,h2),μ2=[a2,b2],Σ2=diag[w2 2 / 4,h2 2 / 4] yields formula (6):

[0061]

[0062] Here, W represents the Wasserstein distance matrix. The W matrix for adjacent videos is shown in formula (7):

[0063]

[0064] Where m represents the number of target trajectories in the previous frame (which includes all detected targets and unmatched targets, i.e., all trajectories in the cache pool), n represents the number of targets in the current frame, and d represents the Wasserstein distance between a target trajectory in the previous frame and a detected target in the current frame.

[0065] After normalizing the distance matrix, the similarity matrix of the (t+1)th video frame is obtained.

[0066] The construction of the normalized similarity matrix specifically includes building a metric for evaluating the similarity between targets based on the Wasserstein distance of a Gaussian distribution. This metric is called the point-to-surface matching association algorithm, denoted as the PS algorithm.

[0067] Since the value range is not within [0,1], W cannot be directly used as a similarity metric matrix for the two-dimensional Gaussian distribution of the target box. However, W can be normalized by taking the natural exponent to obtain a normalized Gaussian Wasserstein similarity matrix, which is simply called normalized Wasserstein Similarity (NWS), as shown in formula (8).

[0068]

[0069] Where C is a constant, mainly related to the size of the target in the dataset. As the pixel area of ​​the target in the image increases, the constant C should be given a larger value. The NWS matrix is ​​shown in formula (9):

[0070]

[0071] Based on the similarity matrix, similarity matching is performed on the low-confidence targets in the (t+1)th video frame, such as... Figure 2 As shown, the step of performing similarity matching on low-confidence targets in the (t+1)th video frame based on the similarity matrix includes setting a similarity threshold and obtaining the similarity matrix S. nm n represents the number of targets in the (t+1)th video frame, and m represents the number of target trajectories in the tth video frame. When S ij If the similarity threshold is not met, it means that the i-th target and the j-th target trajectory are not related, and S is set to... ij Set to 0, otherwise do not modify S. ij The values ​​of are given, where i represents the i-th target in the (t+1)-th video frame, j represents the j-th target trajectory in the t-th video frame, i≦n, j≦m, and the updated S nmThe data is input into a greedy algorithm to obtain the j-th target trajectory most associated with the i-th target. Specifically, during the data association process, this invention uses the PS algorithm based on normalized Wasserstein similarity to measure the similarity between target boxes. This is achieved by calculating the W distance matrix between all target trajectories and all detected targets in the current frame, and then further transforming this W matrix into an NWS matrix that can be used for similarity measurement. A similarity threshold σ is then given. NWS First remove those less than σ NWS The result is then used to find the most similar matching results in the NWS matrix as the basis for trajectory association through a greedy algorithm.

[0072] When a low-confidence target in the (t+1)th video frame meets the similarity matching condition, the low-confidence target in the (t+1)th video frame is added to the target trajectory of the tth video frame, and the updated target trajectory of the tth video frame is stored in the cache pool. When a low-confidence target in the (t+1)th video frame does not meet the matching condition, the low-confidence target in the (t+1)th video frame is discarded.

[0073] Step 5: Determine whether the t-th video frame is the last video frame in the video frame sequence. If yes, obtain all target trajectories. If not, set t = t + 1 and repeat step 4.

[0074] In this embodiment, all detected targets are divided into high-confidence targets and low-confidence targets according to the detection threshold. First, high-confidence targets are matched with trajectories in the cache pool. Then, unmatched target trajectories are matched a second time with low-confidence targets. Finally, the unmatched trajectories are saved to the cache pool. Figure 3 As shown; specifically including S51, a two-dimensional Gaussian distribution similarity measure of the target bounding box.

[0075] When the detector's detection module outputs a sigmoid-normalized heatmap, all pixels represent the confidence level of the target, based on the detection threshold θ. det , with a confidence level greater than θ det The pixel position is preserved, and other confidence levels are less than θ. det The part is set to 0.

[0076] When associating the target trajectory, set a tracking threshold θ. track ∈(θ det ,1), set the target confidence level in [θ track ,1) The center location and bounding box information of high-confidence targets within the range are modeled as a two-dimensional Gaussian distribution.

[0077] S52. Association Matching of High-Confidence Targets

[0078] For the first frame, initialize trajectories for all detected targets.

[0079] For subsequent frames, a bounding box similarity-aware algorithm is used to calculate the normalized Gaussian Wasserstein similarity between all predicted boxes and the trajectory boxes of the previous frame. Matching is performed one-to-one according to the maximum similarity score between the two predicted boxes. Matched predicted boxes are assigned the same ID, while unmatched predicted boxes are given new IDs, generating new trajectories, and the unmatched trajectories are retained.

[0080] S53, Second Matching of Unmatched Trajectories

[0081] Obtain unmatched trajectories and match these trajectories with the target confidence scores in the range [θ]. det ,θ track All predicted bounding boxes within the interval are matched using the PS algorithm. Unmatched predicted bounding boxes will be cleared, while the remaining unmatched historical trajectories will be retained.

[0082] S54, Track Status Caching Mechanism

[0083] Update all trajectory statuses (including target center coordinates, size information, cache duration, and activity status) and add them to the cache pool. Reset the cache duration of matched trajectories and mark them as active; increase the cache duration of unmatched trajectories and mark them as inactive.

[0084] The invention was validated using the MOT17 dataset provided by the MOT Challenge. The specific process is as follows:

[0085] A. Data Source: The multi-object tracking dataset is derived from the MOT17 dataset provided by the MOT Challenge. It contains 7 training sequences and 7 test sequences, each with 3 detectors: DPM, Faster-RCNN, and SDP, for a total of 21 training sequences and 21 test sequences. The video frame rate is 25-30 frames per second.

[0086] Table 1 shows the video sequence names, corresponding image sizes, and sequence lengths of the MOT17 dataset:

[0087] Table 1

[0088]

[0089] B. For this dataset, use the first half of each training sequence as the training set and the second half as the validation set.

[0090] C. This section mainly focuses on the ablation experiments conducted on the validation set of the MOT17 dataset using the Global Context Information Extraction (GCE) module and the PS association algorithm with a secondary association matching strategy and caching mechanism proposed in this invention, as shown in Table 2. The target detection process is as follows: Figure 4 As shown. MOT17_half means that the model is first pre-trained on the CrowdHuman dataset, then trained on the MOT17 semi-training set, and finally evaluated on the MOT17 validation set; Only CrowdH means that the model weights only use the weights trained on the CrowdHuman dataset, and the model is evaluated on the MOT17 validation set; Scratch means that the model is trained only on the MOT17 semi-training set and tested on the MOT17 validation set.

[0091] Experiments show that the GCE module and PS association algorithm proposed in this invention can significantly improve the target tracking model in the first stage of CenterTrack. In the MOT17_half and Only CrowdH. experiments, the multiple object tracking accuracy (MOTA) index improved by 1.6% and 3.2%, respectively, and the identified detection F1 score (IDF1) improved by 6.7% and 5.2%, respectively. The influence curves of C on different indices are shown below. Figure 5(a) , 5(b) As shown, the comparison results of the effects of CenterTrack and PSTrack on the MOTA index under different N values ​​are as follows: Figure 6(a) , 6(b) As shown, in the Scratch experiment, the CenterTrack feature extraction network has pre-trained weights on ImageNet, while the proposed model, after adding the GCE module, does not fully load the ImageNet pre-trained weights. Therefore, the experimental results with the GCE module are not as good as CenterTrack, but the PS association algorithm still shows strong performance. Overall, the proposed model outperforms CenterTrack. Furthermore, for a fairer comparison, the proposed model was tested against the FRCNN public detector provided on the MOT17 dataset, as shown in Table 3. The experiments show that the PS association algorithm performs better than the CenterTrack center point matching algorithm.

[0092] Table 2

[0093]

[0094]

[0095] Table 3

[0096]

[0097] D. The experiment set the trajectory buffer duration N to 30 frames. The evaluation results on the MOT17 test set are shown in Table 4. Using only the PS association algorithm improved the performance on the MOTA metric by 0.6% and on the HOTA metric by 0.4%. It also showed good performance on metrics such as FN, ID Sw, and Rcall. In PSTrack using the GCE module, because the ImageNet pre-trained weights were not fully loaded during training, the final performance was not as good as PSTrack without the GCE module, but the final tracking performance was still stronger than CenterTrack.

[0098] Table 4

[0099]

[0100] Overall beneficial effects:

[0101] This invention provides a multi-target tracking method based on point-to-surface matching association. By measuring the normalized Gaussian similarity distance between targets and their trajectories in two adjacent video frames, the method predicts the target trajectory, improving the accuracy of multi-target trajectory association. A detector is constructed that extracts global contextual information from low-level features before the feature extraction network, enhancing the expressive power of key texture information in low-level features and thus improving the detection confidence of occluded targets. A secondary association matching and caching strategy for multi-target trajectories is implemented. The PS association algorithm is used to perform secondary matching for occluded or unmatched targets, and a caching mechanism is applied to broken trajectories to form long-term stable multi-target trajectory associations.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-target tracking method based on point-area matching association, characterized in that, include, Step 1: Obtain the video to be detected and perform serialization processing on the video to obtain a video frame sequence. Step 2: Construct a detector based on the feature extraction network, and then use the detector to perform target detection on each video frame in the video frame sequence and label all detected targets. Step 3: Let t=1, initialize the target trajectory based on the target detected in the first video frame, and initialize the buffer pool, which is used to store the target trajectory set. Step 4: Obtain the t-th and t+1-th video frames from the video frame sequence. Divide the targets in the t+1-th video frame into high-confidence targets and low-confidence targets. For high-confidence targets, perform a first association matching based on the target bounding box similarity perception algorithm. Add successfully matched targets to the target trajectory of the t-th video frame, and initialize the target trajectories of unmatched targets. Store the target trajectories in a cache pool. For low-confidence targets, perform a second association matching. Add successfully matched targets to the target trajectory of the t-th video frame, and discard unmatched targets. The second association matching includes obtaining low-confidence targets and representing them as targets to be matched, and performing a two-dimensional Gaussian distribution on the targets to be matched and the targets in the t-th video frame respectively. The modeling process involves obtaining the two-dimensional Gaussian distribution probability of each target after modeling. A distance matrix is ​​constructed based on the two-dimensional Gaussian distribution probabilities of all targets in the t-th and t+1-th video frames. After normalizing the distance matrix, a similarity matrix for the (t+1-th)-th video frame is obtained. Based on the similarity matrix, similarity matching is performed on low-confidence targets in the (t+1-th)-th video frame. If a low-confidence target in the (t+1-th)-th video frame meets the similarity matching condition, it is added to the target trajectory of the t-th video frame, and the updated target trajectory is stored in a cache pool. If a low-confidence target in the (t+1-th)-th video frame does not meet the matching condition, it is discarded. Step 5: Determine whether the t-th video frame is the last video frame in the video frame sequence. If yes, obtain all target trajectories. If no, set t=t+1 and repeat step 4. The step of modeling the target to be matched and the target in the t-th video frame using a two-dimensional Gaussian distribution includes modeling according to formulas (1) and (2). (1) (2) in,( a , b () represents the center coordinates of the target. w , h They represent the target center along x The half-axis and along the shaft y The semi-axis in the axial direction, x, μ , Σ Represent the coordinates of the Gaussian distribution ( x , y ), mean vector and covariance matrix, let mean vector μ =[ a , b ], covariance matrix Σ = diag [ w 2 / 4, h 2 / 4], the target follows a two-dimensional Gaussian distribution N( μ , Σ ), denoted as B~N( μ , Σ ).

2. The multi-target tracking method based on point-area matching association according to claim 1, characterized in that, The step of constructing a detector based on a feature extraction network includes modifying the feature extraction network. The modification includes using a deep aggregation network as the backbone of the feature extraction network, employing a deformable convolutional network for feature extraction, using a deep aggregation upsampling module for feature fusion, and adding a global context information extraction module to the feature extraction network. The modified feature extraction network is then used as the detector.

3. The multi-target tracking method based on point-area matching association according to claim 1, characterized in that, The step of performing similarity matching on low-confidence targets in the (t+1)th video frame based on the similarity matrix includes setting a similarity threshold and obtaining the similarity matrix. S nm , n This represents the number of low-confidence targets in the (t+1)th video frame. m This represents the number of target trajectories in the t-th video frame, when... S ij When the similarity threshold is not met, it indicates that the first... i The first goal and the first j The target trajectories are not related, S ij Set to 0, otherwise leave unchanged. S ij The value of , i Represents the (t+1)th video frame. i One goal, j Represents the t-th video frame. j Target trajectories, i≦ n ,j≦ m The updated version S nm Input into a greedy algorithm to obtain the result with respect to the first... i The most relevant to the first goal j The target trajectory.

4. The multi-target tracking method based on point-area matching association according to claim 1, characterized in that, The step of dividing the targets in the (t+1)th video frame into high-confidence targets and low-confidence targets includes setting a tracking threshold, acquiring all targets in the (t+1)th video frame, calculating the confidence level based on the pixel values ​​of the targets, and classifying the targets as high-confidence targets when the confidence level of a target exceeds the tracking threshold, and as low-confidence targets otherwise.

Images