Unsupervised tracking model training method based on contrastive loss

By adopting an unsupervised training method based on contrast loss in the multi-objective tracking algorithm, using the relationship between the targets inside the video frame and adjacent frames to build constraints, the problems of model fitting difficulties and sub-task conflicts in the existing technology are solved, and tracking is improved performance.

CN116580060BActive Publication Date: 2024-11-26CHONGQING UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310631895.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-11-26
Estimated Expiration
2043-05-31

AI Technical Summary

Technical Problem

The existing multi-objective tracking algorithm is difficult to fit the model when there are many trajectories and inconsistent trajectory lengths, resulting in insufficient performance. The JDE paradigm causes conflicts between subtasks due to the shared backbone network, which is not effective.

Method used

Using the unsupervised tracking model training method based on contrast loss, the SSCI module uses the relationship between the internal and adjacent frame targets of the video frame, and uses self-supervised contrast loss to constrain the embedded features to enhance the cross-frame expression capability.

Benefits of technology

This improves the cross-frame feature expression ability of the model, reduces the dependence on trajectory annotation, and improves the performance of multi-objective tracking, especially when there are too many trajectories and inconsistent lengths.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116580060B_ABST
    Figure CN116580060B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of unsupervised tracking technology, and in particular to an unsupervised tracking model training method based on contrast loss. The steps are as follows: S1: using the relationship between the internal and adjacent video frame targets to form a constrained SSCI module; S2: setting the features of different targets in each frame image as negative samples, and setting the adjacent frame targets similar to the adjacent frames as positive sample pairs, and constructing the contrast loss S3: using a variant loss based on self-supervised contrast loss to constrain the embedded feature #imgabs0#. The present invention provides an unsupervised tracking model training method based on contrast loss, which relies on the prior that objects in the frame must be different to infer the similarity between distant targets; then inspired by the self-supervised learning method, similar objects between two frames with a short interval are matched as positive sample pairs to enhance the cross-frame expression ability of features; finally, based on the prior that the forward and reverse matching must be consistent, the cross-frame expression ability of features is further enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of unsupervised tracking, and in particular to an unsupervised tracking model training method based on contrast loss. Background Art

[0002] Mainstream multi-target tracking algorithms are implemented by target detection plus representation vector extraction. In order to improve the tracking effect, researchers first proposed to use an additional appearance feature extractor to increase the available information when correlating the previous and next frames of the tracking task, but the use of multiple models makes it difficult for the model to meet real-time requirements. In order to meet the real-time requirements, researchers proposed a multi-target tracking model based on the Joint Detection and Embedding (JDE) paradigm. However, no matter which approach is used, as long as the correlation information of the objects in the previous and next frames is used in the tracking strategy, extremely labor-intensive trajectory annotation is required;

[0003] Existing methods treat embedding training as classification, which will bring some new problems. They treat each trajectory in the dataset as a category and constrain the embedding branch by classifying the features obtained by the embedding branch. This training method can achieve good results when the number of trajectories is small, but if the number of trajectories is too large, it will make the model difficult to fit (the number of outputs of the fully connected layer is proportional to the number of trajectories), and the inconsistency of the length of the trajectories in the dataset will lead to an imbalance in the number of samples in each category, which will limit the performance of the JDE paradigm tracker. At the same time, since the JDE paradigm uses a common backbone network to extract unified features for multiple tasks, there are actually certain conflicts between subtasks. This situation leads to the JDE paradigm model being insufficient in effect.

[0004] To this end, we designed an unsupervised tracking model training method based on contrast loss to provide another technical solution to the above technical problems. Summary of the invention

[0005] Based on this, it is necessary to provide an unsupervised tracking model training method based on contrast loss to address the above technical problems and solve the technical problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] An unsupervised tracking model training method based on contrast loss, the steps are as follows:

[0008] S1: The SSCI module constrained by using the relationship between the objects within the video frame and the objects in adjacent video frames;

[0009] S2: According to the features of different targets in each frame image, they are set as negative samples, and the targets in adjacent frames that are similar in adjacent frames are set as positive sample pairs, and the contrast loss is constructed in this way;

[0010] S3: Using a variant loss based on self-supervised contrast loss to embed features To impose restraints;

[0011] S4: Enhance the cross-frame expression ability of features through forward matching and reverse matching

[0012] S5: Verify the tracking accuracy using the MOTChallenge dataset.

[0013] As a preferred implementation of the unsupervised tracking model training method based on contrast loss provided by the present invention, the calculation basis of the SSCI module is as follows:

[0014] The targets in the same frame must be different;

[0015] Targets in adjacent frames can obtain matching pairs with higher accuracy based on embedded features.

[0016] As a preferred implementation of the unsupervised tracking model training method based on contrast loss provided by the present invention, the targets of adjacent frames are used to construct positive sample pairs, and the steps are as follows:

[0017] Use two consecutive frames of images to form a short sub-video segment as the model input. At this time, the data of each sub-video can be expressed as

[0018] As a preferred implementation of the unsupervised tracking model training method based on contrast loss provided by the present invention, after these sub-videos are input into the network, the corresponding feature vectors can be obtained according to the detection annotations of the t-th frame and the t+1-th frame. and

[0019] Among them, x represents the feature vector of the corresponding target, and kt and kt+1 represent the number of targets in the frame image respectively.

[0020] As a preferred implementation of the unsupervised tracking model training method based on contrast loss provided by the present invention, the cross-frame expression capability of the feature is enhanced by forward matching and reverse matching, and the steps are as follows:

[0021] The matrix M is divided into four sub-matrices Mt,t, Mt+1,t+1, Mt,t and Mt+1,t+1;

[0022] The Mt,t and Mt+1,t+1 represent the similarity between the objects in the t frame and the t+1 frame respectively; the Mt,t+1 and Mt+1,t represent the similarity between the objects in the frames t and t+1;

[0023] SSCI uses the Hungarian algorithm in Mt,t+1 as a forward match from the target in the tth frame to the target in the t+1th frame to obtain matching pairs of the same objects in adjacent frames;

[0024] The loss function Lcycle acts on the elements in Mt+1,t, using the forward matching diagonal elements as reverse matching.

[0025] As a preferred implementation of the unsupervised tracking model training method based on contrast loss provided by the present invention, the MOTChallenge includes MOT17 and MOT20;

[0026] The MOT17 dataset includes a training set and a test set. The training set contains 5316 frames of images from 7 videos, and the test set also contains 7 videos and a total of 5919 frames.

[0027] The MOT20 dataset includes a training set and a test set. The training set occupies 4 videos and 8931 frames of images, and the test set occupies 4 videos and 4479 frames of images.

[0028] As a preferred implementation of the unsupervised tracking model training method based on contrast loss provided by the present invention, the ratio of the training set to the test set in the MOT17 is 5:5.

[0029] It can be seen without a doubt that the above-mentioned technical solution of the present application can definitely solve the technical problem to be solved by the present application.

[0030] At the same time, through the above technical solutions, the present invention has at least the following beneficial effects:

[0031] The present invention provides an unsupervised tracking model training method based on contrast loss, which relies on the prior that objects in a frame must be different to infer the similarity between distant targets; then, inspired by the self-supervised learning method, similar objects between two frames with a short interval are matched as positive sample pairs to enhance the cross-frame expression ability of features; finally, based on the prior that forward and reverse matches must be consistent, the cross-frame expression ability of features is further enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0033] Figure 1 It is a schematic diagram of the unsupervised contrastive learning training framework of the present invention;

[0034] Figure 2 Schematic diagram of the JDE tracker supervised training framework of the present invention;

[0035] Figure 3 A schematic diagram of the commonly used loss in representation learning of the present invention;

[0036] Figure 4 It is a key a priori schematic diagram of the present invention;

[0037] Figure 5 It is the overall framework diagram of SCI of the present invention;

[0038] Figure 6 It is a simulation tracking structure diagram of the present invention;

[0039] Figure 7 Schematic diagram of the impact of the three types of losses on the matching results during training;

[0040] Figure 8 It is a visualized heat map of the present invention;

[0041] Fig. 9 It is a visualization diagram of the tracking effect of the MOT17 test set of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0044] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0045] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0046] Reference Figure 1-Figure 9 , an unsupervised tracking model training method based on contrast loss, the steps are as follows:

[0047] Unsupervised training is achieved through the SSCI (Self-Supervised Contrastive ID) loss module; SSCI constructs constraints on the embedding branch only based on the association between short-term targets within the video frame and adjacent video frames; SSCI proposes two key prior information based on the inherent relationship between targets within the video frame and adjacent frames:

[0048] 1) The targets in the same frame must be different;

[0049] 2) Targets in adjacent frames can obtain matching pairs with high accuracy based on the embedded features (even if the parameters of the embedding branch are randomly initialized).

[0050] The positive and negative sample pairs required for contrastive loss can be obtained from the two priors, that is, the matching pairs obtained by prior 2) are used as positive sample pairs in contrastive learning, and the embedded features of other targets are used as negative samples, thereby realizing self-supervised training of the embedding branch.

[0051] The JDE tracker during supervised training will have a representation as A data set, where Represents a frame of image. Represents the k in the current frame image t The location of the target, Represents the current frame k t The track number to which the object belongs. These JDE trackers will output the predicted position of the target in a single forward propagation and embedding features (D represents the dimension of the feature vector), and the loss of the JDE tracker is shown in Formula 1:

[0052] L JDE =L DETECTION +L ID (1)

[0053] Where L DETECTION Is With B t The detection loss determined by the gap between ID is the loss of the embedding branch. Embedding features It will be input into a fully connected layer used only during training for classification, and get Last passed With y t Calculate the cross entropy loss to get L ID .

[0054] 1. Common losses for representation learning

[0055] The three most common representation losses are Cross-Entropy Loss, Triplet Loss, and Contrastive Loss. Their relative constraints are as follows: Figure 3 As shown. The calculation formula of cross entropy loss is shown in Formula 2:

[0056]

[0057] According to the formula and Figure 3 As shown in (a), the cross entropy loss requires the features to be classified in advance, and the features of the same type are gathered in the adjacent feature space, while the feature centers of the features of different categories are pushed away. The embedding branch of the supervised JDE tracking is trained using this loss, but since the present invention does not use the trajectory labels of the entire data set, the cross entropy loss cannot be used. The calculation formula of the triplet loss is shown in Formula 3:

[0058]

[0059] The triplet loss no longer needs to determine the specific category of each feature. It only needs to know whether the features for loss calculation are of the same category. The triplet loss is more flexible than the cross entropy loss, but because it does not have a clear feature category center like the cross entropy loss, the effect will be reduced. In addition, the sampling strategy will have a huge impact on the effect of the triplet loss. The farthest positive sample and the nearest negative sample are used instead of random sampling for optimization. Figure 3 As shown in (b), the triplet loss only brings one positive sample closer and pushes one negative sample away each time. This strategy will also affect the effect when the negative samples are more dispersed. The calculation formula of contrast loss is shown in Formula 4:

[0060]

[0061] According to the formula and Figure 3As shown in (c), the contrastive loss does not need to determine the specific category of each feature, just like the triplet loss. This gives the contrastive loss the flexibility of the triplet loss. However, unlike the triplet loss which only pushes away one negative sample each time, the contrastive loss will push away all sampled negative samples at the same time, which makes the category center of the positive sample pair clearer and makes the feature center points of different categories more evenly dispersed in the feature space. The difficulty of the contrastive loss is that a large number of negative samples need to be sampled at the same time to achieve better results. This problem does not exist in the multi-target tracking dataset in dense scenes. Different targets in a smaller batch are sufficient to provide enough negative samples, so the SSCI module will use a contrastive loss that is more suitable for tracking scenarios.

[0062] The SSCI module is a constraint module that uses the relationship between the objects in the video frame and the objects in the adjacent video frames. The SSCI module is only a loss calculation module. The motivation and basis of its design come from two key prior information, namely, the objects in the same frame must be different, and the objects in adjacent frames can be matched with a high accuracy rate according to the embedded features. These two priors are shown as follows Figure 4 As shown;

[0063] according to Figure 4 The two prior information shown in the figure set the features of different targets in each frame of the image as negative samples, and set the adjacent frame targets with similar adjacent frames (matching results of adjacent frames ss) as positive sample pairs, and construct the contrast loss based on this. The overall structure of SSCI can be seen Figure 5 SSCI is a module used only during model training. When using SSCI, the dataset will be different from the supervised learning dataset mentioned above, that is, it will no longer have the trajectory annotation y. The dataset at this time will be represented as At the same time, in order to use the targets of adjacent frames to construct positive sample pairs, SSCI uses two consecutive frames of images to form a short sub-video segment as the model input. At this time, the data of each sub-video can be expressed as

[0064] After these sub-videos are input into the network, their corresponding feature vectors can be obtained based on the detection annotations of the tth frame and the t+1th frame. and Where x represents the feature vector of the corresponding target, k t and k t+1 They represent the number of targets in the frame image. Since trajectory annotation cannot be used, cross entropy loss cannot be used to construct embedded features. Therefore, the present invention uses three variant losses based on self-supervised contrast loss to perform constraints. The original formula of self-supervised contrast loss is shown in Formula 5:

[0065]

[0066] Where sim(x i ,x i + ) means the cosine similarity between the ith sample and its positive sample, sim(x i ,x j ) represents the similarity between the i-th target and samples other than itself, and τ is the temperature that controls the degree of constraint of difficult samples. From this formula, we can also understand that the construction of positive and negative samples is the most important part of contrast loss.

[0067] like Figure 5 As shown, in getting and After that, concatenate them and calculate the cosine similarity matrix between all x The corresponding value m of each point in the matrix i,j Calculated as shown in Formula 6:

[0068]

[0069] m i,j The value of represents the cosine similarity between the corresponding two target embedding vectors. Figure 5 The matrix M shown can be divided into four sub-matrices. t,t and M t+1,t+1 Represents the similarity between the targets in frame t and frame t+1 respectively. t,t+1 and M t+1,t It represents the similarity between objects in frames t and t+1. Based on the prior knowledge that objects in the same frame must be different objects

[0070] Information condition, firstly, we design the loss function L for negative samples in the same frame. same , as shown in Formula 7:

[0071]

[0072] L same The denominator of the first term is M t,t The sum of all elements except the diagonal elements in M, which tends to push the distance between all target features in frame t away. The second term is the sum of all elements except the diagonal elements in M. t+1,t+1 The denominators of these two items are consistent with the denominators of contrast loss, but the numerator of contrast loss is the similarity between positive sample pairs, and it is impossible for positive samples to exist in the same frame image. Therefore, L sameWhile retaining the softmax-like operation in the contrast loss, the similarity of the positive sample pair in the numerator is replaced by the similarity of the negative sample pair, and the log operation and negative operation are no longer performed to ensure that the optimization direction of the loss is consistent with the direction in which the distance of the negative sample increases. same There is another simpler constraint that is easier to think of, which is to set M t,t and M t+1,t+1 The non-diagonal values ​​are directly added together as the loss, but this simple constraint does not produce good results.

[0073] The first loss L same It only works on objects in the same frame and does not constrain targets across frames, which is the most important capability required for tracking tasks. t,t+1 The Hungarian algorithm is used as a forward match from the target in the tth frame to the target in the t+1th frame to obtain matching pairs of the same objects in adjacent frames, that is, Figure 5 Medium cross These matching pairs will be considered as positive pairs, and the second loss L will be calculated according to Formula 8 cross, The formula is as follows:

[0074]

[0075] L cross The calculation method is the same as that of the self-supervised contrast loss, and its purpose is to bring the similarity between matching pairs of adjacent frames closer. cross The matching operation in is interpreted as forward tracking, and it is proposed that the forward tracking result should be consistent with the backward tracking result. Backward tracking is to match the target of the next frame with the target of the first frame. In order to ensure this consistency, this section proposes a third loss function L cycle , and is calculated as shown in Formula 9:

[0076]

[0077] L cycle Acts on M t+1,t The elements in , which use the forward matching diagonal elements as reverse matching, do not use additional matching operations, that is, Figure 5 Medium cycle Reverse operation. This can further shorten the distance between the features of the matching pairs. SSCI defines the loss of the embedding branch as the sum of the above three losses, namely:

[0078] L ID =L same +L cross +L cycle (10)

[0079] At the same time, since the number of negative samples is crucial for contrast loss, SSCI samples target boxes from different scenes in the same batch as additional negative samples. Then calculate To replace the original M for subsequent loss calculation.

[0080] 2 Experiments and analysis

[0081] 2.1 Training Dataset and Indicators

[0082] The present invention will use the MOTChallenge dataset, which includes MOT17 and MOT20. The MOT17 dataset includes a training set and a test set. The training set contains 5316 frames of images from 7 videos, and the test set also contains 7 videos and a total of 5919 frames. MOT20 is a dataset with denser targets than MOT17, in which the training set occupies 4 videos and 8931 frames of images, and the test set occupies 4 videos and 4479 frames of images. Except for the test experiments in this section, the rest of the experiments use the first half of the MOT17 data as the training set and the second half of the data as the validation set for experiments. In the test set experiments, the additional CrowdHuman, ETH, CityPersons, CalTech, CUHK-SYSU and PRW datasets will be used, consistent with JDE, FairMOT and Cstrack.

[0083] In terms of evaluation indicators, the present invention will use the standard MOTChallenge evaluation indicators, and focus on MOTA, IDF1, MT (Mostly Tracked objects), ML (Mostly Lost objects), and IDS (Number of Identity Switches).

[0084] 2.2 Training details and parameter settings

[0085] In order to ensure the adequacy of the experiment, the present invention applies unsupervised training to FairMOT, Cstrack and OMC for corresponding effect comparison. At the same time, in order to ensure the fairness of the comparison, the present invention will maintain the standard hyperparameters of these networks. Both Cstrack and OMC will use the SGD optimizer for training for 30 rounds. The learning rate is initialized to 5*10-4 and decays to 5*10-5 in 20 rounds. The weights of detection loss and embedding loss also use 1:0.02 in the original paper. FairMOT uses the Adam optimizer for training for 30 rounds, and the learning rate is set to 1*10-4, and the detection loss and embedding loss use learnable weights. All training of the present invention will be performed on a Tesla V100 GPU. Continuous frames in unsupervised training will be randomly extracted from the 10 frames before and after the first frame according to the video frame rate.

[0086] 2.3 Verification Experiment

[0087] The present invention will conduct all the verification experiments mentioned above. That is, verify the above: 1) the features extracted by the randomly initialized embedding branch can still distinguish objects in short interval frames; 2) L same The impact of using simple addition as loss and using triplet loss instead of contrast loss on the experiment; 3) verify that the competition problem still exists in Cstrack using the CCN module.

[0088] The randomly initialized embedding branch can still obtain a certain embedding feature when the interval between two frames is small. The key prior will be L cross To verify this prior, the present invention simulates tracking using features of the randomly initialized embedding branch output, and uses these features for matching to check the accuracy.

[0089] Specifically, the present invention passes the 28th frame image in the MOT17-09 sequence and its subsequent 1, 5, 10 and 20 frames through a network that only loads the coco pre-trained weights (since the pre-training is only for the detection branch, the embedding branch is randomly initialized at this time), calculates the similarity matrix M of the embedded features, and uses the Hungarian algorithm to match them according to the similarity to obtain the following: Figure 6 The results are shown in Figure 2. It proves that when the interval between the selected images is short, the untrained embedding branch can still provide effective features, and as the interval increases, this effectiveness decreases. Therefore, in order to ensure that a high-accuracy matching pair can be found during training, subsequent experiments will randomly select the second frame from the 10 frames before and after the first frame.

[0090] What also needs to be verified is the impact of using Formula 3 and Formula 11 to replace Formula 7 on the experiment.

[0091]

[0092] Figure 7 It shows that when using these three loss training, each iter calculates L cross The number of matching pairs and matching accuracy obtained previously are averaged over the entire epoch. The number and accuracy of matching pairs are crucial to the constraints of adjacent frames, so the impact of the loss in a single frame on the matching pairs can also reflect its impact on the training effect to a certain extent.

[0093] from Figure 7 It can be found that the use of formula 7 can maintain a relatively high matching accuracy, and the number of matches increases steadily with the increase in training rounds; while the use of formula 11 can quickly obtain a high number of matches, but its accuracy is difficult to guarantee; the use of formula 3 will result in a continuous increase in the number of matches, but the matching accuracy has not increased significantly. The present invention believes that the reason for this result is that although formula 7 does not directly use the information of the adjacent frame targets as the loss, it uses the adjacent frame information for softmax, which makes the loss make the similarity of the negative samples in the current frame tend to 0 while maintaining the stability of the adjacent frame target features; while formulas 3 and 11 only consider the features of the targets in the current frame, which results in the features of the two frames being irrelevant and reduces the correlation. So in the end L same We chose to use formula 7. Both Cstrack and FairMOT mentioned the problem of branch contention and gave corresponding solutions.

[0094] In order to verify whether the competition problem still exists, the present invention conducted a simple experiment. As shown in Table 1, the first two rows are the results of Cstrack without training the embedding branch and training the embedding branch, and the last two rows are the results of FairMOT. Since the IDF1 indicator better reflects the tracking effect and MOTA better reflects the detection effect, the present invention uses IDF1 to represent the tracking effect and MOTA to represent the detection effect. As can be seen from Table 1, although the training embedding branch can indeed greatly improve the tracking effect.

[0095] Table 1. The impact of training / not training the embedding branch on the indicators

[0096]

[0097] 2.4 Ablation Experiment and Parameter Experiment of Embedding Branch Unsupervised Contrastive Loss Module

[0098] The present invention will conduct ablation studies on three types of losses, number of negative samples, hard sample temperature and training matching threshold, and display the visualization results. All experiments involved in the present invention will be implemented based on FairMOT.

[0099] First, the ablation study on SSCI,

[0100] SSCI consists of 3 sub-losses: L same Responsible for zooming out the features of the target in the same frame; L cross Responsible for narrowing the gap between the positive sample pairs that successfully match adjacent frames; L cycle It is responsible for ensuring that the forward and reverse matching results remain consistent.

[0101] Table 2 shows the effect of using each loss in the validation set, where the fourth row shows the effect of supervised training. same It can achieve the same effect as supervision. cross and L cycle Afterwards, IDF1 is significantly improved and IDS is reduced, that is, the effect of the embedding branch is improved, but at the same time, the recall is reduced (FN is reduced) and the MOTA is reduced. The present invention believes that the competition between the embedding branch and the detection branch causes this result.

[0102] Due to L cross and L cycle Based on contrast loss, the number of negative samples will have a great impact on the effect of contrast loss, so the present invention studies the number of negative samples. cross and L cycle All of them constrain the successfully matched positive sample pairs, so the remaining targets in the current two frames can be naturally regarded as negative samples. At the same time, since the MOT17 dataset is composed of multiple video segments, the targets of different videos can be considered to be different, so the present invention fills the targets of different videos in the same batch as negative samples. Here, the negative samples filled from different video segments are regarded as additional negative samples, and the number of these additional filled negative samples is analyzed. Table 3 shows the effect of FairMOT when using different numbers of negative samples, where N t is the target number of the first frame. From Table 3, we can find that more negative samples generally lead to higher IDF1, but at the same time reduce MOTA. Therefore, in order to balance the most critical MOTA and IDF1 indicators, SSCI finally chose N neg / N t =2.

[0103] Table 2 Ablation experiments of three types of losses

[0104]

[0105] Table 3. Experiments related to the number of additional negative samples

[0106]

[0107] The self-supervised contrast loss uses a temperature to control the weight of difficult samples (see Formula 5, Formula 7, Formula 8 and Formula 9), and sets the temperature to 0.5. It is mentioned that this value will have different optimal values ​​depending on the task, so the present invention compares the effects of different fixed T values ​​in Table 4 and adds the effect comparison of adaptive T values. From the results in the table, it can be seen that T = 2 can still achieve the best results at a fixed value, but T dynamically obtained according to the number of targets can achieve the best results, so the T of SSCI will be set to T = 1 / 2 (log (N t +N t+1 +1)).

[0108] Table 4 Related experiments on difficult sample T values

[0109]

[0110] Table 5 Hungarian algorithm linear allocation threshold related experiments

[0111]

[0112]

[0113] Since during training, L cross and L cycle If the linear matching of the Hungarian algorithm is needed to construct positive sample pairs, then the threshold in the Hungarian algorithm will inevitably affect the correctness and quantity of the pairing, thereby affecting the final effect. The present invention compares the effects of using different thresholds in Table 5, where N match and N right They represent the ratio of successful matches to the total number of targets and the ratio of correct matches to successful matches in the last epoch of training. From the table, we can see that a higher thresh will lead to a significant decrease in the number of successful matches, but will not increase the accuracy rate too much, while a lower thresh will increase the number of matches while reducing the accuracy rate. Based on the experimental results, SSCI finally chose to set thresh = 0.7.

[0114] Finally, a series of visualizations are performed on the features generated by the embedding branch trained using SSCI to demonstrate its performance comparable to that of supervised learning.

[0115] First, the present invention uses feature thermal response diagrams to demonstrate the discriminative power of features obtained through unsupervised embedding training. Figure 8As shown in Figure 1, (b) shows a randomly selected frame from the validation set, and then the images of the subsequent 1, 5, 10, and 20 frames are extracted in sequence. The first frame contains the query instance, and the subsequent extracted frames contain the target instance with the same ID. The thermal response map is obtained by calculating the cosine similarity between the embedding features of the query instance and the output feature map of the entire embedding branch of the subsequent frames.

[0116] Figure 8 (a) and (c) in the figure show the thermal response maps of the tracked target and the frame shown in (b) for the next 1, 5, 10 and 20 frames respectively. The features in (a) come from FairMOT trained with SSCI, while the features in (c) come from FairMOT trained with supervision. It can be seen from (a) and (c) that the heatmap with an interval of 1 frame has a high error response on the adjacent pedestrians regardless of supervision or unsupervised, but from the heatmap with a longer interval, it can be seen that all the positions with similar color information to the selected target in the heatmap of supervised training have high error responses, so it can be inferred that the features of supervised training are more likely to focus on color information. At the same time, the model trained with SSCI has only low response values ​​at these wrong positions, but has high response values ​​at the true positions. This proves the effectiveness of SSCI.

[0117] 2.5 Comparative Analysis of Test Set Effects

[0118] Table 6 lists the results of comparing the multi-target tracking algorithm trained by the present invention with the currently advanced supervised and unsupervised tracking algorithms on the MOT17 dataset. It can be seen that the present invention can achieve performance that is not much different from its corresponding supervised method in terms of major tracking indicators. Achieving results similar to those of the supervised method without using trajectory annotation can be considered a usable training mode. Compared with other unsupervised algorithms, only OUTrack uses additional supervisory signals to achieve better results than the present invention. This result proves that the present invention is close to the best among unsupervised tracking methods. Table 7 lists the results of comparing the multi-target tracking algorithm trained by the present invention with the currently advanced supervised and unsupervised tracking algorithms on the MOT20 dataset.

[0119] Table 6 Comparison of MOT17 test set results

[0120]

[0121]

[0122] Table 7 Comparison of MOT20 test set results

[0123]

[0124] 2.6 Visualization Results

[0125] Fig. 9 The tracking conditions of the present invention in three different scenes on the MOT17 test set are shown. Each row in the figure represents a different scene, and the present invention is used for tracking and the results are taken out at intervals of 30 frames as shown in the pictures in each row. It can be seen from the figure that even for small targets at a long distance, the present invention can still perform long-term tracking well.

[0126] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the present invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for training an unsupervised tracking model based on contrast loss, characterized in that: Here are the steps: S1: A Self-Supervised Contrastive ID module that uses the relationship between objects within a video frame and between adjacent video frames to form constraints; S2: According to the features of different targets in each frame image, set each other as negative samples and set the adjacent frame targets with similar features in adjacent frames as positive sample pairs to construct contrast loss; S3: Using a variant loss based on self-supervised contrast loss to embed features To impose restraints; S4: Enhance the cross-frame expression ability of features through forward matching and reverse matching; S5: Use MOTChal lenge dataset to verify tracking accuracy; The targets of adjacent frames are used to construct positive sample pairs. The steps are as follows: Two consecutive frames of images form a short sub-video segment as the model input. The data of each sub-video can be expressed as After the sub-video is input into the network, its corresponding feature vector can be obtained according to the detection annotation of the tth frame and the t+1th frame and Among them, x represents the feature vector of the corresponding target, k t and k t+1 Respectively represent the number of targets in the frame image; The steps of enhancing the cross-frame expression capability of features through forward matching and reverse matching are as follows The matrix M is divided into four sub-matrices Mt,t, Mt+1,t+1, Mt,t and Mt+1,t+1; The Mt,t and Mt+1,t+1 represent the similarity between the objects in the t frame and the t+1 frame respectively; the Mt,t+1 and Mt+1,t represent the similarity between the objects in the frames t and t+1; SSCI uses the Hungarian algorithm in Mt,t+1 as a forward match from the target in the tth frame to the target in the t+1th frame to obtain matching pairs of the same objects in adjacent frames; Loss function L cycle Acts on M t+1,t For the elements in , use the forward match diagonal elements as the reverse match.

2. The unsupervised tracking model training method based on contrast loss according to claim 1, characterized in that: The calculation basis of the Self-Supervised Contrastive ID module is as follows: The targets in the same frame must be different; Targets in adjacent frames can obtain matching pairs with higher accuracy based on embedded features.

3. The unsupervised tracking model training method based on contrast loss according to claim 1, characterized in that: The MOTChal lenge includes MOT17 and MOT20; The MOT17 dataset includes a training set and a test set. The training set contains 5316 frames of images from 7 videos, and the test set also contains 7 videos and a total of 5919 frames. The MOT20 dataset includes a training set and a test set. The training set occupies 4 videos and 8931 frames of images, and the test set occupies 4 videos and 4479 frames of images.

4. The unsupervised tracking model training method based on contrast loss according to claim 3, characterized in that: The ratio of the training set to the test set in MOT17 is 5:5.