Self-supervised tracking loss optimization method based on multi-cycle contrast enhancement consistency

Abstract

The application discloses a kind of self-supervised tracking loss optimization methods based on multi-cycle contrast enhancement consistency, belong to target tracking technical field.The method is: the video frame in training data set is successively executed forward tracking and reverse tracking, and data enhancement operation is carried out to template frame in last step of forward tracking and reverse tracking;According to the real target label of first frame and the pseudo label generated by its reverse tracking, cycle consistency loss is constructed;According to the difference between enhanced video frame and corresponding original video frame in executing forward tracking and reverse tracking, contrast enhancement loss is constructed;Self-supervised tracking model is optimized by combining cycle consistency loss and contrast enhancement consistency loss.The application simulates target appearance change using data enhancement technology, constructs contrast enhancement consistency loss in video frame to optimize tracking model, and realizes high-performance tracking.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target tracking, and particularly relates to a self-supervised tracking loss optimization method based on multi-cycle contrast enhancement consistency. BACKGROUND

[0002] With the rapid development of artificial intelligence and the Internet of Things, deep learning technology is widely explored and applied in all aspects of life. Among them, the visual data yield cannot be ignored, and images and videos are the premise of the development of artificial intelligence, big data and other technologies, and also one of the important factors to promote the efficient and intelligent development of society. In order to efficiently process massive visual data and reasonably utilize various visual information, computer vision technology emerges as the times require. Target tracking is one of the basic and important research hotspots in the field of computer vision, and has high landing value. The task definition of target tracking is to give the initial position of the tracking object in the first frame in a video sequence, and accurately locate the target in the subsequent video frames. Today, the target tracking method has been widely applied in many fields, such as video monitoring, intelligent human-computer interaction, automatic driving, visual navigation, etc.

[0003] In recent years, since the training of tracking models based on supervised learning needs to rely on large-scale manually annotated data sets, but the data annotation work often needs to consume huge manpower and financial resources. Therefore, the method based on self-supervised learning is introduced into the field of target tracking and shows great advantages, which can break through the limitations of limited annotated data sets.

[0004] The current target tracking method based on self-supervised learning adopts simple random sampling, which cannot extract rich target motion information, and takes the center region as the tracking object, so that the model is not robust to scenes with large appearance changes. The model trained by this kind of method has poor robustness and generalization ability. SUMMARY

[0005] The technical problem solved: In view of the above technical problems, the application provides a self-supervised tracking loss optimization method based on multi-cycle contrast enhancement consistency, which uses data enhancement technology to simulate target appearance changes, constructs a contrast enhancement consistency loss in the video frame to optimize the tracking model, and realizes high-performance tracking.

[0006] Technical scheme: based on the multi-cycle contrast enhancement consistency self-supervised tracking loss optimization method, the video frames in the training data set are sequentially executed forward tracking and reverse tracking, and the template frames in the last step of forward tracking and reverse tracking are subjected to data enhancement operation; according to the true target label of the first frame and the pseudo label generated by reverse tracking, a cycle consistency loss is constructed; according to the difference between the enhanced video frames and the corresponding original video frames in executing forward tracking and reverse tracking, a contrast enhancement loss is constructed; the self-supervised tracking model is optimized by combining the cycle consistency loss and the contrast enhancement consistency loss.

[0007] Preferably, the data enhancement operation includes Gaussian blur, Gaussian noise, brightness transformation and color jitter.

[0008] Preferably, the multi-cycle contrast enhancement consistency self-supervised tracking loss optimization method comprises the following steps:

[0009] S1. The sample data in the training sample pair is sequentially input into the network to execute forward tracking, and the corresponding forward tracking response graph is generated;

[0010] S2. The tracking response graph generated by the last frame sample data in the training sample pair is used as the initial value of reverse tracking, and the sample data in the training sample pair is sequentially executed in reverse order to generate the corresponding reverse tracking response graph;

[0011] S3. According to the true target label existing on the first frame sample data in the training sample pair and the corresponding reverse tracking response graph, a cycle consistency loss l1 is constructed;

[0012] S4. Data enhancement operation is performed on the template frames used in the last step of forward tracking and reverse tracking in the training sample pair to generate corresponding enhanced images;

[0013] S5. The corresponding enhanced branch response graph is generated on the sample data in the last step of executing forward tracking and reverse tracking, respectively;

[0014] S6. According to the enhanced branch response graph corresponding to the same frame in forward tracking and the forward tracking response graph, a contrast enhancement loss l2 is constructed; according to the enhanced branch response graph corresponding to the same frame in reverse tracking and the reverse tracking response graph, a contrast enhancement loss l3 is constructed;

[0015] S7. According to the enhanced branch response graph corresponding to the sample data in the last step of reverse tracking and the true target label existing on the first frame sample data, a contrast enhancement loss l4 is constructed;

[0016] S8. Jointly construct the total loss l 总 , optimize the self-supervised tracking model.

[0017] Further, the sample data is: video frames extracted from a video sequence at a fixed frame interval, and the training sample pair is: a total set of sample data.

[0018] Further, the network is a twin network.

[0019] Further, the total loss l 总 The calculation formula is as follows:

[0020]

[0021] Wherein, alpha represents a predefined weight parameter.

[0022] Further, the value range of alpha is 0-1.

[0023] Beneficial effects: the present application captures valuable target feature information by constructing a fixed video frame interval, and simultaneously performs forward tracking and reverse tracking on a group of training sample pairs by inputting the training sample pairs into the network, and performs data enhancement operation on the template frame in the last step of forward tracking and reverse tracking, so that the model can adapt to the appearance deformation of the target and the scene change, and obtain rich target motion information. The robustness and generalization performance of the model are improved by constructing the cyclic consistency loss and the contrast enhancement loss, so as to improve the tracking ability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flowchart of a self-supervised tracking loss optimization method based on multi-cycle contrast enhancement consistency;

[0025] Figure 2 is a framework diagram of a self-supervised tracking loss optimization method based on multi-cycle contrast enhancement consistency. DETAILED DESCRIPTION

[0026] The present application will be further described below in combination with the drawings and specific embodiments.

[0027] Embodiment 1

[0028] As shown in Figure 1 and Figure 2 , the present application proposes a self-supervised tracking loss optimization method based on multi-cycle contrast enhancement consistency, which specifically includes the following steps:

[0029] Step one: in the training data set, each video sequence is divided into a group of 10 frames in turn, and the training video frame selection strategy is selected at a fixed interval, in each group of 10 frames, one frame is selected every two frames, which are the first frame, the fourth frame, the seventh frame and the tenth frame, and the training sample pair is denoted as {I0, I3, I6, I9};

[0030] Step two: input a set of training sample pairs into the Siamese network, and perform forward tracking on video frames I0, I3, I6, and I9 in turn to generate corresponding forward tracking response maps R(I0, I3), R(I3, I6), and R(I6, I9);

[0031] Step three: take the tracking response map R(I6, I9) generated on the video frame I9 as the initial value of reverse tracking, and perform tracking on the training sample pairs in reverse order, that is, generate corresponding reverse tracking response maps R(I9, I6), R(I6, I3), and R(I3, I0) on video frames I6, I3, and I0;

[0032] Step four: the robust tracker should obtain consistent response maps on the same frame regardless of how the video sequence changes, so define the cycle consistency loss l1 to minimize the difference between the true label in I0 and the reverse tracking response map generated by reverse tracking, that is, the pseudo label;

[0033]

[0034] Step five: use data enhancement techniques such as Gaussian blur, Gaussian noise, brightness transformation, and color jitter to enhance the template frames I6 and I3 of the last step of forward tracking and reverse tracking, and generate enhanced images Z(I6) and Z(I3), respectively;

[0035] Step six: match Z(I6) with the video frame I9 to obtain the enhanced branch response map R(Z(I6), I9), and match Z(I3) with the video frame I0 to obtain the enhanced branch response map R(Z(I3), I0);

[0036] Step seven: to make the tracking model robust enough, minimize the difference between the enhanced branch response map and the corresponding forward tracking response map or reverse tracking response map in the same frame, define the contrast enhancement loss l2 and l3 on video frames I6 and I3, respectively, and the formula is:

[0037]

[0038]

[0039] Step eight: to reasonably optimize the tracking model, constrain the enhanced branch response map R(Z(I3), I0) generated by the enhanced image Z(I3) branch of the last step of reverse tracking to be infinitely close to the target true label value Y0, define the contrast enhancement loss l4, and the formula is:

[0040]

[0041] Step nine: optimize the self-supervised tracking model by combining all the above losses, and the total loss is:

[0042]

[0043] wherein a denotes a predefined weight parameter, taking values from 0 to 1.

Claims

1. A self-supervised tracking loss optimization method based on multi-cycle comparison to enhance consistency, characterized in that, Forward and backward tracking are performed sequentially on the video frames in the training dataset, and data augmentation is performed on the template frames of the last step of forward and backward tracking. A cycle consistency loss is constructed based on the real target label of the first frame and the pseudo label generated by its backward tracking; a contrast enhancement loss is constructed based on the difference between the enhanced video frame and the corresponding original video frame during forward and backward tracking; the self-supervised tracking model is optimized by combining the cycle consistency loss and the contrast enhancement consistency loss, specifically including the following steps: S1. Input the sample data from the training sample pairs into the network sequentially to perform forward tracking and generate the corresponding forward tracking response map; S2. Use the tracking response map generated corresponding to the last frame of sample data in the training sample pair as the initial value for reverse tracking, and perform reverse tracking on the sample data in the training sample pair in reverse order to generate the corresponding reverse tracking response map. S3. Construct a cycle consistency loss based on the true target labels and their corresponding reverse tracking response maps present in the first frame of the training sample pair. l 1; S4. Perform data augmentation on the template frames used in the last step of forward and backward tracking in the training sample pairs to generate the corresponding augmented images; S5. Generate corresponding enhanced branch response diagrams on the sample data of the last step of forward tracking and backward tracking, respectively; S6. Construct the contrast enhancement loss based on the enhancement branch response map and the forward tracking response map corresponding to the same frame in the forward tracking. l 2; Construct a contrastive enhancement loss based on the enhancement branch response map and the backtracking response map corresponding to the same frame in the backtracking process. l 3; S7. Construct the contrast enhancement loss based on the enhanced branch response map corresponding to the sample data in the last step of reverse tracking and the real target labels present in the first frame sample data. l 4; S8. Combine the above losses to construct the total loss. l 总 Optimize the self-supervised tracking model.

2. The self-supervised tracking loss optimization method based on multi-cycle comparison to enhance consistency as described in claim 1, characterized in that, The data augmentation operations include Gaussian blur, Gaussian noise, brightness transformation, and color dithering.

3. The self-supervised tracking loss optimization method based on multi-cycle comparison to enhance consistency as described in claim 1, characterized in that, The sample data consists of video frames extracted from the video sequence at fixed frame intervals, and the training sample pairs consist of the total set of sample data.

4. The self-supervised tracking loss optimization method based on multi-cycle comparison to enhance consistency as described in claim 1, characterized in that, The network in question is a twin network.

5. The self-supervised tracking loss optimization method based on multi-cycle comparison to enhance consistency as described in claim 1, characterized in that, The total loss l 总 The calculation formula is as follows: ， Where α represents a predefined weight parameter.

6. The self-supervised tracking loss optimization method based on multi-cycle comparison to enhance consistency as described in claim 5, characterized in that, The value of α ranges from 0 to 1.

Images