A space-time self-learning target tracking method based on an anti-occlusion mechanism

Abstract

The application provides a kind of space-time self-learning target tracking method based on anti-occlusion mechanism, comprising: obtaining image initial sample set;From image initial sample set, adopt cyclic shift, then utilize cutting matrix to extract background block as real negative sample;According to the difference of adjacent frame response map, design local response change matrix to perceive the change of spatial appearance information, and design adaptive spatial regularization term;According to the positive sample in historical tracking result, construct multimodal template pool to evaluate the confidence of best candidate sample, if best candidate sample is judged as positive sample, then according to local response change matrix, accelerate the learning of filter;Through searching the maximum value of response map to locate tracking object, realize target positioning;The method of the application introduces background sample as real negative sample to participate in the learning of filter, and constructs multimodal template pool with hard positive sample in historical tracking result to evaluate the confidence of best candidate sample, and has good tracking performance and high reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target tracking, in particular to a space-time self-learning target tracking method based on an anti-occlusion mechanism. BACKGROUND

[0002] Visual object tracking is an important research direction of computer vision, and its task nature is to simulate the observation of human eyes and the decision-making behavior of human brains. In the target tracking task, the computer is first provided with the initial position and target size of the first frame of the target, and then the computer is required to identify and track the position, size and other states of the target in the subsequent frames. Target tracking technology has been widely used in intelligent video monitoring, unmanned driving, modern military, human-computer interaction, traffic management system, three-dimensional reconstruction, augmented reality and other fields.

[0003] There are generative and discriminative methods for target tracking. In the generative model, the tracking algorithm generates a large number of random samples based on the geometric state of the last frame target frame, and the sample most similar to the target template is taken as the tracking result, while the discriminative model is to train a classifier to distinguish between target and background, and the sample with the highest response value is taken as the tracking result.

[0004] Since the discriminative correlation filter tracking method based on the minimum output square sum criterion was proposed, the discriminative correlation filter has attracted extensive attention in the academic and industrial communities due to its excellent computational efficiency and tracking effect. The kernel detection technology and kernel method were introduced into the DCF framework, and the tracking algorithm based on the circulant structure of tracking-by-detection with kernels (CSK) was proposed. The kernel correlation filter (KCF) method based on the histogram of oriented gradients (HOG) feature was proposed, and the tracking effect was significantly improved. Although the KCF achieves good tracking performance, it is preset with a fixed target frame size, and cannot effectively track when the target size changes. Some scholars proposed the Staple algorithm, which fuses color information and HOG features. In addition, some scholars visually fuse the hand-crafted features and deep features of samples to accurately describe the appearance representation of the target and the background. However, the deep network used for target tracking has a real-time problem, and further research is needed to reasonably simplify and improve the deep network to improve the tracking performance. Scale estimation is an important part of the target tracking framework. For example, the local-global correlation filter (LGCF) adaptively estimates the scale of the object by using the distance of the center point of the local block, and simultaneously solves the problem of local occlusion. The scale adaptive multi-feature filter (SAMF) performs multi-scale search on the input sample to form a two-dimensional scale sample pool, and then filters the samples of the scale sample pool one by one by using the correlation filter, and takes the scale corresponding to the maximum response as the optimal scale, thereby solving the problem that the KCF cannot adaptively change with the target size change. The discriminative scale space track (DSST) constructs an image pyramid with 33 scales, and designs a multi-channel one-dimensional scale filter to improve the scale search efficiency.

[0005] Although the related filtering method has achieved great success, there are certain limitations, including: (1) the traditional DCF method uses cyclic shift to obtain negative samples, so it lacks negative samples from real scenes, which greatly reduces the robustness of the tracker to cluttered backgrounds; (2) the traditional DCF method is prone to failure in occlusion scenes. This is because the DCF framework determines the target by the position of the maximum response and directly trains the filter with the sample without further checking the confidence of the sample, which leads to the misdirection of the occluder to the tracker, and finally causes tracking failure; (3) the traditional DCF method has poor adaptive learning ability with changes in space-time information, and fails to adaptively adjust the update speed of the filter according to the space-time changes of the appearance. For example, the spatially regularized discriminative correlation filter (SRDCF) tracker constrains the spatial distribution of the filter with a fixed spatial regularization term. Similarly, the spatial-temporal regularized correlation filter (STRCF) cannot adaptively adjust the spatial regularization constraint parameter. SUMMARY

[0006] The main purpose of the present application is to overcome the above-mentioned defects in the prior art, and to provide a space-time self-learning target tracking method based on an anti-occlusion mechanism. The background sample is introduced as a real negative sample to participate in the learning of the filter, and the hard positive samples in the historical tracking results are used to construct a multi-modal template pool to evaluate the confidence of the best candidate sample. The tracking performance is good, and the reliability is high.

[0007] The technical scheme of the present application comprises:

[0008] A space-time self-learning target tracking method based on an anti-occlusion mechanism comprises:

[0009] Obtaining an initial sample set of images;

[0010] Cyclic shift is used from the initial sample set of images, and a cropping matrix is used to extract background blocks as real negative samples;

[0011] A local response change matrix is designed according to the difference between the response maps of adjacent frames to perceive the change of spatial appearance information, and an adaptive spatial regularization term is designed;

[0012] A multi-modal template pool is constructed according to the positive samples in the historical tracking results to evaluate the confidence of the best candidate sample. If the best candidate sample is determined to be a positive sample, the learning of the filter is accelerated according to the local response change matrix;

[0013] The maximum value of the search response map is located to locate the tracking object, and the target positioning is realized.

[0014] Specifically, the objective function of the method is:

[0015]

[0016] Wherein, the middle represents the dth channel feature extracted in the tth frame of training sample, D is the number of channels, vec and mat represent matrix vectorization operator and vector matrix operator respectively; represents two-dimensional correlation operator; y is the output correlation, and the training sample has large space support; is a correlation filter; And λ (t) respectively represent the time regularization adaptive factor and the optimized time regularization balance parameter, represents the spatial regularization adaptive factor.

[0017] Specifically, the spatial regularization adaptive factor is specifically:

[0018] The local response change vector m is introduced into the spatial regularization adaptive factor ;

[0019]

[0020] Wherein, δ is the weight of adjusting the local response change; P is a binary matrix, P T is the transpose of P, and w is the original inherited parameter;

[0021] The jth element m j of the local response change vector m is defined as:

[0022]

[0023] Wherein, [ψΔ] represents a shift operator to make the peak values of two response maps r (t) and r (t-1) coincide; r j represents the jth element in the response map r.

[0024] Specifically, after the multi-modal template pool is constructed, the method further comprises:

[0025] Filling the multi-modal template pool with the first frame of patches, that is: t n =p (1) (n=1, 2, …, N), wherein p (1) represents the column vector drawn by the first frame of target patches, t n represents the nth column vector of the multi-modal target pool T; starting from the second frame, assuming that the patch of the optimal sample obtained by the correlation response is b, the HOG features of T(:, n) and b are extracted, as shown in the following formula:

[0026]

[0027] Where HOG represents the directional gradient histogram extraction operator;

[0028]

[0029] Define reference parameters

[0030]

[0031] Where ζ and υ represent hyperparameters, and τ is a threshold value ranging from [0,1]; when If the value is greater than the set threshold, it indicates that the target is not occluded, and e will be set accordingly. b Update the target template pool and discard the face with the lowest similarity to b among the 2nd to Nth templates in the template pool; when When the value is less than or equal to the set threshold, it indicates that the target is occluded and the response map is distorted, so the correlation filter stops learning.

[0032] Specifically, the objective function is solved as follows:

[0033] Introduce an auxiliary variable in It is of scale The discrete Fourier transform matrix, Represent the Kronecker integration operator; transform the objective function into a frequency domain equation:

[0034]

[0035] in

[0036] The optimal solution to the above equation is obtained by using the alternating direction multiplier method. The augmented Lagrangian form of the frequency domain equation is as follows:

[0037]

[0038] in, Let x be the spectrum of signal x. Let γ be the frequency domain variable of the Lagrange multiplier, and let γ represent the quadratic penalty parameter; let Let be the time regularization parameter; using the augmented Lagrangian form of the frequency domain equations using the completing the square method, it can be rearranged as follows:

[0039]

[0040] The variables can be solved iteratively using the alternating direction multiplier method; first, fix the variables... λ (t), then can be obtained by

[0041]

[0042] where i represents the iteration number, i0, and {λ (t)} (0) = λ (t-1) , The above equation can be simplified by sampling in all channels of each pixel:

[0043]

[0044] where represents the jth column of extracted as a column vector; after derivation using the Sherman-Morrison formula, we have

[0045]

[0046] where is the identity matrix,

[0047] Fix Optimize

[0048]

[0049] Let then we have

[0050]

[0051] Solve for

[0052]

[0053] The division sign in the above equation represents point-wise division operation;

[0054] The optimal solution of (t) can be expressed as

[0055]

[0056] The Lagrange multiplier can be solved by

[0057]

[0058] where The form of (i+1) = min(γmax , β, γ (i) ), γ initial value is 1, β = 10, γ max = 10000;

[0059] After calculating the optimal filter and the time regularization parameter, the correlation filter response calculation for the next frame is used.

[0060] From the above description of the present application, compared with the prior art, the present application has the following beneficial effects:

[0061] The present application provides a kind of space-time self-learning target tracking method based on anti-occlusion mechanism, comprising: obtaining image initial sample set;From image initial sample set, using cyclic shift, then using cutting matrix extracts background block as real negative sample;According to the difference of adjacent frame response map, design local response change matrix to perceive the change of spatial appearance information, and design adaptive spatial regularization term;According to the positive sample in historical tracking result, construct multimodal template pool to evaluate the confidence of best candidate sample, if best candidate sample is judged as positive sample, then according to local response change matrix, accelerate the learning of filter;By searching the maximum value of response map, the target is located, and target positioning is realized;The method of the present application introduces background sample as real negative sample to participate in the learning of filter, and constructs multimodal template pool with hard positive sample in historical tracking result to evaluate the confidence of best candidate sample, and the tracking performance is good, and the reliability is high. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 It is the comparative experiment effect diagram of the present application embodiment provided with or without spatial regularization, wherein (a) is the 10th frame, (b) is the 27th frame, (c) is the 78th frame, and (d) is the 102nd frame;

[0063] Figure 2 It is the comparative experiment effect diagram of the present application embodiment provided with or without time regularization, wherein (a) is the 15th frame, (b) is the 26th frame, (c) is the 47th frame, and (d) is the 68th frame;

[0064] Figure 3 It is the comparative experiment effect diagram of the present application embodiment provided with or without anti-occlusion module, wherein (a) is the 25th frame, (b) is the 70th frame, (c) is the 80th frame, and (d) is the 120th frame;

[0065] The present application is further described in detail below in combination with the drawings and specific embodiments. DETAILED DESCRIPTION

[0066] The application provides a space-time self-learning target tracking method based on an anti-occlusion mechanism.

[0067] 1. Correlation filter tracking under convolutional visual angle

[0068] The expression of the correlation filter energy functional is as follows:

[0069]

[0070] Wherein represents the dth channel feature extracted from the tth frame of training sample (the matrix form of the feature and T=HxW); represents the dth channel filter of the target; represents the set ideal response, which is generally set as a Gaussian function;If the matrix Z=X*Y, then Because ( the matrix is subjected to the anti-convolution operation along the row and column directions respectively), the formula can be rewritten as formula:

[0071]

[0072] Let be , then the formula is rewritten as:

[0073]

[0074] Wherein

[0075] The formula is written as a frequency domain form by using the convolution theorem, and then there is:

[0076]

[0077] The first-order partial derivative of the above formula with respect to is obtained, and the derivative is zero, and then there is:

[0078]

[0079] The division sign in the above formula represents a point-to-point division operator.

[0080] For a new sample z, the multi-channel frequency domain feature of the sample is extracted , and then the correlation filter response of the sample is:

[0081]

[0082] The corresponding spatial response is:

[0083]

[0084] where is the inverse Fourier transform operator, and real denotes the real part operator.

[0085] The objective function for jointly optimizing the filter and the temporal regularization term can be written as:

[0086]

[0087] where is the cropped feature, is a binary matrix used to crop K elements in the spatial sample , and the elements of this matrix are defined as S denotes the set of indices of the selected K elements in . is the transpose of the matrix , and denotes padding the vector with zeros to length T. y is the output correlation with large spatial support of the training sample. is the correlation filter with much smaller spatial size than the training sample. and λ (t) denote the temporal regularization adaptive factor and the optimized temporal regularization balancing parameter, denotes the spatial regularization adaptive factor.

[0088] 2. Spatial regularization adaptive factor learning

[0089] The local response variation reveals the confidence of each pixel in the search region of the current frame. Therefore, during the learning process, the filter with low pixel confidence should be limited. The present application realizes this by introducing the local response variation vector m into the spatial regularization parameter :

[0090]

[0091] δ is a constant used to adjust the weight of the local response variation. The parameter w inherited from the STRCF

[29] is used to alleviate the boundary effect. Due to the spatial penalty, the filter located at the pixel with significant response variation will partially avoid learning new appearance. The jth element of the local response variation vector m j is defined as:

[0092]

[0093] Where [ψΔ] represents a shift operator that makes two response graphs r (t) With r (t-1) The peak values ​​overlap to eliminate the influence of motion. j This represents the j-th element in the response graph r.

[0094] 3. Time-regularized adaptive factor learning

[0095] The values ​​of hyperparameters are automatically and adaptively determined by jointly optimizing their values ​​and filters. First, a multimodal target pool is constructed. For the first frame, since there is no historical data, the multimodal target pool is filled with the facets of the first frame, i.e.: t n =p (1) (n = 1, 2, ..., N), where p (1) t represents the column vector formed by stretching the target patch in the first frame. n Let represent the nth column vector of the multimodal target pool T. Starting from the second frame, assuming the optimal sample patch obtained from the relevant response is b, the HOG features of T(:,n) and b are extracted as follows:

[0096]

[0097] Where HOG represents the directional gradient histogram extraction operator.

[0098]

[0099] And define a reference

[0100]

[0101] Where ζ and υ represent hyperparameters, and τ is a threshold value ranging from [0,1]. When If the value is greater than the set threshold, it indicates that the target is not occluded, and e will be set accordingly. b Update the target template pool and discard the facet with the lowest similarity to b among the 2nd to Nth templates in the pool. At this point, the more drastic the change in the response map, the smaller the reference value. The time variation constraint of the correlation filter can be relaxed, accelerating filter learning in the case of large appearance changes. When the value is less than or equal to the set threshold, it indicates that the target is occluded and the response map is distorted. Therefore, the correlation filter stops learning.

[0102] 4. Solving the objective function

[0103] To optimize the objective function, this invention introduces an auxiliary variable. in ( It is of scale The discrete Fourier transform matrix, (This represents the Kronecker product operator). Converting equation (8) to the frequency domain:

[0104]

[0105] in

[0106] The optimal solution to the above equation can be obtained using the Alternating Direction Multiplier Method (ADMM). The augmented Lagrange form of the equation can be expressed as follows:

[0107]

[0108] in, Let x be the spectrum of signal x. Let be the frequency domain variable of the Lagrange multiplier, and γ represent the quadratic penalty parameter. Let Using the completing the square method, the expression can be rearranged as follows:

[0109]

[0110] Then, the ADMM iteration can be used to solve for each variable. First, fix... λ (t) , but It can be obtained using the following formula:

[0111]

[0112] Where i represents the iteration number, and the initial value of i is 0, and {λ (t)} (0) =λ (t-1) ,

[0113] Due to the complexity of the formula, this invention samples all channels of each pixel. To simplify the above formula:

[0114]

[0115] in Indicates will The elements of the j-th row are extracted to form a column vector. After derivation using the Sherman-Morrison formula, we have:

[0116]

[0117] in It is the identity matrix.

[0118] fixed Optimization can be achieved through the following methods

[0119]

[0120] Let Then we have:

[0121]

[0122] Solving we get:

[0123]

[0124] The division sign in the above formula represents point-by-point division operation.

[0125] Given other variables in the equation, λ (t) The optimal solution can be expressed as:

[0126]

[0127] The Lagrange multiplier can be solved by the following formula:

[0128]

[0129] Where The form of γ (initial value is 1) is γ (i+1) = min (γ max , βγ (i) ), (β = 10, γ max = 10000).

[0130] After calculating the optimal filter And the time regularization parameter, the relevant filter response calculation for the next frame.

[0131] 5、Target positioning

[0132] The tracking object is positioned by searching the maximum value of the response map, and the response calculation is as follows:

[0133]

[0134] Where r (t) Is the response map in frame t, Represents the spectrum of the test sample extraction feature in frame t.

[0135] 6、The superiority of the method of the present application is proved by the following specific experiment:

[0136] The computer used in the experiment is configured as: Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz 2.50GHz, 16GB RAM, Windows 10 64-bit operating system. The development tool uses Matlab R2018b version, configures the OTB platform and downloads the OTB100 video sequence set, and tests the tracking method on this basis.

[0137] For the hyperparameters of STSL, the present application sets δ = 0.2, ν = 2 × 10 -5 , ζ = 13, the threshold of φ is 3000, and the ADMM iteration is set to 4.

[0138] (1) Ablation experiment:

[0139] To detect the effectiveness of the method of the present application, this section compares the STSL algorithm with the STSL algorithm with the corresponding module turned off. The red tracking box is the STSL algorithm, and the green tracking box is the STSL algorithm with the corresponding module turned off.

[0140] (1.1) Comparison experiment with or without spatial regularization

[0141] To detect the influence of adding spatial regularization on the tracking results in the algorithm, this section performs experiments on the STSL algorithm with spatial regularization and the algorithm without spatial regularization (referred to as STSL without SR), and explains the tracking results. As shown in Figure 1 (a), at the 10th frame, the red tracking box representing the STSL algorithm with spatial regularization and the green tracking box representing the STSL without SR algorithm without spatial regularization can both achieve good tracking of the target. As shown in Figs. 1(b) and (c), after the 25th frame, the target enters the shadow and is similar to the background, and the green tracking box cannot distinguish the background from the target, resulting in tracking failure, but the red tracking box can still accurately track the target. As shown in Figure 1 (d), at the 102nd frame, the green tracking box is always unable to re-track the correct target due to the influence of the previous occlusion, while the red tracking box is not disturbed and still accurately and stably tracks the target.

[0142] Experiments show that when the sample is contaminated, the STSL algorithm with spatial regularization can stop the learning of the correlation filter, so that the DCF focuses on the target object, and avoids introducing a large number of contaminated samples to cause tracking failure.

[0143] (1.2) Comparison experiment with or without temporal regularization

[0144] To detect the influence of adding time regularization in the algorithm on the tracking results, this section performs experiments on the STSL algorithm with space-time regularization and the algorithm without time regularization (referred to as STSL without TR) and explains the tracking results. As shown in Figure 2 (a), the target does not have a large and rapid movement at the 15th frame, and the red tracking box representing the STSL algorithm with time regularization and the green tracking box representing the STSL without TR algorithm without time regularization can both achieve good tracking of the target. As shown in Figure 2 (b), (c), and (d), after the 25th frame, the target has a large and rapid jumping movement, the green tracking box has a large tracking drift, and the red tracking box can always accurately track the target.

[0145] Experiments show that, in the case of rapid movement, the STSL algorithm with time regularization can adjust the learning limit of the filter, quickly update the filter, reduce tracking drift, and avoid tracking failure caused by rapid movement of the target

[0146] (1.3) Comparison experiment with / without anti-occlusion module

[0147] To detect the influence of adding the anti-occlusion module in the algorithm on the tracking results, this section performs experiments on the STSL algorithm with the anti-occlusion module and the STSL without anti-occlusion algorithm without the anti-occlusion module and explains the tracking results.

[0148] As shown in Figure 3 (a), when the target is not occluded at the 25th frame, the red tracking box representing the STSL algorithm with the anti-occlusion module and the green tracking box representing the STSL without anti-occlusion algorithm without the anti-occlusion module can both achieve good tracking of the target. As shown in Figure 3 (b) and (c), when the target is strongly occluded at the 70th frame and the 80th frame, the green tracking box cannot capture the target, and the red tracking box can still accurately track the target. As shown in Figure 3 (d), at the 120th frame, the green tracking box is always unable to re-track the correct target due to the influence of the previous occlusion, resulting in tracking failure, while the red tracking box is not disturbed by the occlusion and still accurately and stably tracks the target. Experiments show that, in the case of strong occlusion, the STSL algorithm with the anti-occlusion module can effectively reduce the influence of occlusion on target tracking, reduce tracking drift, and avoid tracking failure caused by strong occlusion of the target.

[0149] (2) Parameter sensitivity analysis

[0150] The position error is minimum and the overlap rate is maximum when τ = 0.6, and the best tracking effect is achieved, but the tracking difference between τ = 0.6 and τ = 0.7 is not obvious, and it can be seen through comparison that if the value of τ is too large or too small, the tracking effect will be poor. When τ = 0.8 and τ = 0.9, the tracking effect of the algorithm is the same, which shows that when τ is not properly valued and the tracking effect is reduced to a certain extent, the tracking effect does not change with the change of τ value;

[0151] When the target is stably running and there is no occlusion, the size of the threshold τ does not affect the tracking effect. When the target reappears after being completely occluded at about frame 120, the algorithm loses the target when τ = 0.5, which shows that if the threshold value is too low, the template will be learned and updated even if it is occluded, and the contaminated sample will be recognized as the target, resulting in tracking failure. Between frames 120 and 190, an object similar to the target appears, and when τ = 0.8 and τ = 0.9, the algorithm will mistakenly think it is the target and fail to update the target template in time, resulting in tracking failure. When the target re-passes the tracking frame of the τ = 0.8 and τ = 0.9 algorithm that has been lost, the algorithm will re-track the target, but the algorithm with τ = 0.5 cannot re-identify the target. When another similar object passes through the target, the τ = 0.8 and τ = 0.9 algorithm again misjudges the target and fails to track. As can be seen from the above, the threshold τ of the anti-occlusion part cannot be too high or too low, and a too high threshold value makes the algorithm unable to identify the appearance update of the target in time, and a too low threshold value will introduce contaminated samples when updating the template, and a suitable threshold value can make the algorithm stably solve the occlusion problem for a long time and continuously track the target.

[0152] (3) Qualitative analysis

[0153] This section will outline the tracking performance comparison of the algorithm STSL proposed in the present application and other 9 advanced algorithms (AutoTrack, BACF, ARCF_H, deepSRDCF, SRDCF, Struck, STAPLE_CA, DSST, KCF) in different scenes of the OTB100 dataset.

[0154] The following is a qualitative analysis of several typical tracking challenge factors:

[0155] (1)Fast motion. In the basketball video sequence, a group of basketball players are playing, the target fast motion and a certain motion blur with rotation, in addition to DSST and STSL always stable tracking target, other algorithm tracking has a certain deviation, ARCF H, BACF, SRDCF, STRUCK algorithm eventually failed to track. The target in the figure skating constantly fast motion, DSST algorithm deviation is larger, other algorithm tracking target is stable. The target in the fast motion will change the apparent state for a short time, but STSL can still follow, this is due to the use of local-global response changes. Local spatial change vector can make DCF focus on the current object, and global time change vector can adjust the filter learning limit and quickly update the learning related filter.

[0156] (2) Occlusion. Between the video frames 110 to 120 frames, the little girl is completely occluded, in addition to the algorithm STSL of the application can always accurately track the target, the rest of the algorithm all misjudgment tracking object, resulting in tracking loss. After video frame 390, deepSRDCF algorithm can re-identify the tracking target. The target is strongly occluded by the telegraph pole from 49 to 66 frames, BACF, ARCF H, Struck, STAPLE CA, KCF cannot effectively track the target. After 114 frames, ARCF H identifies the tracking target again. The STSL algorithm can always stably track the target. The above illustrates that the multi-modal template pool strategy adopted by the algorithm of the application can effectively handle the partial or complete occlusion problem.

[0157] (3) Background clutter. There is background clutter in the video sequence. Among them, the target person wears very similar color with others and background, the target and background are difficult to distinguish and accompanied by light change. Struck algorithm produces a large tracking drift, AutoTrack, SRDCF, STAPLE_CA, KCF algorithm tracking fails. The STSL algorithm proposed in the application can still accurately and stably track the target. This is because the local response spatial vector used in the application can make DCF focus on learning objects, reduce the influence of background clutter on tracking performance, and improve the accuracy of the algorithm.

[0158] Rotation. The video sequences basketball, Girl2, skater exist in-plane and out-of-plane rotation. At the 30th frame, the target turns around, the DSST, ARCF_H, KCF algorithms all have different degrees of tracking drift, and other algorithms are basically not affected by this tracking accuracy. Between 340-600 frames, the target turns from the side to the back, the BACF, SRDCF, STAPLE-CA, ARCF_H, AutoTrack, KCF and SRDCF algorithms have lost track before the target starts to rotate, at the time when the target rotates 90 degrees, the DSST algorithm has a certain tracking drift, the SRDCF algorithm re-identifies the target, and the STSL algorithm proposed in the application always accurately tracks the target. In the skater video sequence, the target rotates 360 degrees throughout the process, the DSST algorithm loses track, the STAPLE_CA and SRDCF algorithms have slight drift, and the remaining algorithms can all continue to track stably. In summary, the space-time adaptive learning framework adopted by the STSL algorithm proposed in the application has reliability.

[0159] (4) Quantitative analysis:

[0160] The accuracy and success rate of the 10 algorithms in the OTB100 dataset, the accuracy of STSL ranks first, reaching 85.5%, and the success rate of STSL ranks first, reaching 77.6%. This shows that the proposed method has good tracking performance.

[0161] Table 1 gives the average tracking overlap rate of each tracking algorithm in different video sequences, where the larger the value, the better the tracking performance, bold, underline and wavy line respectively represent the top 3 algorithms, and the average tracking overlap rate of the STSL algorithm proposed in the application in the BlurCar2, David3, Deer, Girl2, KiteSurf, Jogging-1, Jogging-2, Woman, Faceocc2 sequences is 0.90, 0.77, 0.80, 0.72, 0.80, 0.76, 0.70, 0.74, and the total average value reaches 0.77, ranking first among the 10 algorithms. This nine video sequences can achieve better tracking of the target under the appearance change of the target.

[0162] Table 2 shows the average center point error of each tracking algorithm in different video sequences, wherein the smaller the value is, the smaller the error with the real position of the target is, and the bold, underlined and wavy line respectively represent the three algorithms with the highest tracking accuracy. The center point error of the algorithm STSL of the present application in the video sequences of BlurCar2, David3, Deer, Girl2, KiteSurf, Jogging-1, Jogging-2, Woman, Faceocc2 is respectively 3.29, 4.70, 4.03, 7.39, 4.71, 4.12, 2.71, 10.14 pixels, and the total average pixel error is 5.14 pixels, ranking first among the 10 algorithms. Among them, the Girl2 sequence can still achieve good tracking of the target under the influence of multiple factors such as scale change, occlusion, deformation, motion blur, out-of-plane rotation, and is also the best in tracking effect among the compared algorithms.

[0163] Table 1 average tracking overlap rate of each tracking algorithm in some videos

[0164]

[0165] Table 2 average center point error of each tracking algorithm in some videos

[0166]

[0167]

[0168] In summary, the traditional DCF-based tracker improves filter learning by introducing pre-defined regularization, but it takes a lot of effort to pre-define these parameters and cannot adapt to various situations. In this work, the anti-occlusion mechanism based on a multi-modal template pool helps the relevant filter to utilize the local-global information hidden in the response map to automatically adjust the hyperparameters in real time. The algorithm has the following advantages:

[0169] (1) Introducing automatic spatial regularization to utilize local response variation. Local response variation indicates local credibility, thereby limiting the learning of local correlation filters.

[0170] (2) Introducing automatic temporal regularization to utilize global response variation. Global variation can control the amount of learning of the correlation filter from the entire object.

[0171] (3) Introducing a historical multi-modal template pool. The historical multi-modal template uses a threshold setting method to filter samples, thereby avoiding sample pollution and improving the reliability of tracking samples.

[0172] Comprehensive experiments show that the method has strong performance and can be applied in different algorithm frameworks.

[0173] The above merely illustrates the specific embodiments of the present application, but the design concept of the present application is not limited thereto, and any non-essential modification of the present application by using the concept shall be deemed as an infringement of the protection scope of the present application.

Claims

1. A spatiotemporal self-learning target tracking method based on an anti-occlusion mechanism, characterized in that, include: Obtain the initial image sample set; The image is initially sampled by cyclic shifting, and then the background block is extracted as the true negative sample using the cropping matrix. Based on the differences in response maps of adjacent frames, a local response change matrix is ​​designed to perceive changes in spatial appearance information, and an adaptive spatial regularization term is designed. A multimodal template pool is constructed based on positive samples from historical tracking results to evaluate the confidence of the best candidate sample. If the best candidate sample is determined to be a positive sample, the learning of the filter is accelerated based on the local response change matrix. The target is located by finding the maximum value in the response graph. The objective function of the method is: Among them, China Indicates the first The first frame extracted from the training samples Channel characteristics, where D is the number of channels. and These represent the matrix columnization operator and the vector matrixization operator, respectively; Represents a two-dimensional correlation operator; It is the correlation of the output, with training samples supported by a large space; It is a correlation filter; and These represent the adaptive time regularization factor and the optimized time regularization balance parameter, respectively. This represents the adaptive factor for space regularization.

2. The spatiotemporal self-learning target tracking method based on an anti-occlusion mechanism according to claim 1, characterized in that, The spatial regularization adaptive factor is specifically: Local response change vector Introducing a spatial regularization adaptive factor middle; in, To adjust the weights of local response changes; It is a binary matrix. yes transpose, These are the original inherited parameters; Local response change vector The element Defined as: in, This indicates that a shift operator causes two response graphs. and The peak values ​​overlap; Representation of response graph The first in Each element.

3. The spatiotemporal self-learning target tracking method based on an anti-occlusion mechanism according to claim 1, characterized in that, The process of constructing the multimodal template pool also includes: Fill the multimodal template pool with the facets of the first frame, that is: ,in This represents the column vector formed by stretching the target patch in the first frame. Represents a multimodal target pool The column vectors; starting from the second frame, let the patch of the optimal sample obtained from the relevant response be . ,extract and The HOG features are shown in the following formula: in This represents the directional gradient histogram extraction operator; Define reference parameters : in and Indicates hyperparameters, It is a value range of The threshold; when When the value exceeds the set threshold, it indicates that the target is not occluded, and then... Update the target template pool and remove templates numbered 2 through 3 from the pool. In the template The face with the lowest similarity; when When the value is less than or equal to the set threshold, it indicates that the target is occluded and the response map is distorted, so the correlation filter stops learning.

4. The spatiotemporal self-learning target tracking method based on an anti-occlusion mechanism according to claim 3, characterized in that, The objective function is solved as follows: Introduce an auxiliary variable ,in , It is of scale The discrete Fourier transform matrix, Represent the Kronecker integration operator; transform the objective function into a frequency domain equation: in ; The optimal solution to the above equation is obtained by using the alternating direction multiplier method. The augmented Lagrangian form of the frequency domain equation is as follows: in, For signal The spectrum, These are the frequency domain variables of the Lagrange multipliers. Indicates the quadratic penalty parameter; let , Let be the time regularization parameter; using the augmented Lagrangian form of the frequency domain equations using the completing the square method, it can be rearranged as follows: The variables can be solved iteratively using the alternating direction multiplier method; first, fix the variables... , , ,but It can be obtained using the following formula: in Indicates the number of iterations. The initial value is 0, and , , ; By sampling all channels of each pixel To simplify the above formula: in Indicates will The The row elements are extracted to form a column vector; after derivation using the Sherman-Morison formula, we have: in It is the identity matrix. , ; fixed , Optimize using the following methods : make Then we have: Solving for: The division sign in the above formula indicates point-by-point division; The optimal solution can be expressed as: Lagrange multipliers can be solved using the following formula: in ; The form is , The initial value is 1. =10, =10000; Calculating the optimal filter After adding the time regularization parameter, it is used for calculating the correlation filter response of the next frame.

Images