An improved cosine window correlation filter target tracking method

Abstract

A correlation filtering target tracking method with improved cosine window is proposed. An improvement on the scale-adaptive cosine window is proposed. The initial size of the target is used to clip the cosine window peak. In addition, the scale change of the target is taken into consideration. The position of the cosine window peak clipping is dynamically adjusted through the scale change factor, thereby improving the tracking success rate of the tracking algorithm in scenarios where the target scale changes. The ADTrack algorithm is improved, and self-constraints and mutual constraints between the training target and background filters are introduced to further optimize the objective function, optimize the tracking drift and rapid template degradation of the existing tracker, and the new objective function is optimized and derived by ADMM.

Description

Technical Field

[0001] The invention belongs to the technical field of target tracking, and in particular relates to a correlation filtering target tracking method with an improved cosine window. Background Art

[0002] Image recognition and target tracking are key tasks in computer vision. The specific content is to continuously infer the target state in the video sequence. The core work is to locate the tracking target involved in all frames of the video to obtain the motion trajectory of the corresponding target and provide the activity area of ​​the tracking target for each frame. Target tracking technology has been widely promoted in the military and civilian fields, bringing a lot of convenience to people's lives. With the improvement of human living standards, the requirements for target tracking are getting higher and higher. Although target tracking related technologies are being innovated every year, there is still a need for performance optimization of related algorithms. At this stage, researchers still face many challenges.

[0003] In recent years, many researchers have proposed many methods to design a stable and accurate target tracker for various problems in target tracking. In 2017, Galoogahi et al. proposed the BACF algorithm. In order to improve the quality of samples, they proposed the key "cropping" idea, expanding the original sampling area to obtain more background information, and then performing center cropping on the circular samples, which not only expanded the number of samples but also improved the sample quality. Later, the ADTrack algorithm proposed by Li et al. in 2021 used a 0-1 mask to extract target information, and then trained the target and target background information separately, further improving the BACF algorithm.

[0004] However, ADTrack also has many shortcomings. The first is the cosine window function. The ordinary cosine window function directly uses a model with zero values ​​around the middle extreme value, ignoring the actual size of the target. If the sample is directly preprocessed, the target texture will be increased and the target source information will be polluted. In addition, the constraints on the filter in the ADTrack algorithm are not ideal, and there are problems of overfitting of the filter and rapid degradation of the filter model. Summary of the invention

[0005] The present invention is a method for tracking a target by using a correlation filter of an improved cosine window, and performs filtering training on the target and background information respectively. At present, there are still many problems in the field of target tracking, such as target scale change, target occlusion, etc. In view of the change of target scale, the present invention constructs a scale-adaptive cosine window, performs peak clipping on the cosine window, and updates it in time, and then uses two filters, target and target background, and combines the self-constraint and mutual constraint of the two to improve the correlation filter target tracking model, thereby improving the tracking performance.

[0006] The technical solution adopted by the present invention is roughly as follows:

[0007] An improved cosine window correlation filtering target tracking method comprises the following steps:

[0008] Step 1, preprocessing stage; determine whether it is in a dark scene. If it is determined to be in a dark scene, enhance the video sequence and calculate the mask m for subsequent filter training;

[0009] Step 2, feature extraction stage; obtain feature set x based on training samples g , the adaptive updating of the tracking target scale is realized through the cosine window after peak clipping, and the target feature set x is obtained based on the mask m o , and obtain the scale-adaptive cosine window model;

[0010] Step 3, training phase: using the feature set x obtained in step 2 g and x o Train two filters h for the next frame g and h o , where h g is a filter used to train background information, h o It is a filter used to train target information;

[0011] Step 4, detection phase; based on the trained filter and the extracted sample channel features, a target response map is obtained, and the target position information is determined by the maximum value of the target response map.

[0012] Furthermore, step 1 includes the following sub-steps:

[0013] Step 1-1, assuming that a color image is given Perform photometric integration on the RBG three channels of the image:

[0014]

[0015] Among them Ψ m (I(x,y)) represents the pixel value of the m-channel output image, and α R +α G +α B =1, convert the color image into a single-channel image; then perform logarithmic averaging on the brightness:

[0016]

[0017] Where δ is a constant used to prevent the appearance of log(0). Represents the logarithmic average scene brightness of the current image, and then introduces a threshold τ to determine whether the current image is in a dark scene. If it is less than the threshold, it is a dark scene, denoted as S(I);

[0018] Step 1-2: Enhance the image by using the previously acquired image brightness V(x,y,I) and the image logarithmic average brightness The global enhancement matrix of the image is expressed as:

[0019]

[0020] Where V max (I) represents the maximum value of the image brightness V(x,y,I), and then the three channels of the image are enhanced:

[0021]

[0022] Among them I e represents the enhanced image, Ψ m (I e (x,y)) represents the pixel value of the enhanced image at the position (x,y) of the m channel; from the above enhanced image, the enhanced part information of the image is obtained:

[0023] E(I)=V(I)-V(I e )

[0024] Step 1-3, obtain the mean value μ and standard deviation σ of E(I) through E(I), thereby obtaining the global mask m g :

[0025]

[0026] By clipping the matrix P to m g Cut and get the expected mask m=m g ⊙P,P∈R w×h , used to extract the target size information within the sample.

[0027] Furthermore, step 2 includes the following sub-steps:

[0028] Step 2-1, obtain a large number of training samples through cosine window preprocessing, circulant matrix, and center cropping method. Cosine window processing is to directly multiply the cosine window function on the sample. The operation of circulant matrix and center cropping is shown in the attached Figure 2 . Perform feature extraction on the acquired samples, including grayscale information, color information, gradient information, etc., to obtain the feature set x g ;

[0029] Step 2-2, use the target initial size to clip the cosine window peak, the clipping position is parameter Q∈(0,1), Q is obtained by the following formula:

[0030]

[0031] Among them, cosWin0 is the most original cosine window, W×H, W=H is the cosine window size, and w×h, w≥h is the target size;

[0032] Step 2-3, after each tracking target scale update, obtain the scale update factor S scale , update the peak clipping position Q of the cosine window again scale =Q×S scale , so that the model scale is adaptive; then use the mask m to get x o =m⊙x g , represents the simple target feature, and gets x g and x o Two feature sets; the specific model of the scale-adaptive cosine window is:

[0033]

[0034] In the above formula, cosWin0 is the most primitive cosine window, Q scale is the position of the cosine window clipping peak, S scale is the scale factor.

[0035] Further, step 3 includes the following sub-steps:

[0036] Step 3-1, the objective function of the filter is:

[0037]

[0038] Where P is the sample cropping matrix; represents the target information filter or background information filter of the cth channel; cosWin is the cosine window function proposed earlier that changes with the target scale factor and is used to preprocess the training sample data; y is the ideal Gaussian model; h t and h t-1 Represents the filters of the current frame and the previous frame, the M matrix represents the relationship between the two filters, λ is the constraint parameter of the filter regularization term, and μ is the constraint parameter of the mutual constraint term of the two filters;

[0039] Step 3-2, for the overall objective function, because k∈{g,o}, plus the final mutual constraint between the two, the objective function can be seen as the accumulation of 7 parts. The 1st, 2nd, 4th and 5th parts are conventional linear models with additional clipping matrices, the least squares plus regularization terms, the purpose of the regularization terms is to prevent the filter from overfitting; the 3rd and 6th parts are the self-constraints of the two filters, which can effectively prevent the rapid degradation of the filters; the 7th part is the mutual constraint of the two filters, which are bound to each other during training, making the discrimination ability of the two filters more powerful;

[0040] Step 3-3, solve by ADMM iterative algorithm; because cosWin is the preprocessing of samples, it can be ignored during iteration. o and M, for h g Perform ADMM iterative optimal solution; use augmented Lagrangian method and introduce relaxation variables P T is the transpose of the cropping matrix P, I N is an N×N identity matrix; the augmented Lagrangian form of the objective function is expressed as:

[0041]

[0042] in is the Lagrangian vector, γ is the penalty factor; the ADMM method is used to iteratively transform the above formula into the following three sub-problems:

[0043]

[0044] for Subproblem, find h through the first-order derivative g The closed-form solution of is:

[0045]

[0046] For ξ * Sub-problem, it needs to be converted to the frequency domain for further calculation:

[0047]

[0048] The above formula is decomposed into T sub-problems, T = 42 represents the dimension of the feature, and each sub-problem is have to:

[0049]

[0050] Derivative of the above formula:

[0051]

[0052] Use the Sherman-Morrison equation to optimize the inverse matrix and get:

[0053]

[0054] in is a scalar;

[0055] h g and h o The iteration process is the same as that of , and the iteration of the M matrix is:

[0056] Furthermore, in step 4, the target response graph It is expressed as:

[0057]

[0058] in, The corresponding quantity in the Fourier domain representing the given data, represents the response diagram after inverse Fourier transform, D represents the dimension of the filter, and are the two filters of the fth frame, represents the c-th channel feature of the search area sample extracted from the f+1 frame, It is masked ρ is a control and Weight parameter for the two resulting response plots.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] (1) A scale-adaptive cosine window was invented. The shortcomings of the existing cosine window were analyzed, and the peak of the cosine window was clipped using the initial size of the target. In addition, the scale change of the target was taken into account, and the position of the cosine window peak clipping was dynamically adjusted through the scale change factor, which improved the tracking success rate of the tracking algorithm in scenarios where the target scale changes.

[0061] (2) The ADTrack algorithm was improved by introducing self-constraints and mutual constraints between the training target and background filters, further optimizing the objective function, optimizing the tracking drift and rapid template degradation of the existing tracker, and performing ADMM optimization derivation on the new objective function. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 The scale-adaptive cosine window in the embodiment of the present invention is constructed.

[0063] Figure 2 Demonstration of circulant and clipping matrices.

[0064] Figure 3 It is the overall framework of the algorithm model in the embodiment of the present invention.

[0065] Figure 4 This is the tracking result of the tiger1 video sequence of the TC128 dataset in an embodiment of the present invention.

[0066] Figure 5 This is a schematic diagram of the total accuracy results of comparing algorithms in recent years and the method of the present invention based on the TC128 data set in an embodiment of the present invention.

[0067] Figure 6 This is a schematic diagram of the total success rate results of comparing algorithms in recent years and the method of the present invention based on the TC128 data set in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0069] The present invention is a method for tracking a target by using a correlation filter of an improved cosine window, and performs filtering training on the target and background information respectively. At present, there are still many problems in the field of target tracking, such as target scale change, target occlusion, etc. In view of the change of target scale, the present invention constructs a scale-adaptive cosine window, performs peak clipping on the cosine window, and updates it in time, and then uses two filters, target and target background, and combines the self-constraint and mutual constraint of the two to improve the correlation filter target tracking model, thereby improving the tracking performance.

[0070] The main contents of the technical solution of the present invention are as follows: In order to solve the problem that the cosine window in the correlation filter target tracking algorithm will increase the texture of the sample and pollute the sample, a scale-adaptive cosine window function is constructed. The construction method is to first clip the peak of the cosine window model based on the benchmark size of the target, and then use the scale factor S in the scale detection module of the DSST algorithm to detect the scale factor S. scale , timely update the peak clipping cosine window model to make it scale adaptive, the specific effect is shown in the attached Figure 1 The overall tracking model of the present invention is based on the ADTrack algorithm. The specific effects are shown in the attached Figure 3 In addition to the above-mentioned scale-adaptive cosine window, the self-constraint of the filter is optimized on the ADTrack basic model. and mutual constraints And re-derive the ADMM iteration of the model.

[0071] The specific steps of the present invention are as follows:

[0072] Step 1: In the preprocessing stage, the threshold τ is used to determine whether it is in a dark scene. The optimal value of τ is obtained by adjusting the parameters. If it is judged to be in a dark scene, the video sequence is enhanced to improve the accuracy and robustness of target tracking in the dark state, and the mask m is obtained for the training of the subsequent two filters.

[0073] Assume that a color image is given Perform photometric integration on the RBG three channels of the image:

[0074]

[0075] Among them Ψ m (I(x,y)) represents the pixel value of the m-channel output image, and αR +α G +α B =1, which can be understood as converting a color image into a single-channel image. Then perform logarithmic averaging on the brightness:

[0076]

[0077] Here, δ is a very small value to prevent the occurrence of log(0) errors. It represents the logarithmic average scene brightness of the current image. By introducing the threshold τ, it can be judged whether the current image is in a dark scene. If the value is less than the threshold, it is a dark scene, denoted as S(I).

[0078] The image is then enhanced using the previously acquired image brightness V(x,y,I) and the image logarithmic average brightness The global enhancement matrix of the image can be expressed as:

[0079]

[0080] Where V max (I) represents the maximum value of the image brightness V(x,y,I), and then the three channels of the image can be enhanced:

[0081]

[0082] Among them I e represents the enhanced image, Ψ m (I e (x,y)) means the pixel value of the enhanced image at the position (x,y) of the m channel. From the above enhanced image, it is easy to get the enhanced part information of the image:

[0083] E(I)=V(I)-V(I e )

[0084] Finally, the mean value μ and standard deviation σ of E(I) are obtained through E(I), from which the global mask m can be obtained. g :

[0085]

[0086] Refer to the clipping matrix P in the BACF algorithm, for m g By clipping, we can get the expected mask m=m g ⊙P,P∈R w×h Used to extract object size information within a sample.

[0087] Step 2: Feature extraction stage. A large number of training samples are obtained through cosine window preprocessing, circulant matrix, and center cropping methods. Cosine window processing is to directly multiply the cosine window function on the sample. The operations of circulant matrix and center cropping are shown in the attached Figure 2 Then extract features from the acquired samples, including grayscale information, color information, gradient information, etc., and obtain the feature set x g Considering the shortcomings of the original cosine window, the present invention improves the cosine window. The specific steps are shown in the attached Figure 1 , first use the target initial size to clip the cosine window peak, the clipping position is parameter Q∈(0,1), Q can be obtained by the following formula:

[0088]

[0089] Among them, cosWin0 is the most original cosine window, W×H, W=H is the cosine window size, and w×h, w≥h is the target size.

[0090] After each tracking target scale update, obtain the scale update factor S scale , update the peak clipping position Q of the cosine window again scale =Q×S scale , so that the model scale is adaptive. Then use the mask m to get x o =m⊙x g , represents the simple target feature, so we get x g and x o Two feature sets. The specific model of the scale-adaptive cosine window is:

[0091]

[0092] In the above formula, cosWin0 is the most primitive cosine window, Q scale is the position of the cosine window clipping peak, S scale is the scale factor, W×H, W=H is the cosine window size, and w×h, w≥h is the target size.

[0093] Step 3: Training phase. Using the x obtained in Step 2 g and x o The feature set trains two filters for the next frame: h g and h o ,h g is a filter used to train background information, h o It is a filter used to train target information. This algorithm model is based on the ADTrack model. For the specific model, see the attached Figure 3 , the objective function of the model can be expressed as:

[0094]

[0095] Where P is the sample cropping matrix in the BACF algorithm of the ADTrack benchmark algorithm; represents the target information or background information filter of the cth channel; cosWin is the cosine window function proposed earlier that changes with the target scale factor and is used to preprocess the training sample data; y is the ideal Gaussian model; h t and h t-1 Represents the filters of the current frame and the previous frame, the M matrix represents the relationship between the two filters, λ is the constraint parameter of the filter regularization term, and μ is the constraint parameter of the mutual constraint term of the two filters.

[0096] For the overall objective function, because k∈{g,o}, plus the final mutual constraint between the two, the objective function can be seen as the accumulation of 7 parts. The 1st, 2nd and 4th, 5th parts are conventional linear models with additional clipping matrices, least squares plus regularization terms, the purpose of the regularization term is to prevent the filter from overfitting; the 3rd and 6th parts are the self-constraints of the two filters, which can effectively prevent the rapid degradation of the filters; the 7th part is the mutual constraint of the two filters, which are bound to each other during training, making the discrimination ability of the two filters more powerful.

[0097] Next, we use the ADMM iterative algorithm to solve the problem. In the iterative process, we can first assume that h o and M are known, for h g Perform ADMM iteration to find the optimal solution. Since the W cosine window function is only a preprocessing of the sample, it can be ignored during iteration. Use the augmented Lagrangian method to introduce relaxation variables P T is the transpose of the cropping matrix P, I N is an N×N identity matrix; the augmented Lagrangian form of the objective function is expressed as:

[0098]

[0099] in is the Lagrangian vector, and γ is the penalty factor. Using the ADMM method, the above equation can be transformed into the following three sub-problems by iteration:

[0100]

[0101] for Subproblem, h can be directly obtained through the first-order derivative g The closed-form solution of is:

[0102]

[0103] For the subproblem ξ *, it needs to be converted to the frequency domain for further calculation:

[0104]

[0105] The above formula can be decomposed into T sub-problems, T = 42 represents the dimension of the feature, and each sub-problem is We can get:

[0106]

[0107] Taking the derivative of the above formula, we can get:

[0108]

[0109] Due to the occurrence of matrix division, the amount of calculation is large, and the Sherman-Morrison equation is needed to optimize the solution of the inverse matrix, and we get:

[0110]

[0111] in is a scalar.

[0112] h g and h o The iteration process of is roughly the same, so I will not go into details. The iteration of the M matrix is:

[0113] Step 4: Detection phase. Target response graph It can be expressed as:

[0114]

[0115] in, Represents the corresponding quantity in the Fourier domain of the given data, Represents the response diagram after inverse Fourier transform, D corresponds to the dimension of the filter, and are the two filters of the fth frame, represents the c-th channel feature of the search area sample extracted from the f+1 frame, It is masked ρ is a control and The weight parameters of the two response maps generated. Finally, according to the response map The maximum value can determine the location information of the target.

[0116] The present invention improves the tracking performance of the target tracking algorithm in the scale-changing scene through the self-constraint and mutual constraint of the scale-adaptive cosine window and the filter, reduces the template drift problem in the tracking process, and greatly improves the accuracy and success rate of the algorithm. Figure 5-6 As shown in Table 1, the tracking success rate of this method ranks first, and the accuracy rate is tied with the AutoTrack algorithm for first place.

[0117] Table 1 The total accuracy and total success rate of each algorithm on the TC128 dataset

[0118] Ours CPCF ADTrack AutoTrack BACF KCF SRDCF BiCF Precision 0.702 0.697 0.689 0.702 0.644 0.544 0.644 0.641 Success 0.649 0.617 0.619 0.629 0.610 0.454 0.584 0.559

[0119] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. An improved cosine window correlation filtering target tracking method, characterized in that: The method comprises the following steps: Step 1, preprocessing stage; determine whether it is in a dark scene. If it is determined to be in a dark scene, enhance the video sequence and calculate the mask m for subsequent filter training; Step 2, feature extraction stage; Based on the training samples, we get the feature set x g , the adaptive updating of the tracking target scale is realized through the cosine window after peak clipping, and the target feature set x is obtained based on the mask m o , and obtain the scale-adaptive cosine window model; Step 3, training phase: using the feature set x obtained in step 2 g and x o Train two filters h for the next frame g and h o , where h g is a filter used to train background information, h o It is a filter used to train target information; Step 3 includes the following sub-steps: Step 3-1, the objective function of the filter is: Where P is the sample cropping matrix; represents the target information filter or background information filter of the cth channel; cosWin is the cosine window function proposed earlier that changes with the target scale factor and is used to preprocess the training sample data; y is the ideal Gaussian model; h t and h t-1 Represents the filters of the current frame and the previous frame, the M matrix represents the relationship between the two filters, λ is the constraint parameter of the filter regularization term, and μ is the constraint parameter of the mutual constraint term of the two filters; Step 3-2, for the overall objective function, because k∈{g,o}, plus the final mutual constraint between the two, the objective function is considered as the accumulation of 7 parts. The 1st, 2nd, 4th and 5th parts are conventional linear models with additional clipping matrices. The least squares plus regularization term prevents the filter from overfitting; the 3rd and 6th parts are the self-constraints of the two filters to prevent the rapid degradation of the filters; the 7th part is the mutual constraint of the two filters, which are bound to each other during training, making the discrimination ability of the two filters more powerful; Step 3-3, solve by ADMM iterative algorithm; because cosWin is the preprocessing of samples, it is ignored during iteration; for known h o and M, for h g Perform ADMM iterative optimal solution; use augmented Lagrangian method and introduce relaxation variables P T is the transpose of the cropping matrix P, I N is the N×N identity matrix; Step 4, detection phase; based on the trained filter and the extracted sample channel features, a target response map is obtained, and the target position information is determined by the maximum value of the target response map.

2. The target tracking method of an improved cosine window correlation filter according to claim 1, characterized in that: Step 1 includes the following sub-steps: Step 1-1, assuming that a color image is given Perform photometric integration on the RBG three channels of the image: Among them Ψ m (I(x,y)) represents the pixel value of the m-channel output image, and α R +α G +α B =1, convert the color image into a single-channel image; then perform logarithmic averaging on the brightness: Where δ is a constant used to prevent the appearance of log(0). Represents the logarithmic average scene brightness of the current image, and then introduces a threshold τ to determine whether the current image is in a dark scene. If it is less than the threshold, it is a dark scene, denoted as S(I); Step 1-2: Enhance the image by using the previously acquired image brightness V(x,y,I) and the image logarithmic average brightness The global enhancement matrix of the image is expressed as: Where V max (I) represents the maximum value of the image brightness V(x,y,I), and then the three channels of the image are enhanced: Among them I e represents the enhanced image, Ψ m (I e (x,y)) represents the pixel value of the enhanced image at the position (x,y) in the m channel; From the above enhanced image, we can get the enhanced part information of the image: E(I)=V(I)-V(I e ) Step 1-3, obtain the mean value μ and standard deviation σ of E(I) through E(I), thereby obtaining the global mask m g : By clipping the matrix P to m g Cut and get the expected mask m=m g ⊙P,P∈R w×h , used to extract the target size information within the sample.

3. The target tracking method of an improved cosine window correlation filter according to claim 1, characterized in that: Step 2 includes the following sub-steps: Step 2-1, obtain a large number of training samples through cosine window preprocessing, circulant matrix, and center cropping method. Cosine window processing is to directly multiply the cosine window function on the sample, and extract features of the obtained samples, including grayscale information, color information, gradient information, etc., to obtain the feature set x g ; Step 2-2, use the target initial size to clip the cosine window peak, the clipping position is parameter Q∈(0,1), Q is obtained by the following formula: Among them, cosWin0 is the most original cosine window, W×H, W=H is the cosine window size, and w×h, w≥h is the target size; Step 2-3, after each tracking target scale update, obtain the scale update factor S scale , update the peak clipping position Q of the cosine window again scale =Q×S scale , so that the model scale is adaptive; then use the mask m to get x o =m⊙x g , represents the simple target feature, and gets x g and x o Two feature sets; the specific model of the scale-adaptive cosine window is: In the above formula, cosWin0 is the most primitive cosine window, Q scale is the position of the cosine window clipping peak, S scale is the scale factor.

4. The target tracking method of an improved cosine window correlation filter according to claim 1, characterized in that: In step 3-3, the augmented Lagrangian form of the objective function is expressed as: in is the Lagrangian vector, γ is the penalty factor, I N is the N×N identity matrix, F N is an N×N Fourier matrix. Using the ADMM method, the above formula is transformed into the following three sub-problems in an iterative manner: for Subproblem, find h through the first-order derivative g The closed-form solution of is: For ξ * Subproblem, it needs to be converted to frequency domain for further calculation: The above formula is decomposed into T sub-problems, T = 42 represents the dimension of the feature, and each sub-problem is have to: Derivative of the above formula: Use the Sherman-Morrison equation to optimize the inverse matrix and get: in is a scalar; h g and h o The iteration process is the same as that of , and the iteration of the M matrix is:

5. The target tracking method of an improved cosine window correlation filter according to claim 1, characterized in that: In step 4, the target response plot It is expressed as: in, The corresponding quantity in the Fourier domain representing the given data, represents the response diagram after inverse Fourier transform, D represents the dimension of the filter, and are the two filters of the fth frame, represents the c-th channel feature of the search area sample extracted from the f+1 frame, It is masked ρ is a control and Weight parameter for the two resulting response plots.

Images