A method for automatic target tracking using correlation filters guided by dynamic and static conditions

Abstract

The invention discloses a method for automatically tracking a target with a correlation filter guided by dynamic and static conditions, constructs a model for automatically tracking a target with a correlation filter guided by dynamic and static conditions, and optimizes and solves the model for automatically tracking a target with a correlation filter by using an ADMM method; inputs a first single-frame image and a target to be tracked; step 3: inputs the next single-frame image, performs dynamic and static guidance and target detection; step 4: performs N optimization iterations on the model for automatically tracking a target with a correlation filter, and updates the model for automatically tracking a target with a correlation filter by using a corresponding update formula; if the last single-frame image has not been reached, returns to step 3; if the video has reached the last single-frame image, outputs the target obtained by the final tracking of each frame of video and the average video tracking rate FPS. The features of a single-frame image are used as static guidance, and the correlation filter of the previous single-frame image is used as dynamic guidance, and the update of the correlation filter of the next single-frame image is jointly guided.

Description

Technical Field

[0001] The invention relates to a method for automatically tracking a target using a correlation filter guided by dynamic or static conditions, and belongs to the technical field of video image processing. Background Art

[0002] Object tracking is a challenging task in computer vision because it requires tracking a given moving object in a complex environment. When the object in the video moves, the tracking task is to establish the trajectory of the object on these video sequences. Since the emergence of the MOSSE correlation filter algorithm, its excellent performance compared to traditional algorithms has attracted more and more people to study this direction. In addition, the use of deep algorithms and their combination with hand-crafted features have also significantly improved the tracking performance. In recent years, with the optimization of tracking models and the improvement of feature extraction methods, the tracking performance and accuracy of correlation filters have been continuously improved.

[0003] Many tasks in computer vision can basically be expanded into ill-posed inverse problems. Therefore, in most real-world problems, a prerequisite constraint condition is required to turn the problem into a convex optimal function problem. Especially in target tracking tasks, this is used to obtain a smoothly changing solution. In previous processing, the target model was constrained in two ways: the first is to perform regularized dynamic guidance constraints on the target, which basically directly or indirectly constrains its related filters. Although this method can track the target well, it ignores the information static guidance of the video features itself. The second method is to use the extraction method or processing method of the image features itself to statically guide the update of the constraint related filters, which can reflect the internal characteristics of the input image itself. This framework only determines the structure of the output image by referencing the structure of the guidance image, without considering the structural (or statistical) dependence and inconsistency between the input and guidance images, which has certain limitations.

[0004] The work on joint regularization or joint joint filtering provides a new perspective on regularization, and its applications are also very wide, including stereo correspondence, optical flow, joint sampling, denoising, noise reduction and texture removal. The idea of ​​joint regularization is to transfer the structure of the guidance image to the input image. It assumes that the guidance image has enough information to restore the noise or changed structure in the input image, ensuring that the correlation filter structure of the previous and next frame videos is stable and there is no target tracking loss problem. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method for automatically tracking a target with a correlation filter under dynamic and static guidance. The automatic tracking target model of the correlation filter introduces dynamic and static guidance, and no longer relies solely on the dynamic guidance of the correlation filter for updating, thereby avoiding static guidance of information about the characteristics of the video itself when the target is deformed, which in turn leads to structural update errors of the correlation filter.

[0006] To achieve the above object, the present invention provides a method for automatically tracking a target using a correlation filter guided by dynamic or static conditions, comprising:

[0007] Step 1: Decompose the sequence video shot by the drone into multiple single-frame images in chronological order, and assign t = 1;

[0008] Step 2, when t=1, input the first single frame image, the target to be tracked and the expected output response into the constructed dynamic and static joint guided correlation filter automatic tracking target model, and obtain the initial correlation filter for tracking the target to be tracked in the second single frame image and the dynamic and static guidance of the first single frame image; assign t=2;

[0009] Step 3, when t≥2, input the t-th single frame image into the dynamic and static joint guided correlation filter automatic tracking target model, and perform target tracking detection on the t-th single frame image based on the dynamic and static guidance of the t-1-th single frame image and the correlation filter of the t-1-th single frame image;

[0010] Step 4, performing N optimization iterations on the dynamic and static joint guided correlation filter automatic tracking target model, and updating the parameters of the correlation filter automatic tracking target model through an update formula;

[0011] Step 5, judging whether the candidate target area of ​​the current single-frame image is smaller than the set threshold value, if the candidate target area of ​​the current single-frame image is smaller than the set threshold value, the candidate target area of ​​the current single-frame image is used as the final tracked target of the current single-frame image, the iterative operation ends, the value of t is increased by 1, and the process returns to step 3, otherwise, the process returns to step 4, and the process continues to judge the next candidate target area of ​​the current single-frame image;

[0012] If the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is not the last single-frame image, return to step 3; if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is the last single-frame image, then output the final tracked target of each single-frame image and the average video tracking rate FPS of each single-frame image.

[0013] Prioritize the construction of a dynamic and static guided correlation filter automatic tracking target model, including:

[0014] A dynamic and static joint guided correlation filter automatic tracking target model is established, and the alternating iterative multiplier method is used to optimize and solve the correlation filter automatic tracking target model.

[0015] Prioritize the construction of a dynamic and static joint guided correlation filter automatic tracking target model, including:

[0016] make is the k-th channel input image feature of the t-th single frame image, R L×1 Represents a real vector of length L×1, where L is the length of the k-th channel input image feature of the t-th single frame image extracted, y∈R L×1 is the expected Gaussian response, is the correlation filter of the kth channel of the tth single frame image; S t is the time regularization parameter to be optimized, initially the unit matrix; is the correlation filter of all feature channels of the t-th single frame image, initially the unit matrix;

[0017] The static joint guided correlation filter automatic tracking target model formula is:

[0018]

[0019] Among them, ε(H t , S t ) represents the value of the following equation, is the circular convolution, ⊙ is the Hadamard product; is the local response, where P L ∈R L×L Refers to the cropping matrix with a side length of L used to crop the candidate target area where the center of the candidate correlation filter is located. δ is a parameter used to control the local response weight, l is a parameter used to eliminate the boundary effect, and Π is the local response change vector; is the global response, α and v are both hyperparameters. When the overall change value is higher than the set threshold When , it indicates that there is distortion in the response map. At this time, the static joint guided correlation filter automatically tracks the target model and stops the learning of the correlation filter of the subsequent candidate target area;

[0020] Ω(d t ) is the dynamic and static guidance, and the specific expression is: in β and v are constants; p x is the center pixel of the current predicted search box. and x k t-1 are the static guidance of the t-th single frame image and the static guidance of the t-1-th single frame image respectively; d tis the dynamic guidance of the response of all channels of the correlation filter, Initially set to the identity matrix. Prioritize, use the alternating iterative multiplier method to optimize and solve the static joint guided correlation filter automatic tracking target model, including:

[0021] Set auxiliary variable where F∈I L×L represents the standard orthogonal matrix, I L×L is an L×L unit matrix, the symbol ∧ represents the discrete Fourier transform of the signal, and the augmented Lagrangian form of formula (1) is:

[0022]

[0023] Among them, when the value on the right side of the equation is the smallest, the corresponding H is output at this time t , S t and is the global response of the t-1th single frame image; use To express; H t is the correlation filter of all feature channels of the t-th single frame image, H t and The initial values ​​are all set to the identity matrix;

[0024] In order to obtain the optimal solution of formula (2) through the alternating iterative multiplier method, its augmented Lagrangian form is modified as follows:

[0025]

[0026] in is the Fourier transform of the Lagrange multiplier, γ is the step-size regularization parameter, T is the matrix transpose, is the Fourier transform of the Lagrange multiplier of the kth channel of the tth single frame image;

[0027] Assumptions Solving for optimization The corresponding auxiliary variable of iteration; r is a constant, and formula (3) is transformed into:

[0028]

[0029] in, V t Fourier transform of

[0030] Introducing slack variables S′ t =S t , formula (4) is constructed as the following Lagrangian function augmentation function:

[0031]

[0032] Where, the Lagrange multiplier Γ and The same size, Γ k is the penalty for the corresponding k-th channel, and η is the corresponding penalty parameter;

[0033] Formula (5) is decomposed into five sub-problems:

[0034]

[0035] Solve the above five sub-problems. For the first sub-problem, restore S t , S′ t , The default value gives:

[0036]

[0037] in, Represents a diagonal matrix, diag() is a diagonal function. Perform diagonalization.

[0038] For the second of the five subproblems, the closed solution is:

[0039]

[0040] Among them, η is the penalty parameter of the Lagrangian term, and its initial value is 1;

[0041] For the third of the five subproblems, the closed solution is:

[0042]

[0043] For the fourth of the five subproblems, the solution is:

[0044]

[0045] Based on Sherman Morrison's theorem, formula (10) is transformed into:

[0046]

[0047] Where I is the identity matrix, For each pixel in all channels The Fourier transform after sampling, for Transpose, Y is the expected Gaussian response after pixel sampling of all channels, The vector ρ is

[0048] After solving the above four sub-problems, the Lagrange multiplier is updated as:

[0049]

[0050] Where i represents the i-th iteration index, i+1 represents the i+1-th iteration index; γ is the step-size regularization constant, whose initial value is 1, and the subsequent update formula is: γ i+1 =min(γ max , βγ i )(β=10,γ max =10000); Γ is the Lagrange multiplier, initialized to 1, and the subsequent update formula is η i+1 =η i +ξ(S t -S′ t ), ξ is the corresponding penalty.

[0051] Preferably, the first single-frame image, the target to be tracked, and the expected output response are input into the constructed dynamic and static joint guided correlation filter automatic tracking target model to obtain an initial correlation filter for tracking the target to be tracked in the next single-frame image and the dynamic and static guidance of the first single-frame image, including:

[0052] The first single-frame image, the given position of the target to be tracked and the expected output response y are input into the dynamic and static joint guided correlation filter automatic tracking target model. The position of the target to be tracked includes the center point coordinates of the position of the target to be tracked and the length and width of the target to be tracked. The color features, grayscale features, directional gradient histogram features of the given first single-frame image of the target to be tracked and the correlation filter corresponding to the given target to be tracked are convolved, that is,

[0053]

[0054] in is the kth channel feature of the target to be tracked in the first single frame image, is the correlation filter corresponding to the kth channel of the target to be tracked in the given first single frame image, and y is the expected output response y of the target to be tracked in the given first single frame image;

[0055] Perform discrete Fourier transform on the above formula and match the correlation filter template with the single frame image:

[0056]

[0057] in is the discrete Fourier transform of the expected output response y of the target to be tracked in the first single frame image, is the dual form of the discrete Fourier transform of the target to be tracked in the first single frame image, is the correlation filter h corresponding to the target to be tracked in the first single frame image 1 The discrete Fourier transform of the time domain is transformed into the Hadamard operation in the frequency domain to reduce the amount of calculation.

[0058] Changing the above formula yields:

[0059] Based on this formula, the correlation filter of the first single frame image is obtained;

[0060] The color features, grayscale features and directional gradient histogram features of the target to be tracked in the first single-frame image are used as static guides for the second single-frame image;

[0061] The calculation formula for the dynamic guidance of the first single frame image is:

[0062] Among them, d 1 For dynamic guidance, x 1 It is all channel features of the target to be tracked in the given first single frame image.

[0063] Preferably, the t-th single frame image is input into the dynamic and static joint guidance correlation filter automatic tracking target model, and the target to be tracked in the t-th single frame image is tracked and detected based on the dynamic and static guidance of the t-1-th single frame image and the correlation filter of the t-1-th single frame image, including:

[0064] Input the tth single frame image x into the dynamic and static joint guided correlation filter automatic tracking target model t , t≥2, the static guidance x of the color features, grayscale features and directional gradient histogram features of the target obtained by the final tracking of the t-1th single frame image is input into the dynamic and static joint guidance correlation filter automatic tracking target model t-1 ;

[0065] Input the t-1th single frame image and the expected output response of the target finally tracked as the dynamic guide h t-1 is the correlation filter h corresponding to the final tracking target of the t-1th single frame image t-1 .

[0066] Preferably, step 4: performing N optimization iterations on the dynamic and static joint guided correlation filter automatic tracking target model, and updating the parameters of the correlation filter automatic tracking target model through the update formula, as follows:

[0067] Input y, r, ξ, η, Γ, N;

[0068] Initialize 0 =g 0 =S t =S′ t =0, i=0;

[0069] When (i<<N), iterate:

[0070] (1) By updating the formula renew

[0071] (2) By updating the formula renew

[0072] (3) By updating the formula Update S′ t ;

[0073] (4) By updating the formula

[0074] renew

[0075] (5) By updating the formula renew

[0076] (6)γ i+1 =min(γ max , βγ i )(β=10,γ max =10000) Update the step length regularization parameter γ based on η i+1 =η i +ξ(S t -S′ t ) formula updates the Lagrange multiplier η;

[0077] (7) The value of i increases by 1;

[0078] (8) Output the correlation filter H that matches the next single frame image t+1 .

[0079] Prioritize, determine whether the candidate target area of ​​the current single-frame image is smaller than the set threshold value, if the candidate target area of ​​the current single-frame image is smaller than the set threshold value, take the candidate target area of ​​the current single-frame image as the final tracked target of the current single-frame image, end the iterative operation, add 1 to the value of t, and return to step 3, otherwise return to step 4, and continue to determine the next candidate target area of ​​the current single-frame image, including:

[0080] Enter the features x of the candidate regions in order from left to right and from top to bottom. t ;

[0081] Before each iteration of the candidate target region of the current single frame image, restore y, r, ξ, η, Γ, N to the initial default values ​​and set υ 0 =g 0 =S t =S′ t =0, i=0;

[0082] The loop is iterated N times (i<N). In each iteration, the static guidance and dynamic guidance of the previous single frame image are substituted into formula (5) for calculation. If the value calculated by formula (5) in a certain iteration is less than the set threshold Then this candidate target area is taken as the final tracked target of the current single frame image, the iterative operation of the subsequent candidate target areas is stopped, the value of t is increased by 1, and the process returns to step 3; otherwise, the process returns to step 4 and the judgment of the next candidate target area of ​​the current single frame image is continued. If the values ​​calculated by formula (5) do not reach the set threshold, Then the candidate target area corresponding to the minimum value calculated by formula (5) is taken as the final tracked target of the current single frame image.

[0083] Preferably, if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is not the last single-frame image, return to step 3; if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is the last single-frame image, then the final tracked target of each single-frame image is output, and the video tracking average rate FPS of each single-frame image is output, and the video tracking average rate FPS of each single-frame image = the total time spent tracking the target in the sequence video taken by the drone divided by the total number of frames of the video image.

[0084] Preferably, β and v are set to 1, and r is 1.

[0085] The beneficial effects achieved by the present invention are:

[0086] The algorithm proposed in the present invention makes full use of dynamic and static guidance, and can better combine the static features of the target itself and the dynamic structure of the correlation filter of the previous frame to guide the correlation filter of the current video sequence, rather than using only the dynamic guidance of the correlation filter structure. Its technical effects include: First, since the current single guidance cannot effectively avoid the structural error of the correlation filter structure caused by target deformation, we use dynamic and static joint guidance to improve the robustness and noise resistance of the model; second, to effectively track in various video environments, we use global response and local response to automatically adjust the structure of the correlation filter, rather than fixed parameter penalty; third, in order to effectively verify the effectiveness and accuracy of our method, we conducted experiments on the UAV@10fps123, UAVDT-S and UAVDT-M datasets. The experimental results are as follows: Figure 3-Figure 8 .

[0087] In summary, it can be seen that the method for automatically tracking a target using a correlation filter under dynamic and static guidance proposed by the present invention has higher accuracy and robustness, and is less complex, simple to implement, and has a faster calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 is a flow chart of the method of the present invention;

[0089] Figure 2 It is a schematic diagram of the tracking effect of the method of the present invention;

[0090] Figure 3 This is a comparison chart of the quantitative analysis of the success rate of the method of the present invention in the UAV@10fps123 data set;

[0091] Figure 4 This is a quantitative analysis comparison chart of the accuracy of the method of the present invention in the UAV@10fps123 data set;

[0092] Figure 5 This is a comparison chart of the quantitative analysis of the success rate of the method of the present invention in the UAVDT-S data set;

[0093] Figure 6 This is a comparison chart of the accuracy of the method of the present invention in the UAVDT-S data set;

[0094] Figure 7 This is a comparison chart of the quantitative analysis of the success rate of the method of the present invention in the UAVDT-M data set;

[0095] Figure 8 This is a comparison chart of the quantitative analysis of the accuracy of the method of the present invention in the UAVDT-M data set. DETAILED DESCRIPTION

[0096] The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and are not intended to limit the protection scope of the present invention.

[0097] The Chinese meaning of ADMM method is alternating iterative multiplier method.

[0098] A method for automatically tracking a target using a correlation filter guided by dynamic and static conditions, comprising:

[0099] Step 1: Decompose the sequence video shot by the drone into multiple single-frame images in chronological order, and assign t = 1;

[0100] Step 2, when t=1, input the first single frame image, the target to be tracked and the expected output response into the constructed dynamic and static joint guided correlation filter automatic tracking target model, and obtain the initial correlation filter for tracking the target to be tracked in the second single frame image and the dynamic and static guidance of the first single frame image; assign t=2;

[0101] Step 3, when t≥2, input the t-th single frame image into the dynamic and static joint guided correlation filter automatic tracking target model, and perform target tracking detection on the t-th single frame image based on the dynamic and static guidance of the t-1-th single frame image and the correlation filter of the t-1-th single frame image;

[0102] Step 4, performing N optimization iterations on the dynamic and static joint guided correlation filter automatic tracking target model, and updating the parameters of the correlation filter automatic tracking target model through an update formula;

[0103] Step 5, judging whether the candidate target area of ​​the current single-frame image is smaller than the set threshold value, if the candidate target area of ​​the current single-frame image is smaller than the set threshold value, the candidate target area of ​​the current single-frame image is used as the final tracked target of the current single-frame image, the iterative operation ends, the value of t is increased by 1, and the process returns to step 3, otherwise, the process returns to step 4, and the process continues to judge the next candidate target area of ​​the current single-frame image;

[0104] If the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is not the last single-frame image, return to step 3; if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is the last single-frame image, then output the final tracked target of each single-frame image and the average video tracking rate FPS of each single-frame image.

[0105] Furthermore, a dynamic and static guided correlation filter automatic tracking target model is constructed, including:

[0106] A dynamic and static joint guided correlation filter automatic tracking target model is established, and the alternating iterative multiplier method is used to optimize and solve the correlation filter automatic tracking target model.

[0107] Furthermore, a dynamic and static joint guided correlation filter automatic tracking target model is constructed, including:

[0108] make is the k-th channel input image feature of the t-th single frame image, R L×1 Represents a real vector of length L×1, where L is the length of the k-th channel input image feature of the t-th single frame image extracted, y∈R L×1 is the expected Gaussian response, is the correlation filter of the kth channel of the tth single frame image; S t is the time regularization parameter to be optimized, initially the unit matrix; is the correlation filter of all feature channels of the t-th single frame image, initially the unit matrix;

[0109] The static joint guided correlation filter automatic tracking target model formula is:

[0110]

[0111] Among them, ε(H t , S t ) represents the value of the following equation, is the circular convolution, ⊙ is the Hadamard product; is the local response, where P L ∈R L×L Refers to the cropping matrix with a side length of L used to crop the candidate target area where the center of the candidate correlation filter is located. δ is a parameter used to control the local response weight, l is a parameter used to eliminate the boundary effect, and Π is the local response change vector; is the global response, α and v are both hyperparameters. When the overall change value is higher than the set threshold When , it indicates that there is distortion in the response map. At this time, the static joint guided correlation filter automatically tracks the target model and stops the learning of the correlation filter of the subsequent candidate target area;

[0112] Ω(d t ) is the dynamic and static guidance, and the specific expression is: in β and v are constants; p x is the center pixel of the current predicted search box. and x k t-1 are the static guidance of the t-th single frame image and the static guidance of the t-1-th single frame image respectively; d t is the dynamic guidance of the response of all channels of the correlation filter, Initially set to the identity matrix.

[0113] Furthermore, the alternating iterative multiplier method is used to optimize and solve the static joint guided correlation filter automatic tracking target model, including:

[0114] Set auxiliary variable where F∈I L×L represents the standard orthogonal matrix, I L×L is an L×L unit matrix, the symbol ∧ represents the discrete Fourier transform of the signal, and the augmented Lagrangian form of formula (1) is:

[0115]

[0116] Among them, when the value on the right side of the equation is the smallest, the corresponding H is output at this time t , S t and is the global response of the t-1th single frame image; use To express; H t is the correlation filter of all feature channels of the t-th single frame image, H t and The initial values ​​are all set to the identity matrix;

[0117] In order to obtain the optimal solution of formula (2) through the alternating iterative multiplier method, its augmented Lagrangian form is modified as follows:

[0118]

[0119] in is the Fourier transform of the Lagrange multiplier, γ is the step-size regularization parameter, T is the matrix transpose, is the Fourier transform of the Lagrange multiplier of the kth channel of the tth single frame image;

[0120] Assumptions Solving for optimization The corresponding auxiliary variable of iteration; r is a constant, and formula (3) is transformed into:

[0121]

[0122] in, V t Fourier transform of

[0123] Introducing slack variables S′ t =S t , formula (4) is constructed as the following Lagrangian function augmentation function:

[0124]

[0125] Where, the Lagrange multiplier Γ and The same size, Γ k is the penalty for the corresponding k-th channel, and η is the corresponding penalty parameter;

[0126] Formula (5) is decomposed into five sub-problems:

[0127]

[0128] Solve the above five sub-problems. For the first sub-problem, restore S t , S′ t , The default value gives:

[0129]

[0130] in, Represents a diagonal matrix, diag() is a diagonal function. Perform diagonalization.

[0131] For the second of the five subproblems, the closed solution is:

[0132]

[0133] Among them, η is the penalty parameter of the Lagrangian term, and its initial value is 1;

[0134] For the third of the five subproblems, the closed solution is:

[0135]

[0136] For the fourth of the five subproblems, the solution is:

[0137]

[0138] Based on Sherman Morrison's theorem, formula (10) is transformed into:

[0139]

[0140] Where I is the identity matrix, For each pixel in all channels The Fourier transform after sampling, for Transpose, Y is the expected Gaussian response after pixel sampling of all channels, The vector ρ is

[0141] After solving the above four sub-problems, the Lagrange multiplier is updated as:

[0142]

[0143] Where i represents the i-th iteration index, i+1 represents the i+1-th iteration index; γ is the step-size regularization constant, whose initial value is 1, and the subsequent update formula is: γ i+1 =min(γ max , βγ i )(β=10,γ max =10000); Γ is the Lagrange multiplier, initialized to 1, and the subsequent update formula is η i+1 =η i +ξ(S t -S′ t ), ξ is the corresponding penalty.

[0144] Furthermore, the first single-frame image, the target to be tracked, and the expected output response are input into the constructed dynamic and static joint guided correlation filter automatic tracking target model to obtain the initial correlation filter for tracking the target to be tracked in the next single-frame image and the dynamic and static guidance of the first single-frame image, including:

[0145] The first single-frame image, the given position of the target to be tracked and the expected output response y are input into the dynamic and static joint guided correlation filter automatic tracking target model. The position of the target to be tracked includes the center point coordinates of the position of the target to be tracked and the length and width of the target to be tracked. The color features, grayscale features, directional gradient histogram features of the given first single-frame image of the target to be tracked and the correlation filter corresponding to the given target to be tracked are convolved, that is,

[0146]

[0147] in is the kth channel feature of the target to be tracked in the first single frame image, is the correlation filter corresponding to the kth channel of the target to be tracked in the given first single frame image, and y is the expected output response y of the target to be tracked in the given first single frame image;

[0148] Perform discrete Fourier transform on the above formula and match the correlation filter template with the single frame image:

[0149]

[0150] in is the discrete Fourier transform of the expected output response y of the target to be tracked in the first single frame image, is the dual form of the discrete Fourier transform of the target to be tracked in the first single frame image, is the correlation filter h corresponding to the target to be tracked in the first single frame image1 The discrete Fourier transform of the time domain is transformed into the Hadamard operation in the frequency domain to reduce the amount of calculation.

[0151] Changing the above formula yields:

[0152] Based on this formula, the correlation filter of the first single frame image is obtained;

[0153] The color features, grayscale features and directional gradient histogram features of the target to be tracked in the first single-frame image are used as static guides for the second single-frame image;

[0154] The calculation formula for the dynamic guidance of the first single frame image is:

[0155] Among them, d 1 For dynamic guidance, x 1 It is all channel features of the target to be tracked in the given first single frame image.

[0156] Furthermore, the t-th single frame image is input into the dynamic and static joint guided correlation filter automatic tracking target model, and the target to be tracked in the t-th single frame image is tracked and detected based on the dynamic and static guidance of the t-1-th single frame image and the correlation filter of the t-1-th single frame image, including:

[0157] Input the tth single frame image x into the dynamic and static joint guided correlation filter automatic tracking target model t , t≥2, the static guidance x of the color features, grayscale features and directional gradient histogram features of the target obtained by the final tracking of the t-1th single frame image is input into the dynamic and static joint guidance correlation filter automatic tracking target model t-1 ;

[0158] Input the t-1th single frame image and the expected output response of the target finally tracked as the dynamic guide h t-1 is the correlation filter h corresponding to the final tracking target of the t-1th single frame image t-1 .

[0159] Further, step 4: perform N optimization iterations on the dynamic and static joint guided correlation filter automatic tracking target model, and update the parameters of the correlation filter automatic tracking target model through the update formula, as follows:

[0160] Input y, r, ξ, η, Γ, N;

[0161] Initialize 0 =g 0 =S t =S′t =0, i=0;

[0162] When (i<<N), iterate:

[0163] (1) By updating the formula renew

[0164] (2) By updating the formula renew

[0165] (3) By updating the formula Update S′ t ;

[0166] (4) By updating the formula

[0167] renew

[0168] (5) By updating the formula renew

[0169] (6)γ i+1 =min(γ max , βγ i )(β=10,γ max =10000) Update the step length regularization parameter γ based on η i+1 =η i +ξ(S t -S′ t ) formula updates the Lagrange multiplier η;

[0170] (7) The value of i increases by 1;

[0171] (8) Output the correlation filter H that matches the next single frame image t+1 , get the coordinates of the target center point and the length and width of the target obtained by tracking in the current single frame image.

[0172] Further, it is determined whether the candidate target area of ​​the current single-frame image is smaller than the set threshold. If the candidate target area of ​​the current single-frame image is smaller than the set threshold, the candidate target area of ​​the current single-frame image is used as the final tracked target of the current single-frame image, the iterative operation is terminated, the value of t is increased by 1, and the process returns to step 3. Otherwise, the process returns to step 4, and the process continues to determine the next candidate target area of ​​the current single-frame image, including:

[0173] Enter the features x of the candidate regions in order from left to right and from top to bottom. t ;

[0174] Before each iteration of the candidate target region of the current single frame image, restore y, r, ξ, η, Γ, N to the initial default values ​​and set υ 0 =g 0 =S t =S′ t =0, i=0;

[0175] The loop is iterated N times (i<N). In each iteration, the static guidance and dynamic guidance of the previous single frame image are substituted into formula (5) for calculation. If the value calculated by formula (5) in a certain iteration is less than the set threshold Then this candidate target area is taken as the final tracked target of the current single frame image, the iterative operation of the subsequent candidate target areas is stopped, the value of t is increased by 1, and the process returns to step 3; otherwise, the process returns to step 4 and the judgment of the next candidate target area of ​​the current single frame image is continued. If the values ​​calculated by formula (5) do not reach the set threshold, Then the candidate target area corresponding to the minimum value calculated by formula (5) is taken as the final tracked target of the current single frame image.

[0176] Furthermore, if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is not the last single-frame image, return to step 3; if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is the last single-frame image, the final tracked target of each single-frame image is output, and the video tracking average rate FPS of each single-frame image is output, and the video tracking average rate FPS of each single-frame image = the total time spent tracking the target in the sequence video taken by the drone divided by the total number of frames of the video image.

[0177] Furthermore, β and v are set to 1, and r is 1. Figure 3-Figure 8 The comparative results of tracking success rate and accuracy of our algorithm in different single-target video target tracking datasets are shown.

[0178] In summary, the method for automatically tracking a target with a correlation filter under dynamic and static guidance proposed in the present invention introduces dynamic and static guidance, and no longer relies solely on the dynamic guidance of the correlation filter for updating, thereby avoiding static guidance of information about the characteristics of the video itself when the target is deformed, which in turn leads to structural update errors of the correlation filter.

[0179] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for automatically tracking targets using a correlation filter guided by dynamic and static conditions. It is characterized in that include: Step 1: Decompose the sequence video taken by the drone into multiple single-frame images in chronological order, and assign t=1; Step 2, when t=1, input the first single frame image, the target to be tracked and the expected output response into the constructed dynamic and static joint guided correlation filter automatic tracking target model, and obtain the initial correlation filter for tracking the target to be tracked in the second single frame image and the dynamic and static guidance of the first single frame image; assign t=2; Step 3, when t≥2, input the t-th single frame image into the dynamic and static joint guided correlation filter automatic tracking target model, and perform target tracking detection on the t-th single frame image based on the dynamic and static guidance of the t-1-th single frame image and the correlation filter of the t-1-th single frame image; Step 4, performing N optimization iterations on the dynamic and static joint guided correlation filter automatic tracking target model, and updating the parameters of the correlation filter automatic tracking target model through an update formula; Step 5, judging whether the candidate target area of ​​the current single-frame image is smaller than the set threshold value, if the candidate target area of ​​the current single-frame image is smaller than the set threshold value, the candidate target area of ​​the current single-frame image is used as the final tracked target of the current single-frame image, the iterative operation ends, the value of t is increased by 1, and the process returns to step 3, otherwise, the process returns to step 4, and the process continues to judge the next candidate target area of ​​the current single-frame image; If the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is not the last single-frame image, return to step 3; if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is the last single-frame image, then output the final tracked target of each single-frame image and the average video tracking rate FPS of each single-frame image.

2. According to the method of automatic target tracking by correlation filter guided by dynamic and static joint guidance according to claim 1, It is characterized in that Construct a dynamic and static guided correlation filter automatic tracking target model, including: A dynamic and static joint guided correlation filter automatic tracking target model is established, and the alternating iterative multiplier method is used to optimize and solve the correlation filter automatic tracking target model.

3. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 2, It is characterized in that Construct a dynamic and static joint guided correlation filter automatic tracking target model, including: make Input image features for the kth channel of the tth single frame image, represents a real number vector of length L×1, where L is the length of the k-th channel input image feature of the extracted t-th single frame image, is the expected Gaussian response, is the correlation filter of the kth channel of the tth single frame image; is the time regularization parameter to be optimized, initially the unit matrix; is the correlation filter of all feature channels of the t-th single frame image, initially the unit matrix; The static joint guided correlation filter automatic tracking target model formula is: ;in, represents the value of the following equation, is the circular convolution, is the Hadamard product; is the local response, where Refers to the cropping matrix with a side length of L used to crop the candidate target area where the center of the candidate correlation filter is located. is a parameter used to control the local response weight, is a parameter used to eliminate the influence of boundaries. is the local response change vector; is the global response, and are all hyperparameters. When the overall change value is higher than the set threshold When , it indicates that there is distortion in the response map. At this time, the static joint guided correlation filter automatically tracks the target model and stops the learning of the correlation filter of the subsequent candidate target area; For dynamic and static guidance, the specific expression is: ,in , , and is a constant; is the center pixel of the current predicted search box. and They are the static guidance of the t-th single frame image and the static guidance of the t-1-th single frame image respectively; is the dynamic guidance of the response of all channels of the correlation filter, , which is initially set to the identity matrix.

4. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 3, It is characterized in that The alternating iterative multiplier method is used to optimize the static joint guided correlation filter automatic tracking target model, including: Set auxiliary variable ,in represents a standard orthogonal matrix, for L×L The identity matrix, symbol Representing the discrete Fourier transform of the signal, the augmented Lagrangian form of formula (1) is: ; When the value on the right side of the equation is the smallest, the corresponding output is H t , S t and ; is the global response of the t-1th single frame image; ,use To express; , H t is the correlation filter of all feature channels of the t-th single frame image, , H t and The initial values ​​are all set to the identity matrix; In order to obtain the optimal solution of equation (2) through the alternating iterative multiplier method, its augmented Lagrangian form is modified as follows: ;in , is the Fourier transform of the Lagrange multiplier, is the step size regularization parameter, T is the matrix transpose, is the Fourier transform of the Lagrange multiplier of the kth channel of the tth single frame image; Assumptions , Solving for optimization The corresponding auxiliary variable of iteration; r is a constant, and formula (3) is transformed into: ;in, for Fourier transform of Introducing slack variables , formula (4) is constructed as the following Lagrangian function augmentation function: ; Where, the Lagrange multiplier and The same size, is the penalty corresponding to the kth channel, is the corresponding penalty parameter; Formula (5) is decomposed into five sub-problems: ； Solve the above five sub-problems. For the first sub-problem, recover The default value gives: ;in, , Represents a diagonal matrix, diag() is a diagonal function. Perform diagonalization. For the second of the five subproblems, the closed solution is: ;in, is the penalty parameter of the Lagrangian term, with an initial value of 1; For the third of the five subproblems, the closed solution is: ； For the fourth of the five subproblems, the solution is: ； Based on Sherman Morrison's theorem, formula (10) is transformed into: ； Where I is the identity matrix, For each pixel in all channels The Fourier transform after sampling, for Transpose, Y is the expected Gaussian response after pixel sampling of all channels, , vector for ; After solving the above four sub-problems, the Lagrange multiplier is updated as: ； In the formula, i Representative i Iteration index, i+1 Representative i+1 Iteration index; is the step size regularization constant, its initial value is 1, and the subsequent update formula is: ; is the Lagrange multiplier, initialized to 1, and the subsequent update formula is , for corresponding punishment.

5. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 4, It is characterized in that The first single-frame image, the target to be tracked and the expected output response are input into the constructed dynamic and static joint guided correlation filter automatic tracking target model to obtain the initial correlation filter for tracking the target to be tracked in the next single-frame image and the dynamic and static guidance of the first single-frame image, including: The first single-frame image, the given position of the target to be tracked and the expected output response y are input into the dynamic and static joint guided correlation filter automatic tracking target model. The position of the target to be tracked includes the center point coordinates of the position of the target to be tracked and the length and width of the target to be tracked. The color features, grayscale features, directional gradient histogram features of the given first single-frame image target to be tracked and the correlation filter corresponding to the given target to be tracked are convolved: ;in, is the kth channel feature of the target to be tracked in the first single frame image, is the correlation filter corresponding to the kth channel of the target to be tracked in the given first single frame image, and y is the expected output response y of the target to be tracked in the given first single frame image; Perform discrete Fourier transform on the above formula and match the correlation filter template with the single frame image: ;in The expected output response of the target to be tracked in the first single frame image The discrete Fourier transform of is the dual form of the discrete Fourier transform of the target to be tracked in the first single frame image, is the correlation filter corresponding to the target to be tracked in the first single frame image The discrete Fourier transform of the time domain is transformed into the Hadamard operation in the frequency domain to reduce the amount of calculation. Changing the above formula yields: , based on this formula, the correlation filter of the first single frame image is obtained; The color features, grayscale features and directional gradient histogram features of the target to be tracked in the first single-frame image are used as static guides for the second single-frame image; The calculation formula for the dynamic guidance of the first single frame image is: ,in, d 1 For dynamic guidance, x 1 It is all channel features of the target to be tracked in the given first single frame image.

6. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 5, It is characterized in that Inputting the t-th single frame image into the dynamic and static joint guided correlation filter automatic tracking target model, tracking and detecting the target to be tracked in the t-th single frame image based on the dynamic and static guidance of the t-1-th single frame image and the correlation filter of the t-1-th single frame image, including: Input the tth single frame image into the dynamic and static joint guided correlation filter automatic tracking target model , t≥2, static guidance of the color features, grayscale features and directional gradient histogram features of the target obtained by the final tracking of the t-1th single frame image into the dynamic and static joint guidance correlation filter automatic tracking target model ; Input the t-1th single frame image and the expected output response of the target finally tracked as the dynamic guide , is the correlation filter corresponding to the final tracking target of the t-1th single frame image .

7. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 6, It is characterized in that Step 4: Perform N optimization iterations on the dynamic and static joint guided correlation filter automatic tracking target model, and update the parameters of the correlation filter automatic tracking target model through the update formula, as follows: enter ; initialization ; when , iterate: (1) By updating the formula ,renew ; (2) By updating the formula ,renew ; (3) By updating the formula ,renew ; (4) By updating the formula ,renew ; (5) By updating the formula ,renew ; (6) Update the step length regularization parameter ,based on Formula to update Lagrange multipliers ; (7) i The value of is increased by 1; (8) Output the correlation filter that matches the next single frame image .

8. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 7, It is characterized in that Determine whether the candidate target area of ​​the current single-frame image is smaller than the set threshold. If the candidate target area of ​​the current single-frame image is smaller than the set threshold, the candidate target area of ​​the current single-frame image is used as the final tracked target of the current single-frame image, and the iterative operation ends. The value of t is increased by 1, and the process returns to step 3. Otherwise, the process returns to step 4, and the process continues to determine the next candidate target area of ​​the current single-frame image, including: Enter the features of the candidate regions in order from left to right and from top to bottom. ; Before each candidate target region of the current single frame image is iterated, restore is the initial default value and sets ; Iterate N times (i < N) in a loop. In each iteration, substitute the static guidance and dynamic guidance of the previous single-frame image into Equation (5) for calculation. If the value obtained from the calculation of Equation (5) in a certain iteration is less than the set threshold , then take this candidate target region as the target finally tracked in the current single-frame image, stop the iterative operation of subsequent candidate target regions, increment the value of t by 1, and return to Step 3; otherwise, return to Step 4 to continue the judgment of the next candidate target region in the current single-frame image. If the values obtained from the calculation of Equation (5) do not reach the set threshold , then take the candidate target region corresponding to the minimum value obtained from the calculation of Equation (5) as the target finally tracked in the current single-frame image.

9. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 1, It is characterized in that If the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is not the last single-frame image, return to step 3; if the single-frame image input in the dynamic and static joint guided correlation filter automatic tracking target model is the last single-frame image, then the final tracked target of each single-frame image is output, and the video tracking average rate FPS of each single-frame image is output. The video tracking average rate FPS of each single-frame image = the total time spent tracking the target in the sequence video taken by the drone divided by the total number of frames of the video image.

10. The method for automatically tracking a target using a correlation filter guided by dynamic and static joint guidance according to claim 3, It is characterized in that and Set to 1, Take 1.

Images