A target tracking method based on adaptive context awareness

Abstract

The present invention proposes a target tracking method based on adaptive context perception, including: initializing a Kalman filter, a position correlation filter and a scale correlation filter; using the Kalman filter and the position correlation filter in the tth frame to estimate the target position in the t+1th frame respectively, and determining the target position in the t+1th frame according to the tracking confidence of the position correlation filter; using the scale correlation filter in the tth frame to estimate the target scale in the t+1th frame; updating the Kalman filter; when the tracking confidence is greater than a certain threshold, using the position, scale and high-credibility interference background area of ​​the optimal target area to update the position correlation filter, and updating the scale correlation filter. The present invention uses the high-credibility interference background to update the correlation filter, thereby improving the discrimination of the target and the background, and only updating the correlation filter model when the tracking is credible, reducing the model drift and the number of updates, and improving the tracking speed.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, and in particular relates to a target tracking method based on adaptive context perception. Background Art

[0002] Visual tracking is a cutting-edge research area in computer vision, and has a wide range of applications in robotics, traffic monitoring, industrial and agricultural production, medicine, and military. Currently, domestic and foreign scholars are mainly concerned with improving tracking robustness and accuracy under conditions of target occlusion, target rotation, changes in light intensity and scale, and real-time tracking of target tracking systems in practical applications.

[0003] In recent years, the target tracking method based on correlation filter has begun to rise. Due to its high computational efficiency, it has gradually shown superior performance. Bolte D et al. first applied correlation filtering to target tracking in their paper “Visual object tracking using adaptive correlation filters” and proposed the tinitut output sut of squared error filter (tOSSE) algorithm using single-channel grayscale features. Henriques J et al. introduced the concepts of circulant matrix and kernel on the basis of tOSSE algorithm in their paper “Exploiting the circulant structure of tracking-by-detection with kernels filters” and proposed the Exploiting the circulant structure of tracking-by-detection with kernels (CSK) algorithm. Later, based on the CSK algorithm, the single-channel features were converted into multi-channel Histogram of Oriented Gradients (HOG) features and the Kernelized correlation filters (KCF) target tracking algorithm was proposed, which further improved the tracking accuracy and real-time performance. Danelljan t et al. proposed using color naming (CN) features to improve the target appearance model in the paper “Adaptive color attributes for real-tite visual tracking”.

[0004] However, since the above four algorithms are mainly focused on improving the performance of target position estimation, they are prone to tracking drift when the target scale changes significantly. To address this problem, Tuellert et al. proposed the CACF (Context Aware Correlation Filter Tracking) algorithm that combines global context for background training: a background area is adaptively selected near the target position, added to the tracking filter, and background supervision training is performed on them. This algorithm not only does not affect the tracking speed, but also makes full use of background information. The disadvantage is that the entire context-aware area is regarded as a whole during the tracking process, and the contribution of different partitions to target tracking is not considered, which will weaken the role of context, the filter tracking model obtained is not robust enough, and the target occlusion problem is not solved. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a target tracking method based on adaptive context perception, and the technical solution adopted is as follows:

[0006] A target tracking method based on adaptive context perception comprises the following steps:

[0007] Step 1: Initialize the Kalman filter, the position correlation filter and the scale correlation filter according to the position and scale information of the target area in the first frame of the video sequence;

[0008] Step 2: Use the Kalman filter and position correlation filter updated in the tth frame to estimate the target position in the t+1th frame, respectively, and obtain the Kalman filter estimation value and the correlation filter estimation value; calculate the tracking confidence E of the correlation filter t+1 , when E t+1 When it is greater than a certain threshold, the correlation filter estimation value is used as the target position in the t+1th frame, otherwise the Kalman filter estimation value is used as the target position in the t+1th frame; the scale correlation filter is used to estimate the target scale in the t+1th frame;

[0009] Step 3: Update the Kalman filter according to the target position in the t+1th frame; if E t+1 When it is greater than a certain threshold, the position-related filter is updated based on the response matrix of the position-related filter by fusing the target position and scale in the t+1th frame with the interference background area with high credibility, and the scale-related filter is updated using the target position and scale information in the t+1th frame. Otherwise, the position-related filter and the scale-related filter remain unchanged.

[0010] Step 4: Repeat steps 2 and 3 until the last frame of the video sequence.

[0011] Furthermore, in step 1, the coefficient w of the position-dependent filter is obtained by optimizing the following formula: t :

[0012]

[0013] Among them, B 1,t is the characteristic circulant matrix formed by cyclic shift of the target area, {B 2,t ,B 3,t ,B 4,t ,B 5,t} respectively represent the characteristic circulant matrices formed by cyclic displacement of the four background regions adjacent to the target region, λ1 and λ2 are the regularization parameters of the corresponding items, y t is the response label matrix of Gaussian distribution, whose peak position corresponds to the center position of the target;

[0014] By using the diagonal property of the circulant matrix, the above equation is transformed into the frequency domain and solved, and we get:

[0015]

[0016] in, They are w t ,y t The frequency domain form of is the frequency domain form of the target area characteristics, is the frequency domain form of the background area characteristics, express The conjugate of , ⊙ represents the Hadamard product.

[0017] Furthermore, in step 1, the coefficient h of the scale-dependent filter is obtained by optimizing the following formula: l,t :

[0018]

[0019] Among them, s target regions of different scales are extracted with the target position of the tth frame as the center, and the HOG features of the target region at each scale are extracted and arranged into s×d feature vectors, where d is the dimension of the feature vector and h is l,t 、f l,t h t 、f t The lth dimension of , * represents the correlation operation, λ is the regularization parameter, g t is a 1×s Gaussian regression label vector whose peak position corresponds to the optimal scale of the target;

[0020] After converting the above equation to the frequency domain, we get:

[0021]

[0022] in, h l,t 、f l,t , g t The frequency domain form of g t , The conjugation of.

[0023] Furthermore, in step 2, the Kalman filter posterior estimate of the target position in the t+1th frame is:

[0024]

[0025] Among them, e t+1 =[x t+1 ,y t+1 ,v x,t+1 ,v y,t+1 ] T is the posterior estimate of the target state vector in the t+1th frame, x t+1 ,y t+1 is the center coordinate of the target, v x,t+1 、v y,t+1 For the target at x t+1 ,y t+1 The velocity component in the z direction t+1 is the measurement value, H is the observation matrix; is the prior estimate of the target state vector in the t+1th frame and A is the state transfer matrix, G(σ) is the Gaussian noise with standard deviation σ; K is the Kalman filter gain and is the prior estimated covariance matrix of the t+1th frame and R and Q are both constant matrices.

[0026] Furthermore, in step 2, the calculation process of the correlation filter estimation value of the target position in the t+1th frame is:

[0027] Taking the target position in the tth frame as the center, select the same size image area in the t+1th frame to extract the HOG features and convert them to the frequency domain to obtain pass Get the position-dependent response matrix L in the t+1th frame t+1 :

[0028]

[0029] Among them, F -1 represents inverse Fourier transform; L t+1 The corresponding position of the maximum value is the correlation filter estimate of the target position in the t+1th frame.

[0030] Furthermore, in step 2, the tracking confidence E of the position-dependent filter t+1 The average peak correlation energy is measured as:

[0031]

[0032] Among them, E t+1 represents the average peak correlation energy in the t+1th frame, max(L t+1 )、min(L t+1 ) are the position-dependent response matrices L t+1 The maximum and minimum values ​​in L m,n,t+1 For L t+1 The position-related response value of the mth row and nth column in , mean means the mean;

[0033] when When , the correlation filter estimation value is used as the target position in the t+1th frame, otherwise the Kalman filter estimation value is used as the target position in the t+1th frame, and β is the proportional value.

[0034] Furthermore, in step 2, s target regions of different scales are obtained with the target position in the t+1th frame as the center, and the HOG features of the target region at each scale are extracted and arranged into s×d feature vectors. The scale-related response vector Y is obtained according to the following formula: s,t+1 :

[0035]

[0036] in, is the t+1th frame The molecular conjugation, D t+1 for The denominator is Y s,t+1 The scale corresponding to the maximum value is used as the scale of the target in the t+1th frame.

[0037] Furthermore, in step 3, the Kalman filter is updated according to the target position in the t+1th frame: P t+1 is the prior estimated covariance matrix of the t+1th frame;

[0038] when When , the position-related filter and scale-related filter are updated respectively, and the update formula of the position-related filter is:

[0039]

[0040] Where η is the learning rate, w t+1 The frequency domain form of w t+1 By optimizing the following formula:

[0041]

[0042] Among them, the position-dependent response matrix L of the t+1th frame is selected t+1 Less than max(L t+1 ) and the corresponding positions of k response values ​​greater than the set threshold are taken as the center of the interference background area, G r,t+1 represents the characteristic circulant matrix of the rth interference background area after cyclic shift, and λ3 is the regularization parameter of the item; after converting the above formula to the frequency domain, the solution is:

[0043]

[0044] in, is the frequency domain form of the interference background area characteristics, for conjugation of;

[0045] The update formula of the scale-dependent filter is:

[0046]

[0047]

[0048] The beneficial effects of the present invention are:

[0049] (1) The present invention uses both the Kalman filter and the correlation filter to estimate the target position. When the tracking confidence of the correlation filter is low, the position estimation of the Kalman filter shall prevail. When the tracking confidence is high, the position estimation of the correlation filter shall prevail. The two filters assist each other.

[0050] (2) The present invention integrates the center, scale and high-confidence interference background area of ​​the optimal target area to update the position-related filter, and adaptively updates the scale, thereby improving the discrimination of the target and the background, reducing model drift and the number of updates, and improving the tracking speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the process of the present invention;

[0052] Figure 2 is a tracking accuracy diagram in an embodiment of the present invention;

[0053] Figure 3 Graph showing the tracking success rate in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention will now be described in further detail with reference to the accompanying drawings.

[0055] like Figure 1As shown, the present invention mainly comprises the following steps:

[0056] (1) Initialize the Kalman filter, the position correlation filter and the scale correlation filter according to the position and scale information of the target area in the first frame of the video sequence.

[0057] First, initialize the state transfer matrix and other related parameters of the Kalman filter.

[0058] Then, the coefficient w of the position-dependent filter is obtained by optimizing the following formula: t :

[0059]

[0060] Among them, B 1,t is the characteristic circulant matrix formed by cyclic shift of the target area, {B 2,t ,B 3,t ,B 4,t ,B 5,t} respectively represent the characteristic circulant matrices formed by cyclic displacement of the four background regions adjacent to the target region, λ1 and λ2 are the regularization parameters of the corresponding items, y t is the response label matrix of Gaussian distribution, whose peak position corresponds to the center position of the target;

[0061] By using the diagonal property of the circulant matrix, the above equation is transformed into the frequency domain and solved, and we get:

[0062]

[0063] in, They are w t ,y t The frequency domain form of is the frequency domain form of the target area characteristics, is the frequency domain form of the background area characteristics, express The conjugate of , ⊙ represents the Hadamard product.

[0064] Finally, the coefficient h of the scale-dependent filter is obtained by optimizing the following formula: l,t :

[0065]

[0066] Among them, s target regions of different scales are extracted with the target position of the tth frame as the center, and the HOG features of the target region at each scale are extracted and arranged into s×d feature vectors. Among them, d is the dimension of the feature vector, h is l,t 、f l,t h t 、f tThe lth dimension of , * represents the correlation operation, λ is the regularization parameter, g t is a 1×s Gaussian regression label vector whose peak position corresponds to the optimal scale of the target;

[0067] After converting the above equation to the frequency domain, we get:

[0068]

[0069] in, h l,t 、f l,t , g t The frequency domain form of g t , The conjugation of.

[0070] (2) Using the Kalman filter and position correlation filter updated in the tth frame, estimate the target position in the t+1th frame respectively, and obtain the Kalman filter estimation value and the correlation filter estimation value; calculate the tracking confidence E of the correlation filter t+1 , when E t+1 When it is greater than a certain threshold, the correlation filter estimation value is used as the target position in the t+1th frame, otherwise the Kalman filter estimation value is used as the target position in the t+1th frame; the scale correlation filter is used to estimate the target scale in the t+1th frame.

[0071] First, the Kalman filter estimate of the target position in t+1 is:

[0072]

[0073] Among them, e t+1 =[x t+1 ,y t+1 ,v x,t+1 ,v y,t+1 ] T is the posterior estimate of the target state vector in the t+1th frame, is the prior estimate of the target state vector in the t+1th frame, x t+1 ,y t+1 is the center coordinate of the target, v x,t+1 、v y,t+1 For the target at x t+1 ,y t+1 The velocity component in the z direction t+1 is the measurement value, H is the observation matrix; A is the state transfer matrix, G(σ) is Gaussian noise with standard deviation σ; is the prior estimated covariance matrix of the t+1th frame, K is the Kalman filter gain, and R and Q are both constant matrices.

[0074] Then, the calculation process of the relevant filter estimate of the target position in the t+1th frame is:

[0075] Taking the target position in the tth frame as the center, select the same size image area in the t+1th frame to extract the HOG features and convert them to the frequency domain to obtain pass Get the position-dependent response matrix L in the t+1th frame t+1 :

[0076]

[0077] Among them, F -1 represents inverse Fourier transform; L t+1 The corresponding position of the maximum value is the correlation filter estimate of the target position in the t+1th frame.

[0078] The tracking confidence of the correlation filter estimate is calculated using the average peak correlation energy E t+1 To measure, E t+1 The criterion reflects the oscillation degree of the response graph. The target occlusion and other states can be further judged according to the average peak correlation energy. t+1 When it decreases suddenly, it means that the target is lost or blocked.

[0079]

[0080] Among them, E t+1 represents the average peak correlation energy in the t+1th frame, max(L t+1 )、min(L t+1 ) are the position-dependent response matrices L t+1 The maximum and minimum values ​​in L m,n,t+1 For L t+1 The position-related response value of the mth row and nth column in , mean means the mean;

[0081] when When , the correlation filter estimation value is used as the target position in the t+1th frame, otherwise the Kalman filter estimation value is used as the target position in the t+1th frame, and β is the proportional value.

[0082] Finally, the scale is estimated. Taking the target position in the t+1th frame as the center, s target regions of different scales are obtained. The HOG features of the target region at each scale are extracted and arranged into s×d feature vectors. The scale-related response vector Y is obtained according to the following formula. s,t+1 :

[0083]

[0084] in, is the t+1th frame The molecular conjugation, D t+1 for The denominator is Y s,t+1 The scale corresponding to the maximum value is used as the scale of the target in the t+1th frame.

[0085] (3) Update the Kalman filter according to the target position in the t+1th frame; if E t+1 When it is greater than a certain threshold, according to the response matrix of the position-related filter, the center and scale of the optimal target area are fused with the high-confidence interference background area in the t+1th frame to update the position-related filter, and the optimal target position and scale information in the t+1th frame are used to update the scale-related filter, otherwise the position-related filter and the scale-related filter remain unchanged.

[0086] Update the Kalman filter based on the target position in frame t+1: P t+1 is the prior estimated covariance matrix of the t+1th frame;

[0087] when When , the position-related filter and scale-related filter are updated respectively, and the update formula of the position-related filter is:

[0088]

[0089] Where η is the learning rate, w t+1 The frequency domain form of w t+1 By optimizing the following formula:

[0090]

[0091] Among them, the position-dependent response matrix L of the t+1th frame is selected t+1 Less than max(L t+1 ) and the corresponding positions of k response values ​​greater than the set threshold are taken as the center of the interference background area, G r,t+1 represents the characteristic circulant matrix of the rth interference background area after cyclic shift, and λ3 is the regularization parameter of the item; after converting the above formula to the frequency domain, the solution is:

[0092]

[0093] in, is the frequency domain form of the interference background area characteristics, for conjugation of;

[0094] The update formula of the scale-dependent filter is:

[0095]

[0096]

[0097] (4) Repeat steps (2) and (3) until the last frame of the video sequence.

[0098] Figure 2 and Figure 3 The present invention is applied to the tracking accuracy graph and success rate graph on the video sequence with a resolution of 640×480. The position error in the accuracy graph is the average pixel distance between the center position of the tracked target and the center position of the manually annotated target. The position error threshold of 20 pixels is used as the accuracy evaluation index. The tracking algorithms are ranked according to the area under each curve in the success rate graph, and the performance of the tracking algorithm is measured by the area overlap ratio between the target bounding box obtained by the statistical tracking algorithm and the manually annotated target bounding box. In order to verify the robustness of the algorithm of the present invention, the selected video set is OTB100. In order to more intuitively evaluate the overall performance of the algorithm of the present invention, a large number of comparative experiments are carried out on the algorithm of the present invention (OURS) and the current mainstream algorithms DCF_CA, KCF and DSST. As can be seen from the figure, the success rate of the method of the present invention is 0.711, and the accuracy rate is 0.781. The scores are higher than those of other algorithms, which proves the robustness of the algorithm of the present invention.

[0099] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A target tracking method based on adaptive context awareness, characterized in that: The following steps are involved: Step 1: Initialize the Kalman filter, position-correlation filter, and scale-correlation filter according to the position and scale information of the target area in the first frame of the video sequence; in step 1, the coefficient w of the position-correlation filter is obtained by optimizing the following formula: t : Among them, B 1,t is the characteristic circulant matrix formed by cyclic shift of the target area, {B 2,t ,B 3,t ,B 4,t ,B 5,t } respectively represent the characteristic circulant matrices formed by cyclic displacement of the four background regions adjacent to the target region, λ1 and λ2 are the regularization parameters of the corresponding items, y t is the response label matrix of Gaussian distribution, whose peak position corresponds to the center position of the target; By using the diagonalization property of the circulant matrix, the above equation is transformed into the frequency domain and solved, and we get: in, They are w t ,y t The frequency domain form of is the frequency domain form of the target area characteristics, is the frequency domain form of the background area characteristics, express The conjugate of , ⊙ represents the Hadamard product; In step 1, the coefficient h of the scale-dependent filter is obtained by optimizing the following formula: l,t : Among them, s target regions of different scales are extracted with the target position of the tth frame as the center, and the HOG features of the target region at each scale are extracted and arranged into s×d feature vectors, where d is the dimension of the feature vector and h is l,t 、f l,t h t 、f t The lth dimension of , * represents the correlation operation, λ is the regularization parameter, g t is a 1×s Gaussian regression label vector whose peak position corresponds to the optimal scale of the target; After converting the above equation to the frequency domain, we get: in, h l,t 、f l,t , g t The frequency domain form of g t , conjugate; Step 2: Use the Kalman filter and position correlation filter updated in the tth frame to estimate the target position in the t+1th frame, respectively, and obtain the Kalman filter estimation value and the correlation filter estimation value; calculate the tracking confidence E of the correlation filter t+1 , when E t+1 When it is greater than a certain threshold, the correlation filter estimate is used as the target position in the t+1th frame, otherwise the Kalman filter estimate is used as the target position in the t+1th frame; the scale correlation filter is used to estimate the target scale in the t+1th frame; in step 2, the Kalman filter posterior estimate of the target position in the t+1th frame is: Among them, e t+1 =[x t+1 ,y t+1 ,v x,t+1 ,v y,t+1 ] T is the posterior estimate of the target state vector in the t+1th frame, x t+1 ,y t+1 is the center coordinate of the target, v x,t+1 、v y,t+1 For the target at x t+1 ,y t+1 The velocity component in the z direction t+1 is the measurement value, H is the observation matrix; is the prior estimate of the target state vector in the t+1th frame and A is the state transfer matrix, G(σ) is the Gaussian noise with standard deviation σ; K is the Kalman filter gain and is the prior estimated covariance matrix of the t+1th frame and R and Q are both constant matrices; In step 2, the calculation process of the correlation filter estimate value of the target position in the t+1th frame is: Taking the target position in the tth frame as the center, select the same size image area in the t+1th frame to extract the HOG features and convert them to the frequency domain to obtain pass Get the position-dependent response matrix L in the t+1th frame t+1 : Among them, F -1 represents inverse Fourier transform; L t+1 The corresponding position of the maximum value is the correlation filter estimate of the target position in the t+1th frame; In step 2, the tracking confidence E of the position-dependent filter t+1 The average peak correlation energy is measured as: Among them, E t+1 represents the average peak correlation energy in the t+1th frame, max(L t+1 )、min(L t+1 ) are the position-dependent response matrices L t+1 The maximum and minimum values ​​in L m,n,t+1 For L t+1 The position-related response value of the mth row and nth column in , mean means the mean; when When , the correlation filter estimation value is used as the target position in the t+1th frame, otherwise the Kalman filter estimation value is used as the target position in the t+1th frame, and β is the proportional value; In step 2, s target regions of different scales are obtained with the target position in the t+1th frame as the center, and the HOG features of the target region at each scale are extracted and arranged into s×d feature vectors. The scale-related response vector Y is obtained according to the following formula: s,t+1 : in, is the t+1th frame The molecular conjugation, D t+1 for The denominator is Y s,t+1 The scale corresponding to the maximum value is used as the scale of the target in the t+1th frame; Step 3: Update the Kalman filter according to the target position in the t+1th frame; if E t+1 When it is greater than a certain threshold, the position-related filter is updated based on the response matrix of the position-related filter by fusing the target position and scale in the t+1th frame with the interference background area with high credibility, and the scale-related filter is updated using the target position and scale information in the t+1th frame. Otherwise, the position-related filter and the scale-related filter remain unchanged. In step 3, the Kalman filter is updated based on the target position in the t+1th frame: P t+1 is the prior estimated covariance matrix of the t+1th frame; when When , the position-related filter and scale-related filter are updated respectively, and the update formula of the position-related filter is: Where η is the learning rate, w t+1 The frequency domain form of w t+1 By optimizing the following formula: Among them, the position-dependent response matrix L of the t+1th frame is selected t+1 Less than max(L t+1 ) and the corresponding positions of k response values ​​greater than the set threshold are taken as the center of the interference background area, G r,t+1 represents the characteristic circulant matrix formed by cyclic shift of the rth interference background area, and λ3 is the regularization parameter of the item; after converting the above formula to the frequency domain, we can get: in, is the frequency domain form of the interference background area characteristics, for conjugation of; The update formula of the scale-dependent filter is: Step 4: Repeat steps 2 and 3 until the last frame of the video sequence.

Images