Correlation filtering tracking algorithm based on spatiotemporal regularization and context awareness

Abstract

The present invention discloses a correlation filtering tracking algorithm based on spatiotemporal regularization and context perception. It belongs to the field of computer vision, and the specific steps are: according to the initialization sample, determine the target template, initialize the context information, and perform frame sampling on it; perform HOG+CN feature extraction; introduce the target information and spatial regularization of the previous frame to constrain the update of the filter; solve the optimal spatiotemporal regularization and context-aware filter model, perform convolution operation with the sample in the Fourier domain, and then update the target template; repeat the above steps until the target tracking is completed. The present invention adds spatiotemporal information and context information during the training process to alleviate the influence of boundary effects and improve the overall performance of the algorithm; the tracking model uses the iterative method of the alternating multiplier tracking method to achieve the effect of real-time tracking; spatial regularization can alleviate the influence of model degradation; in the experimental results of the classic data sets OTB100 and TC128.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, in particular to the field of single target tracking, and specifically relates to a correlation filtering tracking algorithm based on spatiotemporal regularization and context perception. Background Art

[0002] The correlation filter tracking algorithm has become one of the mainstream research methods in the field of target tracking due to its high speed, but there may still be problems such as illumination changes, target occlusion, deformation, and violent movement during the target movement. The discriminant correlation filter is the mainstream research method in the correlation filter tracking algorithm, but the circulant matrix it introduces will affect the tracking effect. Although the performance of the discriminant correlation filter has gradually improved after the introduction of feature fusion, scale estimation, spatial regularization, and deep features, it still needs to be improved in terms of tracking speed and accuracy.

[0003] To solve this problem, the common approach is to optimize the algorithm model, such as spatial regularization, adding background information, iteratively updating filters to improve accuracy, deep features, etc., which can effectively improve the tracking speed and accuracy of the algorithm. However, it is difficult to balance tracking speed and accuracy, because the fast tracking speed is due to the introduction of circulant matrices, which also causes a decrease in tracking accuracy; on the other hand, the tracking accuracy is improved by optimizing the algorithm model, but it often leads to a decrease in tracking speed. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide an improved correlation filtering target tracking algorithm; specifically, a correlation filtering tracking algorithm based on spatiotemporal regularization and context perception; which can effectively improve the speed and accuracy of target tracking.

[0005] Technical solution: The relevant filtering tracking algorithm based on spatiotemporal regularization and context awareness described in the present invention is specifically implemented in the following steps:

[0006] (1) According to the initialization sample, determine the target template, initialize the context information, and perform frame sampling;

[0007] (2) Extract HOG+CN features from the context of the target;

[0008] (3) Introduce the target information of the previous frame and spatial regularization to constrain the update of the filter to prevent large differences in time and space, prevent overfitting, solve the target equation, and obtain the filter template;

[0009] (4) Solve the optimal spatiotemporal regularization and context-aware filter model, perform convolution operations with the sample in the Fourier domain, and the area with the highest response value score is the target area. Then, the target template is updated.

[0010] (5) Repeat steps 2 to 4 until target tracking is completed.

[0011] Furthermore, in step (1), the specific method of determining the target template based on the initialization sample, initializing the context information, and performing frame sampling is:

[0012] Initializing the target position: Given the target position of the initial frame, i.e. the x, y coordinates, length and width of the target, finally a rectangular frame enclosing the target;

[0013] The context information: in order to improve the accuracy of the algorithm, process the size change of the target during the tracking process, and ensure that more discrimination information is obtained, 4 background frames are fixedly selected in the target frame.

[0014] Furthermore, in step (2), the specific method of performing HOG+CN feature extraction on the selected target and context is: HOG and CN features are used, HOG features are used to count the directional gradients that appear locally in the image, and CN color features are used to count the target color, which are fused from the gradient and color to handle target occlusion and deformation.

[0015] Furthermore, in step (3), the time term and the space regularization term are introduced to constrain the target solution, prevent overfitting and model degradation, and constrain the target equation in time and space. The time term controls the difference between filters between the previous and next frames in time, and the space regularization term processes the distribution of weight filters in space, thereby highlighting the target area and suppressing the background area.

[0016] Furthermore, in step (4), the solution for the spatiotemporal regularization and context-aware correlation filter is obtained by minimizing the objective function, which can be expressed as:

[0017]

[0018] Among them, w is the filter template, y is the correlation output, and w t-1 represents the correlation filter of the (t-1)th frame, γ is the regularization coefficient; represents the spatial regularization term; Pw-w t-1 P represents the spatial term; Represents an item of contextual information.

[0019] Furthermore, in the minimization process, in order to obtain the optimal correlation filter w, the alternating multiplier direction method is used to solve the minimization problem; the specific process is as follows:

[0020] Introducing an auxiliary variable g, setting w = q, and setting the step size parameter to η, equation (1) can be expressed as:

[0021]

[0022] Where f is the Lagrange multiplier, γ is the penalty factor; Formula (2) can be transformed into the following form:

[0023]

[0024] The ADMM algorithm is used to solve the following sub-problems:

[0025]

[0026] The first line in equation (4) is rewritten in the Fourier domain using Parseval's theorem and then differentiated to obtain

[0027]

[0028] Closed-form solution:

[0029] The closed-form solution of the second line of equation (4) is:

[0030]

[0031] The update step size parameter is:

[0032] η (i+1) =min(η max ,ξη (i) ). (7)

[0033] Furthermore, in step (5), steps (2) to (4) are repeated, i.e., the same operation is performed on each frame until the end of the video sequence, feature extraction is performed on the initial frame and subsequent frames, the target equation is constructed, the optimal target function is solved, a convolution operation is performed, the target position is determined, and the template is updated. The target template of the initial frame is given, and the target template of the subsequent frame is updated by the target position of the previous frame.

[0034] Beneficial effects: Compared with the prior art, the present invention has the following features: the present invention adds spatiotemporal information and contextual information during the training process to alleviate the impact of boundary effects and improve the overall performance of the algorithm; the tracking model utilizes the iterative method of the alternating multiplier tracking method, and the sub-problems of the proposed model have closed-loop solutions, which can achieve the effect of real-time tracking; spatial regularization can alleviate the impact of model degradation; in the experimental results of the classic data sets OTB100 and TC128, it can be shown that the performance of the tracker proposed in this paper is better than other advanced trackers, and the overall algorithm improves accuracy and speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is an operation flow chart of the present invention;

[0036] Figure 2 It is a comparison chart of the overall performance of the present invention and other classic algorithms in the OTB100 data set comparison experiment, and the area under the success rate curve;

[0037] Figure 3 It is a comparison chart of the accuracy and area under the success rate curve of the overall performance of the present invention and other classic algorithms in the TC128 data set comparison experiment. DETAILED DESCRIPTION

[0038] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0039] As shown in the figure, the correlation filtering tracking algorithm based on spatiotemporal regularization and context awareness described in the present invention is specifically implemented in the following steps:

[0040] (1) According to the initialization sample, determine the target template, initialize the context information, and perform frame sampling;

[0041] (2) Extract HOG+CN features from the context of the target;

[0042] (3) Introduce the target information of the previous frame and spatial regularization to constrain the update of the filter to prevent large differences in time and space, prevent overfitting, solve the target equation, and obtain the filter template;

[0043] (4) Solve the optimal spatiotemporal regularization and context-aware filter model, perform convolution operations with the sample in the Fourier domain, and the area with the highest response value score is the target area. Then, the target template is updated.

[0044] (5) Repeat steps (2) to (4) until target tracking is completed.

[0045] Furthermore, in step (1), the specific method of determining the target template based on the initialization sample, initializing the context information, and performing frame sampling is:

[0046] Initializing the target position: Given the target position of the initial frame, that is, the x, y coordinates, length and width of the target, the target is finally framed by a rectangular frame;

[0047] The context information: In order to improve the accuracy of the algorithm, better handle the size change of the target during the tracking process, and ensure that more discrimination information is obtained, 4 background frames are fixedly selected in the target frame.

[0048] Furthermore, in step (2), the specific method of performing HOG+CN feature extraction on the selected target and context is: HOG and CN features are used, HOG features can be used to count the directional gradients that appear locally in the image, and CN color features are used to count the target color. The fusion of gradient and color can be better used to handle target occlusion and deformation.

[0049] Furthermore, in step (3), the time term and the space regularization term are introduced to constrain the target solution, prevent overfitting and model degradation, and constrain the target equation in time and space. The time term controls the difference between filters between the previous and next frames in time, and the space regularization term processes the distribution of weight filters in space, thereby highlighting the target area and suppressing the background area.

[0050] Further, in step (4), the solution to the spatiotemporal regularization and context-aware correlation filter is obtained by minimizing the objective function: (The process of constructing the spatiotemporal regularization and context-aware correlation filter is explained by taking one-dimensional space as an example. The objective function in the spatial domain can be expressed as:)

[0051]

[0052] Among them, w is the filter template, y is the correlation output, and w t-1 represents the correlation filter of the (t-1)th frame, γ is the regularization coefficient; represents the spatial regularization term; Pw-w t-1 P represents the spatial term; represents a context information item;

[0053] The form of the objective function is relatively complex, and the solution process requires the introduction of merging and decomposition of sub-problems through the ADMM method.

[0054] Furthermore, in the minimization process, in order to obtain the optimal correlation filter w, the alternating multiplier direction method is used to solve the minimization problem. The specific process is as follows:

[0055] Introducing an auxiliary variable g, setting w = q, and setting the step size parameter to η, equation (1) can be expressed as:

[0056]

[0057] Where f is the Lagrange multiplier and γ is the penalty factor. Formula (2) can be transformed into the following form:

[0058]

[0059] The ADMM algorithm is used to solve the following sub-problems;

[0060]

[0061] The first line of equation (4) is rewritten in the Fourier domain using Parseval's theorem, and further differentiated to obtain a closed-form solution:

[0062]

[0063] The closed-form solution of the second line of equation (4) is:

[0064]

[0065] The update step size parameter is:

[0066] η (i+1) =min(η max ,ξη (i) ) (7);

[0067] Through the target tracking method adopted above, the spatiotemporal information and contextual information in the tracking process can be effectively utilized, the influence of the boundary effect can be effectively alleviated, and the overall performance of the algorithm can be improved.

[0068] Furthermore, in step (5), steps (2) to (4) are repeated, i.e., the same operation is performed on each frame until the end of the video sequence, feature extraction is performed on the initial frame and subsequent frames, the target equation is constructed, the optimal target function is solved, a convolution operation is performed, the target position is determined, and the template is updated. The target template of the initial frame is given, and the target template of the subsequent frame is updated by the target position of the previous frame.

[0069] The purpose of the present invention is achieved in this way: according to the initialization sample, the target template is determined, the context information is initialized, and frame sampling is performed; the HOG+CN feature is extracted for the context of the target; the target information and spatial regularization of the previous frame are introduced to constrain the update of the filter to prevent large differences in time and space, prevent overfitting, solve the target equation, and obtain the filter template; solve the optimal spatiotemporal regularization and context-aware filter model, perform convolution operation with the sample in the Fourier domain, the response value score is the highest The target area, and then update the target template; repeat steps (2) to (4) until the target tracking is completed.

[0070] The STCACF algorithm is compared with cutting-edge tracking algorithms in the OTB-2013 and OTB-2015 datasets, namely ECO_HC, STAMPLE_CA, LADCF-HC, ARCF-H, SRDCF, BACF, AutoTrack, and STAPLE, and compared with the OTB-2015 and TC128 datasets. These two datasets are mainstream datasets in the field of target tracking. OTB-2015 contains 100 video sequences and TC128 contains 128 video sequences. These video sequences contain most of the situations and challenges encountered in target tracking in real life, such as illumination changes, scale changes, deformation, etc. Therefore, these two datasets can reflect the tracking status of the tracker in various situations and reflect the performance of the algorithm.

[0071] OTB-2013, OTB-2015 and TC128 usually use one-time evaluation as the evaluation indicator. The one-time evaluation includes two evaluation results: the success rate graph and the precision graph of the tracking algorithm. At the same time, the area under the success rate curve is also one of the main evaluation indicators of the tracking algorithm performance. Figure 2 As can be seen from the figure, STCACF achieved the highest AUC score (65.4%) and the highest accuracy (86.5%) in the OTB2015 dataset; Figure 3 As can be seen, STCACF achieved the highest accuracy (74.4%) and the second AUC score (55%) in the TC128 dataset; this shows that the framework of the algorithm is effective.

[0072] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A correlation filtering tracking method based on spatiotemporal regularization and context awareness, characterized in that: The specific implementation steps are as follows: (1) According to the initialization sample, determine the target template, initialize the context information, and perform frame sampling; (2) Extract HOG+CN features from the context information of the target; (3) Introduce the target information of the previous frame and spatial regularization to constrain the update of the filter to prevent large differences in time and space, prevent overfitting, solve the target equation, and obtain the filter template; (4) Solve the optimal spatiotemporal regularization and context-aware filter model, perform convolution operations with the sample in the Fourier domain, and the area with the highest response value score is the target area. Then update the target template. In step (4), the solution for the spatiotemporal regularization and context-aware correlation filter is obtained by minimizing the objective function; wherein the minimization objective function can be expressed as: Among them, w is the filter template, y is the correlation output, and w t-1 represents the correlation filter of the (t-1)th frame, γ is the regularization coefficient; In the process of minimization, in order to obtain the optimal correlation filter w, the alternating multiplier direction method is used to solve the minimization problem; the specific process is as follows: Introducing an auxiliary variable q, setting w = q, and setting the step size parameter to η, equation (1) can be expressed as: Where f is the Lagrange multiplier, introduced Formula (2) can be transformed into the following form: The ADMM algorithm is used to solve the following sub-problems: The first line in equation (4) is rewritten in the Fourier domain using Parseval's theorem and then differentiated to obtain Closed-form solution: The closed-form solution of the second line of equation (4) is: The update step size parameter is: or (i+1) =min(η max ,xy (i) ) (7); (5) Repeat steps (2) to (4) until target tracking is completed.

2. The correlation filtering tracking method based on spatiotemporal regularization and context awareness according to claim 1, characterized in that: In step (1), according to the initialization sample, the target template is determined, the context information is initialized, and frame sampling is performed: Determining the target template according to the initialization sample specifically includes: given the target position of the initial frame, that is, the x, y coordinates, length and width of the target, and finally a rectangular frame enclosing the target; The initialization of context information and frame sampling thereof are specifically as follows: in order to improve the accuracy of the algorithm, better handle the size change of the target during the tracking process, and ensure that more discrimination information is obtained, 4 background frames are fixedly selected in the target frame.

3. The correlation filtering tracking method based on spatiotemporal regularization and context awareness according to claim 1, characterized in that: In step (2), the specific method of performing HOG+CN feature extraction on the selected target and context is: using HOG and CN features; The HOG feature is used to count the directional gradients that appear locally in the image, and the CN feature is used to count the target color. The gradient and color are fused to handle target occlusion and deformation.

4. The correlation filtering tracking method based on spatiotemporal regularization and context awareness according to claim 1, characterized in that: In step (3), the target information and spatial regularization of the previous frame are introduced to constrain the update of the filter. Specifically, the target information and spatial regularization of the previous frame are introduced to constrain the target solution, prevent overfitting and model degradation, and constrain the target equation in time and space; wherein the time term controls the difference in filters between the previous and next frames in time, and the spatial regularization term processes the distribution of the weighted filter in space, thereby highlighting the target area and suppressing the background area.

5. The correlation filtering tracking method based on spatiotemporal regularization and context awareness according to claim 1, characterized in that: In step (5), steps (2) to (4) are repeated, i.e., the same operation is performed on each frame until the end of the video sequence, feature extraction is performed on the initial frame and subsequent frames, the target equation is constructed, the optimal target function is solved, a convolution operation is performed, the target position is determined, and the template is updated. The target template of the initial frame is given, and the target template of the subsequent frames is updated by the target position of the previous frame.

Images