A model-optimized target tracking method

Abstract

The application provides a model-optimized target tracking method and relates to the field of computer vision. The application introduces time regularization to effectively deal with the boundary effect problem and improve the accuracy of target tracking. In view of the complexity of the objective function, the application converts the objective function into a frequency domain and adopts an alternating direction multiplier method (ADMM) to effectively optimize, so that each sub-problem has a corresponding optimal solution. In addition, in terms of feature representation, the application combines traditional manual features and deep convolution features to obtain more semantic information and improve the discrimination performance of the filter. A large number of experiments show that the algorithm proposed in the application performs well on many advanced trackers.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision, and in particular to a target tracking method with model optimization. Background Technology

[0002] Visual object tracking is one of the hottest problems in the field of computer vision, attracting widespread attention and being widely applied in practical applications such as intelligent video surveillance, autonomous vehicles, virtual reality, high-speed translational positioning, and drone tracking. Despite numerous efforts over the past decade, various challenging problems remain, such as occlusion, rapid motion, and deformation.

[0003] Recently, mainstream target tracking methods are generally divided into two categories: one is tracking methods based on the discriminant correlation filter framework, and the other is tracking methods based on deep learning networks. Among them, the DCF-based method has achieved good performance due to its efficiency in the Fourier domain. Therefore, improved target tracking methods based on the DCF framework have been proposed one after another, such as methods based on multi-scale strategies, multi-feature fusion methods, spatiotemporal context methods, as well as methods based on particle filtering and sparse representation.

[0004] Despite significant improvements to DCF-based methods, several drawbacks remain. First, cyclic shift sampling within the DCF framework is always susceptible to boundary effects, easily leading to model drift and reduced tracking performance. Second, negative training samples surrounding the target object are underutilized, making it difficult to extract target information from similar backgrounds; thus, complex backgrounds further degrade the filter's robustness. Third, most traditional DCF-based trackers typically use simple, shallow, handcrafted features to represent the target. These features often struggle to capture more complex semantic information, resulting in inaccurate target localization. These shortcomings will suppress the discriminative performance of DCF-based trackers from various angles. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a model-optimized target tracking method. The target is tracked based on the optimized model. The feature extraction method that combines manual features and convolutional features in the tracking framework effectively handles the boundary effect problem and improves the accuracy of target tracking.

[0006] This invention is implemented as follows:

[0007] A target tracking method with model optimization includes: the construction and optimization process of the target tracking model and the target tracking process of the video sequence;

[0008] The process of constructing and optimizing the target tracking model includes:

[0009] Temporal regularization is introduced to effectively handle boundary effect problems, and a background-aware target tracking model based on temporal regularization is constructed. The formula of the background-aware target tracking model based on temporal regularization is as follows:

[0010]

[0011] Where D represents the total number of feature channels, and d = {1, 2, 3, ..., D}, B is the shearing matrix that clusters sample information onto the target region, and x d f is the target feature of the d-th channel. d It is the correlation filter of the d-th channel, f d-1 Here, f represents the correlation filter for the (d-1)th channel, y is the ideal output in Gaussian form, α1 and α2 represent regularization parameters, and * represents convolution operation. It is the introduced spatiotemporal regularization term that enables the filter to establish a spatiotemporal relationship between the current frame and historical frames;

[0012] The background-aware target tracking model based on time regularization is transformed to the frequency domain, and then optimized using the alternating direction multiplier method. It is then decomposed into M sub-problems, leading to a local optimum. Finally, the Sherman-Morrison theorem is used to optimize the calculation formula for the local optimum, resulting in the optimized target tracking model.

[0013]

[0014] Where ^ denotes the discrete Fourier transform of the signal, μ is the penalty parameter for the introduced error term, and M is the number of subproblems. For tags For the sample for for T represents the conjugate transpose operation of a complex vector or matrix;

[0015] The target tracking process for the video sequence includes:

[0016] Step 1: Input a video sequence and determine if it is the first frame. If it is, initialize the target's position and the optimized target tracking model. If not, proceed to Step 2.

[0017] Step 2: Based on the target location in the (t-1)th frame image, obtain the target region in the tth frame. Using this region as the target region, extract the HOG features, color features, and hierarchical convolution features of the target region. Perform fusion processing on the HOG features, CN features, and hierarchical convolution features of the tth frame to obtain the fused feature X, where t is a natural number.

[0018] Step 3: Based on the fused feature X and the optimized time-regularized background-aware target tracking model, predict the position of the target in the t-th frame image and output the result. The time-regularized background-aware target tracking model has been trained in advance on the target image in the (t-1)-th frame image.

[0019] Step 4: Update the optimized target tracking model based on the feature map of frame t;

[0020] Step 5: Determine if there is a next frame. If yes, return to step 1; otherwise, end the process.

[0021] Furthermore, in step 2, the hierarchical convolutional features employ the conv5-4 and conv4-4 layers from VGG-19.

[0022] Furthermore, in step 4, the Lagrange parameters are updated using an online adaptive template update strategy, and the filter model update formula is as follows:

[0023]

[0024] Where m and m-1 represent the m-th and (m-1)-th frames of the video sequence, respectively, and λ represents the learning efficiency, λ = 0.0192. The filter template learned for the current frame. This is the filter template obtained from the previous frame.

[0025] The present invention has the following advantages:

[0026] By incorporating a spatiotemporal regularization module into the background-aware correlation filter framework, the boundary effect is effectively addressed, resulting in better robustness. The proposed optimization model can be efficiently solved using the Alternating Direction Multiplier Method (ADMM) optimization algorithm, accelerating the algorithm's convergence. Furthermore, from a feature representation perspective, the proposed model effectively combines higher-level deep features with shallow handcrafted features, enabling the filter to acquire more abstract semantic information and thus improving its discriminative ability. Compared to many excellent trackers, the tracker proposed in this invention exhibits superior performance. Attached Figure Description

[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0028] Figure 1 This is a schematic flowchart of the method of the present invention;

[0029] Figure 2 This is a flowchart illustrating the target tracking process of the present invention. Detailed Implementation

[0030] This invention provides a model-optimized target tracking method that effectively addresses the boundary effect problem and improves the accuracy of target tracking.

[0031] like Figure 1 As shown, an embodiment of the present invention provides a model-optimized target tracking method, including the construction and optimization process of a target tracking model and the target tracking process of a video sequence;

[0032] The process of constructing and optimizing the target tracking model includes:

[0033] Model Establishment: Temporal regularization is introduced to effectively handle boundary effect problems, and a background-aware target tracking model based on temporal regularization is constructed. The formula for the time-regularized background-aware target tracking model is as follows:

[0034]

[0035] Where D represents the total number of feature channels, and d = {1, 2, 3, ..., D}, B is the shearing matrix that clusters sample information onto the target region, and x d f is the target feature of the d-th channel. d It is the correlation filter of the d-th channel, f d-1 Here, f represents the correlation filter for the (d-1)th channel, y is the ideal output in Gaussian form, α1 and α2 represent regularization parameters, and * represents convolution operation. It is the introduced spatiotemporal regularization term that enables the filter to establish a spatiotemporal relationship between the current frame and historical frames;

[0036] Model optimization: Equation (1) is transformed to the frequency domain and optimized using the ADMM method to obtain a local optimum. Equation (1) is transformed into the frequency domain form corresponding to Equation (2):

[0037]

[0038]

[0039] Where ^ denotes the discrete Fourier transform of the signal, and F denotes the orthogonal matrix. This represents auxiliary variable parameters. I represents the Kronecker product. D It is a D×D identity matrix, and T represents the conjugate transpose operation of a complex vector or matrix.

[0040] The specific augmented Lagrange form can be restated as the following equation:

[0041]

[0042] Where μ is the introduced error term penalty parameter, μ is calculated as μ(i+1) = min(μ max ,βμ (i) Updated in the manner of ), where β = 10, μ max =10 3 , It is a Lagrange multiplier with a size of DT×1. The ADMM optimization method is used to iteratively solve equation (3), and each subproblem has a corresponding closed solution.

[0043] Solve the subproblem f * :

[0044]

[0045] Where g and ρ can be obtained through the following inverse Fourier transform operation:

[0046]

[0047] Solve subproblem g * :

[0048]

[0049] Consider dividing formula (6) into M subproblems.

[0050]

[0051] Equation (7) can be solved to obtain:

[0052]

[0053] Using the Sherman–Morrison theorem, the calculation formula (8) is optimized to obtain formula (9).

[0054]

[0055] in, The variables mentioned above have no practical meaning; they are simply combined for ease of calculation.

[0056] The final formula (9) is the optimized target tracking model. The target's position information, i.e. the result of target tracking, can be obtained by solving formula (9).

[0057]

[0058] Where ^ denotes the discrete Fourier transform of the signal, μ is the introduced error term penalty parameter, and M is the number of subproblems. For tags For the sample for for T represents the conjugate transpose operation of a complex vector or matrix.

[0059] like Figure 2 As shown, the target tracking process of the video sequence includes:

[0060] Step 1: Input a video sequence and determine if it is the first frame. If it is, initialize the target's position and the optimized target tracking model. If not, proceed to Step 2.

[0061] Step 2: Based on the target location in the (t-1)th frame image, obtain the target region in the tth frame. Using this region as the target region, extract the HOG features, color features, and hierarchical convolution features of the target region. Perform fusion processing on the HOG features, CN features, and hierarchical convolution features of the tth frame to obtain the fused feature X, where t is a natural number.

[0062] Step 3: Based on the fused feature X and the optimized time-regularized background-aware target tracking model, predict the position of the target in the t-th frame image and output the result. The time-regularized background-aware target tracking model has been trained in advance on the target image in the (t-1)-th frame image.

[0063] Step 4: Update the optimized target tracking model based on the feature map of frame t;

[0064] Step 5: Determine if there is a next frame. If yes, return to step 1; otherwise, end the process.

[0065] In one possible implementation, in step 2, the hierarchical convolutional features employ conv5-4 and conv4-4 layers from VGG-19.

[0066] In step 4, an online adaptive template update strategy is used to update the Lagrange parameters. The filter model update formula is as follows:

[0067]

[0068] Where m and m-1 represent the m-th and (m-1)-th frames of the video sequence, respectively, and λ represents the learning efficiency, λ = 0.0192. The filter template learned for the current frame. This is the filter template obtained from the previous frame.

[0069] By incorporating a spatiotemporal regularization module into the background-aware correlation filter framework, the boundary effect is effectively addressed, resulting in better robustness. The proposed optimization model can be efficiently solved using the Alternating Direction Multiplier Method (ADMM) optimization algorithm, accelerating the algorithm's convergence. Furthermore, from a feature representation perspective, the proposed model effectively combines higher-level deep features with shallow handcrafted features, enabling the filter to acquire more abstract semantic information and thus improving its discriminative ability. Compared to many excellent trackers, the proposed tracker exhibits superior performance.

[0070] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A target tracking method with model optimization, characterized in that: This includes the process of building and optimizing the target tracking model, as well as the target tracking process for video sequences; The process of constructing and optimizing the target tracking model includes: Temporal regularization is introduced to effectively handle boundary effect problems, and a background-aware target tracking model based on temporal regularization is constructed. The formula of the background-aware target tracking model based on temporal regularization is as follows: Where D represents the total number of feature channels, and B is the shearing matrix that clusters sample information onto the target region. It is the first d Target characteristics of the channel It is the first d Correlation filter for the channel, It is the first d -1 channel correlation filter, Indicates the correlation filter for the current frame. y It is the ideal output in Gaussian function form. and This represents the regularization parameter, and * represents the convolution operation. It is the introduced spatiotemporal regularization term that enables the filter to establish a spatiotemporal relationship between the current frame and historical frames; The background-aware target tracking model based on time regularization is transformed to the frequency domain, and then optimized using the alternating direction multiplier method. It is then decomposed into M sub-problems, leading to a local optimum. Finally, the Sherman-Morrison theorem is used to optimize the calculation formula for the local optimum, resulting in the optimized target tracking model. Where ^ denotes the discrete Fourier transform of the signal. The introduced error term penalty parameter, where M is the number of subproblems, For tags , For the sample , for , for , , , , , T represents the conjugate transpose operation of a complex vector or matrix; The target tracking process for the video sequence includes: Step 1: Input a video sequence and determine if it is the first frame. If it is, initialize the target's position and the optimized target tracking model. If not, proceed to Step 2. Step 2: Based on the target location in the (t-1)th frame image, obtain the target region in the tth frame. Using this region as the target region, extract the HOG features, color features, and hierarchical convolution features of the target region. Perform fusion processing on the HOG features, CN features, and hierarchical convolution features of the tth frame to obtain the fused feature X, where t is a natural number. Step 3: Based on the fused feature X and the optimized time-regularized background-aware target tracking model, predict the position of the target in the t-th frame image and output the result. The time-regularized background-aware target tracking model has been trained in advance on the target image in the (t-1)-th frame image. Step 4: Update the optimized target tracking model based on the feature map of frame t; update the optimized target tracking model using an online adaptive template update strategy, and the filter model update formula is as follows: in m and m -1 represents the nth video sequence. m Frame and the m -1 frame, Indicates learning efficiency. =0.0192, The filter template learned for the current frame. The filter template obtained from the previous frame; Step 5: Determine if there is a next frame. If yes, return to step 1; otherwise, end the process.

2. The method according to claim 1, characterized in that: In step 2, the hierarchical convolutional features use the conv5-4 and conv4-4 layers from VGG-19.

Images