A related filtering tracking method based on a double model competition mechanism

Abstract

The application relates to a related filtering tracking method based on a double-model competition mechanism, which fuses color features and gradient features and combines feature dimension reduction technology to construct a feature descriptor, and uses the feature descriptor to train a background-aware related filter, so that the robustness of the filter in response to interference factors such as light changes, target deformation and motion blur is improved; and the double-model competition mechanism is introduced, the tracking response graph confidence of a static model and a dynamic model is used to determine a tracking result, the reliability of the tracking result is improved, and the model drift problem caused by factors such as occlusion and target motion out of the field of view is solved. The tracking speed of the application can reach 45 frames per second under the support of a general desktop CPU, and the real-time demand of most actual application scenes can be met.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and relates to a target tracking method, specifically a correlation filter tracking method based on a dual-model competition mechanism. Background Technology

[0002] Target tracking is a crucial research area in computer vision, with wide-ranging applications in video surveillance, autonomous driving, intelligent robotics, and human-computer interaction. After more than a decade of rapid development, target tracking technology has made significant progress. However, due to various interference factors such as changes in lighting, target deformation, motion blur, and occlusion, achieving robust online tracking remains a highly challenging task.

[0003] The background-aware correlation filter (BACF) tracking method proposed in the paper "Learning background-aware correlation filters for visualtracking, IEEE International Conference on Computer Vision, 2017: 1135-1143" utilizes a mask matrix to crop samples within a large search region, using the resulting real background as negative samples to train the filter. This effectively solves the boundary effect problem caused by cyclic sampling in traditional correlation filter tracking methods. Although BACF has achieved remarkable tracking performance, it still has the following two shortcomings: First, BACF only uses a single HOG feature to model the appearance of the target, resulting in limited feature representation capabilities; second, BACF uses a fixed method for unidirectional model updates in each frame to adapt to changes in the target's appearance during motion, but this is susceptible to model drift due to factors such as occlusion and the target moving out of the field of view when dealing with complex environments or long-term tracking scenarios. Therefore, there is still room for improvement in feature representation and model drift suppression in the BACF tracking method. Summary of the Invention

[0004] Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention designs a correlation filter tracking method based on a dual-model competition mechanism. This method solves the problems of insufficient feature representation and model drift in background-aware correlation filters, thereby improving the tracking stability of the tracker in complex environments while ensuring real-time tracking speed.

[0006] Technical solution

[0007] A correlation filter tracking method based on a dual-model competition mechanism is characterized by the following steps:

[0008] Step 1: Read the image data of the first frame and the initial position information of the target to be tracked from the video sequence. ,in and Represent the x and y coordinates of the target's center point. and Indicates the width and height of the target, superscript Indicates the first frame;

[0009] Step 2: Extract by point Centered on, with side length as The image patch corresponding to the square region is converted from RGB color space to HSV color space. 31-dimensional HOG features are extracted from each of the three channels in the HSV space. All HOG features are concatenated to form a 93-dimensional feature vector. PCA is then used for feature dimensionality reduction, resulting in a feature channel count of [number missing]. eigenvectors , Setting it to 60 will result in the target static template. Initialize the target's dynamic template in the first frame. ;

[0010] Step 3: In the time domain, calculate the background-aware correlation filter by solving the following objective function. :

[0011] (1)

[0012] In the formula, The expected correlation filter response output is predefined as a Gaussian function distribution. It is a channel index. Indicates the relevant operation. It is a binary diagonal matrix whose purpose is to allow relevant operations to be applied directly to the real foreground and background. Represents the complex conjugate of a vector or matrix. For regularization parameters, Set to 0.001, equation (1) can be converted to the frequency domain using Pasvald's theorem for fast calculation, i.e., its equivalent form in the frequency domain is:

[0013] (2)

[0014] In the formula, It is an auxiliary variable. This represents the discrete Fourier transform of a variable. This indicates element-wise multiplication. It is an orthogonal. A matrix is ​​used to store any The complex basis vector of dimension is mapped to the frequency domain. Equation (2) can be solved iteratively by establishing the augmented Lagrange equation and using the Alternating Direction Method of Multipliers (ADMM). The solution obtained is... That is, the static filter, and then the dynamic filter in the first frame. ;

[0015] Step 4: Read the next frame of the image, i.e., the... Frame image, extract the target center point from the previous frame. Centered on, with side length as The image patch corresponding to the square region is used for feature extraction according to the method in step 2, resulting in a feature channel number of [number missing]. eigenvectors After Fourier transform, its first... The feature vectors of each channel are ;

[0016] Step 5: Calculate the static filter using equation (3) and dynamic filters In the Frame correlation filter response diagram and :

[0017] (3)

[0018] In the formula, This represents the inverse Fourier transform. Indicates the previous frame number Dynamic filters for each channel;

[0019] Step 6: The correlation filter response plot is represented as a two-dimensional matrix. For any given... Response diagram Its confidence level The following can be calculated:

[0020] (4)

[0021] In the formula, and Representing response diagrams The maximum and minimum values ​​in the response graph can be calculated using equation (4). and confidence level and ;

[0022] Step 7: When condition (5) is met, the dominant model of the current frame tracking result is determined to be the dynamic model:

[0023] (5)

[0024] In the formula, As a balance factor, If the value is set to 0.84, and the condition of equation (5) is not met, then the dominant model of the current frame is the static model.

[0025] Step 8: Take the coordinate position corresponding to the maximum value in the dominant model response map determined in Step 7 as the new target center of the current frame, denoted as point . That is, the goal is in the first The predicted position of the frame is based on points. Centered on For width, A rectangular region of height;

[0026] Step 9: Based on the new target center point Then, perform feature extraction again using the method described in step 2, and denote the resulting feature vector as... And use equation (6) to calculate the updated target dynamic template. :

[0027] (6)

[0028] In the formula, For learning rate, Set to 0.0125;

[0029] Step 10: Substitute them into equations (1) and (2) in step 3. Solve equation (2) again using the method described in step 3 to obtain the new dynamic filter. ;

[0030] Step 11: Determine whether all image frames in the video have been processed. If so, the algorithm ends; otherwise, proceed to step 4.

[0031] Beneficial effects

[0032] This invention, based on a background-aware correlation filter tracking framework, designs a correlation filter tracking method based on a dual-model competition mechanism. On one hand, by fusing color features and HOG features and combining them with feature dimensionality reduction techniques, a feature descriptor suitable for the correlation filter framework is constructed. This feature descriptor, used for filter learning, effectively improves the model's feature representation ability and anti-interference capability. On the other hand, a dual-model competition mechanism is designed based on the interaction between static and dynamic models. This allows the dynamic model to adaptively perform forward or reverse update operations based on the tracking results, effectively suppressing model drift caused by interference factors such as occlusion.

[0033] Superiority: Compared with the basic tracking method BACF, the improvements in feature representation and model drift suppression in this invention significantly enhance the tracking accuracy and robustness of the tracker in complex environments such as changes in illumination, target deformation, motion blur, occlusion, and target movement out of the field of view. It has demonstrated superior overall performance on multiple public datasets such as OTB2015, TinyTLP, and UAV20L, and can achieve a real-time tracking speed of 45 frames per second with the support of a general desktop CPU. Attached Figure Description

[0034] Figure 1 Flowchart of a correlation filter tracking method based on a dual-model competition mechanism Detailed Implementation

[0035] The present invention will now be further described in conjunction with the embodiments and accompanying drawings:

[0036] Step 1: Read the image data of the first frame and the initial position information of the target to be tracked from the video sequence. ,in and Represent the x and y coordinates of the target's center point. and Indicates the width and height of the target, superscript This indicates the first frame.

[0037] Step 2: Extract by point Centered on, with side length as The image patch corresponding to the square region is converted from RGB color space to HSV color space. 31-dimensional HOG features are extracted from each of the three channels in the HSV space. All HOG features are concatenated to form a 93-dimensional feature vector. PCA is then used for feature dimensionality reduction, resulting in a feature channel count of [number missing]. eigenvectors , Setting it to 60 will result in the target static template. Initialize the target's dynamic template in the first frame. .

[0038] Step 3: In the time domain, calculate the background-aware correlation filter by solving the following objective function. :

[0039] (1)

[0040] In the formula, The expected correlation filter response output is predefined as a Gaussian function distribution. It is a channel index. Indicates the relevant operation. It is a binary diagonal matrix whose purpose is to allow relevant operations to be applied directly to the real foreground and background. Represents the complex conjugate of a vector or matrix. For regularization parameters, Set to 0.001, equation (1) can be converted to the frequency domain using Pasvald's theorem for fast calculation, i.e., its equivalent form in the frequency domain is:

[0041] (2)

[0042] In the formula, It is an auxiliary variable. This represents the discrete Fourier transform of a variable. This indicates element-wise multiplication. It is an orthogonal. A matrix is ​​used to store any The complex basis vector of dimension is mapped to the frequency domain. Equation (2) can be solved iteratively by establishing the augmented Lagrange equation and using the Alternating Direction Method of Multipliers (ADMM). The solution obtained is... That is, the static filter, and then the dynamic filter in the first frame. .

[0043] Step 4: Read the next frame of the image, i.e., the... Frame image, extract the target center point from the previous frame. Centered on, with side length as The image patch corresponding to the square region is used for feature extraction according to the method in step 2, resulting in a feature channel number of [number missing]. eigenvectors After Fourier transform, its first... The feature vectors of each channel are .

[0044] Step 5: Calculate the static filter using equation (3). and dynamic filters In the Frame correlation filter response diagram and :

[0045] (3)

[0046] In the formula, This represents the inverse Fourier transform. Indicates the previous frame number Dynamic filters for each channel.

[0047] Step 6: The correlation filter response diagram is represented as a two-dimensional matrix. For any given... Response diagram Its confidence level The following can be calculated:

[0048] (4)

[0049] In the formula, and Representing response diagrams The maximum and minimum values ​​in the response graph can be calculated using equation (4). and confidence level and .

[0050] Step 7: When condition (5) is met, the dominant model of the current frame tracking result is determined to be the dynamic model.

[0051] (5)

[0052] In the formula, As a balance factor, If the value is set to 0.84, the dominant model of the current frame is the static model when the condition of equation (5) is not met.

[0053] Step 8: The coordinates corresponding to the maximum value in the dominant model response map determined in Step 7 are taken as the new target center of the current frame, denoted as point . That is, the goal is in the first The predicted position of the frame is based on points. Centered on For width, A rectangular region of height.

[0054] Step 9: Based on the new target center point Then, perform feature extraction again using the method described in step 2, and denote the resulting feature vector as... And use equation (6) to calculate the updated target dynamic template. :

[0055] (6)

[0056] In the formula, For learning rate, Set to 0.0125.

[0057] Step 10 will Substitute them into equations (1) and (2) in step 3. Solve equation (2) again using the method described in step 3 to obtain the new dynamic filter. .

[0058] Step 11 determines whether all image frames in the video have been processed. If so, the algorithm ends; otherwise, proceed to step 4 to continue execution.

Claims

1. A correlation filter tracking method based on a dual-model competition mechanism, characterized in that... The steps are as follows: Step 1: Read the image data of the first frame and the initial position information of the target to be tracked from the video sequence. ,in and Represent the x and y coordinates of the target's center point. and Indicates the width and height of the target, superscript Indicates the first frame; Step 2: Extract by point Centered on, with side length as The image patch corresponding to the square region is converted from RGB color space to HSV (Hue, Saturation, Value) color space. 31-dimensional HOG (Histogram of Oriented Gradient) features are extracted from each of the three channels in the HSV space. All HOG features are concatenated to form a 93-dimensional feature vector. Principal Component Analysis (PCA) is then used for feature dimensionality reduction, resulting in a feature channel count of [value missing]. eigenvectors , Setting it to 60 will result in the target static template. Initialize the target's dynamic template in the first frame. ; Step 3: In the time domain, calculate the background-aware correlation filter by solving the following objective function. : （1） In the formula, The expected correlation filter response output is predefined as a Gaussian function distribution. It is a channel index. Indicates the relevant operation. It is a binary diagonal matrix whose purpose is to allow relevant operations to be applied directly to the real foreground and background. Represents the complex conjugate of a vector or matrix. For regularization parameters, Set to 0.001, equation (1) can be converted to the frequency domain using Pasvald's theorem for fast calculation, i.e., its equivalent form in the frequency domain is: （2） In the formula, It is an auxiliary variable. This represents the discrete Fourier transform of a variable. This indicates element-wise multiplication. It is an orthogonal system. A matrix is ​​used to store any The complex basis vector of dimension is mapped to the frequency domain. Equation (2) can be solved iteratively by establishing the augmented Lagrange equation and using the alternating direction method of multipliers (ADMM). The solution obtained is... That is, the static filter, and then the dynamic filter in the first frame. ; Step 4: Read the next frame of the image, i.e., the... Frame image, crop the target center point of the previous frame. Centered on, with side length as The image patch corresponding to the square region is used for feature extraction according to the method in step 2, resulting in a feature channel number of [number missing]. eigenvectors After Fourier transform, its first... The feature vectors of each channel are ; Step 5: Calculate the static filter using equation (3) and dynamic filters In the Frame correlation filter response diagram and : （3） In the formula, This represents the inverse Fourier transform. Indicates the previous frame number Dynamic filters for each channel; Step 6: The correlation filter response plot is represented as a two-dimensional matrix. For any given... Response diagram Its confidence level The following can be calculated: （4） In the formula, and Representing response diagrams The maximum and minimum values ​​in the response graph can be calculated using equation (4). and confidence level and ; Step 7: When condition (5) is met, the dominant model of the current frame tracking result is determined to be the dynamic model: （5） In the formula, As a balance factor, If the value is set to 0.84, and the condition of equation (5) is not met, then the dominant model of the current frame is the static model. Step 8: Take the coordinate position corresponding to the maximum value in the dominant model response map determined in Step 7 as the new target center of the current frame, denoted as point . That is, the goal is in the first The predicted position of the frame is based on points. Centered on For width, A rectangular region of height; Step 9: Based on the new target center point Then, perform feature extraction again using the method described in step 2, and denote the resulting feature vector as... And use equation (6) to calculate the updated target dynamic template. : （6） In the formula, For learning rate, Set to 0.0125; Step 10: Substitute them into equations (1) and (2) in step 3. Solve equation (2) again using the method described in step 3 to obtain the new dynamic filter. ; Step 11: Determine whether all image frames in the video have been processed. If so, the algorithm ends; otherwise, proceed to step 4.

Images