An anti-occlusion tracking method based on kernel correlation filter
By calculating the maximum response value and average peak-related energy of the kernel-related filter, judging the occlusion situation, and using particle filter redetection strategy to reposition the target when the occlusion occurs, the target box drift problem of traditional kernel-related filters in complex backgrounds and occlusion situations is solved, and more stable target tracking is achieved.
Patent Information
- Application Number
- CN202210628250.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-06
- Publication Date
- 2025-03-21
- Estimated Expiration
- 2042-06-06
AI Technical Summary
Traditional core-related filters are prone to the problem of target frame drifting under complex backgrounds and occlusions, and it is difficult to effectively deal with complex backgrounds and occlusions.
By calculating the maximum response value Fmax and the average peak correlation energy APCE of the correlation filter, we judge whether the current frame is occluded, and the tracking results are judged based on these two confidence indicators. If the threshold condition is met, the filter model will continue to be updated; otherwise, it will be judged as untrusted and go to the particle filter redetection strategy to reposition the target.
It effectively reduces tracking drift caused by model pollution problems during the tracking process, realizes re-tracking of the target, and solves the full occlusion problem during the tracking process.
Smart Images
Figure CN115035444B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of computer vision, and in particular to an anti-occlusion tracking method based on a kernel correlation filter. Background Art
[0002] Visual object tracking technology is widely used in video surveillance, drone visual navigation, autonomous driving and other fields. In this work, we consider the single target tracking problem, which has made great progress in the past few decades. However, due to the complexity of actual tracking scenes including target deformation, motion blur, scale transformation and occlusion, target tracking still faces many challenges. Therefore, how to implement efficient and robust tracking algorithms has always been a problem worth studying.
[0003] At present, target tracking algorithms are mainly divided into two categories: generative and discriminative. Generative methods do not fully utilize video information, which leads to unstable tracking when the target changes drastically. In contrast to generative methods, discriminative methods regard target tracking as a binary classification problem of target and background, and extract target and background information to train classifiers to distinguish the target from the surrounding background. Discriminative methods can effectively distinguish target information from background information, improve the shortcomings of generative models, and have become the mainstream in the field of visual target tracking research. Among them, the kernel correlation filter belongs to the generative method. The correlation filter is trained by the samples collected in the previous frame, and the maximum response area is selected in the current frame as the tracking result. Although the tracker is competitive among the most advanced trackers in terms of computing time and performance, it is prone to target box drift when the target background environment is complex or interference occurs due to excessive reliance on the response value for positioning during the tracking process. Therefore, improving the tracking ability that can handle complex backgrounds has always been a challenging task.
[0004] Particle filter is an effective method to obtain the target object through particle resampling and is widely used in the field of target tracking. Its main idea is to use random particles with different weights to represent the posterior density and realize the dynamic fitting of the target state. The particle resampling strategy can cover the target state that undergoes large-scale changes. Therefore, particle filter can effectively relocate and track the target to solve the occlusion problem. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide an anti-occlusion tracking method based on a kernel correlation filter, which can effectively improve the tracking effect of the target when it is occluded in the traditional correlation filtering algorithm.
[0006] In order to solve the above technical problems, the present invention provides an anti-occlusion tracking method based on a kernel correlation filter, comprising the following steps:
[0007] Step 1: Initialize the relevant parameters of the KCF tracker, extract the target window features, generate positive and negative sample training classifiers through circulant matrix dense sampling, and convert them into frequency domain for training and updating;
[0008] Step 2: Calculate the maximum response value F of the correlation filter max The average peak correlation energy APCE is compared with the set threshold to determine whether occlusion occurs in the current frame;
[0009] Step 3: The tracking result is judged according to the two confidence indicators. If both confidence indicators meet the threshold condition, the filter model is updated continuously; if both confidence indicators do not meet the threshold condition and are judged as untrustworthy, the target is relocated in step 4.
[0010] Step 4: When it is determined that tracking fails, the particle filter re-detection strategy is used to relocate the target, calculate the response value corresponding to each particle respectively, select the particle with the largest response value as the new target area, and update the relevant filter.
[0011] Preferably, in step 1, the purpose of training is to find a function f(x)=w T x can minimize the sample image x i and regression target y i The square error between them is converted into the frequency domain for training, and the mathematical description is:
[0012]
[0013] Among them, (x i ,y i ) is a sequence of training samples and regression targets, λ is a regularization parameter used to prevent overfitting, and w is a training weight.
[0014] Preferably, in step 1, extracting the target window features specifically includes the following steps:
[0015] (1) Based on the given target frame, initialize the Hanning window to find the appropriate size of the detection area;
[0016] (2) Extract Hog features, perform normalization and PCA dimensionality reduction;
[0017] (3) Extract Lab features and connect the results with Hog features;
[0018] (4) Weight all features according to the location of the cell.
[0019] Preferably, in step 2, the operation of calculating the maximum response value of the correlation filtering specifically includes the following steps:
[0020] (1) Obtain the actual position of the target frame in the image frame and extract features;
[0021] (2) Perform convolution operation on the extracted features in the frequency domain;
[0022] (3) The target response diagram is obtained by transforming it into the time domain through IFFT.
[0023] Preferably, in step 2, the highest point of the response graph reflects the confidence of the tracking result, and the position where the maximum response value is located is selected as the target of the new frame:
[0024] F max =maxF(s,y;w)
[0025] Among them, s represents the position of the target image block in the previous frame, y is the target position, w is the parameter of the classifier, and F is the possibility function of judging (s, y) as the target. max The larger it is, the higher the tracking accuracy;
[0026] The average peak correlation energy APCE can reflect the sharpness and fluctuation of the response graph, and the calculation formula is:
[0027]
[0028] Among them: F max 、F min and F w，h Represents the maximum value, minimum value, and element in the wth row and hth column of F(s, y; w).
[0029] Preferably, in step 3, the tracking result is judged according to the two confidence indicators. If both confidence indicators meet the threshold condition, the filter model continues to be updated. When one of the two confidence indicators does not meet the condition, it indicates that there is occlusion at this time, and the template of the current filter is saved, and the learning rate is set to 0 to stop updating the current filter parameters.
[0030] The update model of the filter is:
[0031]
[0032]
[0033] Where: t is the serial number of the current frame, η is the learning rate, is the target appearance model for the predicted position, are the trained filter parameters.
[0034] Preferably, in step 4, when it is determined that the target tracking fails, the target is relocated by re-detection using particle filtering, and the N particles generated by the particle filter are Corresponding to the image blocks of the same size as the search window in the N correlation filters, the response value of the correlation filter is calculated by extracting the feature information in the image block and converting it into the frequency domain. The optimal state of the target is estimated by the response value, and finally the position information of the target is detected. The corresponding response value of each particle is defined as:
[0035]
[0036] in, represents the i-th particle at time t Corresponding observations, x and α represent the appearance model and coefficients of the correlation filter, respectively. is the i-th particle at time t The response obtained by the correlation filter, φ is the kernel function.
[0037] The beneficial effects of the present invention are as follows: (1) The anti-occlusion method based on kernel correlation filtering proposed in the present invention utilizes the maximum value of the correlation response and the value of APCE to identify the target occlusion problem, thereby constraining the update conditions of the filter template and reducing the tracking drift caused by the model contamination problem in the tracking process; (2) The re-detection model proposed in the present invention can detect the corresponding object through the particle resampling strategy and regain the position information, thereby realizing the re-tracking of the target and effectively solving the full occlusion problem in the tracking process. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 It is a schematic diagram of the method flow of the present invention.
[0039] FIG. 2( a ) is a diagram of a tracking sequence (21 frames) during the tracking process of the present invention.
[0040] FIG. 2( b ) is a diagram of a tracking sequence (108 frames) during the tracking process of the present invention.
[0041] FIG. 2( c ) is a confidence response diagram corresponding to a tracking sequence (21 frames) during the tracking process of the present invention.
[0042] FIG2( d ) is a confidence response diagram corresponding to the tracking sequence (108 frames) during the tracking process of the present invention.
[0043] Figure 3 This is a diagram of target occlusion effects corresponding to the KCF algorithm and the method of the present invention. DETAILED DESCRIPTION
[0044] like Figure 1 As shown, an anti-occlusion tracking method based on a kernel correlation filter comprises the following steps:
[0045] Step 1: Initialize the relevant parameters of the KCF tracker, extract the target window features, generate positive and negative sample training classifiers through cyclic matrix dense sampling, and transform them into the frequency domain for training and updating. The mathematical description of the training process can be expressed as:
[0046]
[0047] Among them, (x i ,y i ) is a sequence of training samples and regression targets, λ is a regularization parameter used to control overfitting, and w is a training weight.
[0048] The steps of extracting the features of the target window mainly include: (1) initializing the Hanning window to find the appropriate size of the detection area according to the given target box; (2) extracting the Hog feature, normalizing it and performing PCA dimensionality reduction; (3) extracting the Lab feature and connecting the result with the Hog feature; (4) weighting all features according to the position of the cell.
[0049] Step 2: Calculate the maximum response value F of the correlation filter max The average peak correlation energy APCE is compared with the set threshold to determine whether occlusion occurs in the current frame.
[0050] In step 2, the operation of calculating the maximum response value of the correlation filter includes: (1) obtaining the actual position of the target frame in the image frame and extracting the features; (2) performing a convolution operation on the extracted features in the frequency domain; and (3) converting to the time domain through IFFT to obtain a response map of the target.
[0051] The highest point of the response graph can reflect the confidence of the tracking result, and the position of the maximum response value is selected as the target of the new frame:
[0052] F max =maxF(s,y;w)
[0053] Where s represents the position of the target image block in the previous frame, y is the target position, and w is the parameter of the classifier. F is the probability function for judging (s, y) as the target. Usually, F max The larger it is, the higher the tracking accuracy is.
[0054] The average peak correlation energy APCE can reflect the sharpness and fluctuation of the response graph, and the calculation formula is:
[0055]
[0056] Among them: F max 、F min and F w，hRepresents the maximum value, minimum value, and element in the wth row and hth column of F(s, y; w).
[0057] Step 3: The tracking results are judged based on the two confidence indicators. If both confidence indicators meet the threshold conditions, the filter model is updated. If one of the two confidence indicators does not meet the conditions, it indicates that there is occlusion. The template of the current filter is saved, and the learning rate is set to 0 to stop updating the current filter parameters.
[0058] The update model of the filter is:
[0059]
[0060]
[0061] Where: t is the serial number of the current frame, η is the learning rate, is the target appearance model for the predicted position, are the trained filter parameters.
[0062] Step 4: When it is determined that the target tracking fails, the particle filter is used to re-detect the target and relocate it. Corresponding to the image blocks of the same size as the search window in the N correlation filters, the response value of the correlation filter is calculated by extracting the feature information in the image block and converting it into the frequency domain. The optimal state of the target is estimated by the response value, and finally the position information of the target is detected. The corresponding response value of each particle is defined as:
[0063]
[0064] in, represents the i-th particle at time t Corresponding observations, x and α represent the appearance model and coefficients of the correlation filter, respectively. is the i-th particle at time t The response obtained by the correlation filter, φ is the kernel function.
[0065] FIG2(a)-(d) show the tracking sequence and the corresponding confidence response graph of the present invention during the tracking process. It is observed that when the tracking target is blocked, the peak value changes from a single peak to multiple peaks, and the response graph fluctuates significantly. Figure 3The video sequences in Figure 1 are Girl2, Jogging-1 and Tiger2. During the tracking process, these three targets are all occluded. The KCF algorithm does not determine the occlusion of the target. When the target is occluded, the updated template is gradually replaced by the occluder because the appearance information of the target cannot be extracted. The algorithm proposed in the present invention determines the occlusion of the target by combining two confidence indicators. When the target is occluded, the template update is stopped to reduce the contamination of the sample. When the tracking fails, the target is retrieved again and re-tracked. Based on the comparison results of the three sequences, the present invention provides an effective solution to occlusion.
[0066] The present invention mainly constrains the filter template update conditions based on the occlusion judgment mechanism. At the same time, when it is determined that the tracking fails, the particle filter is started to re-perceive the target state to achieve relocation. The anti-occlusion method based on correlation filtering is the maximum response peak F max When the average peak correlation energy is greater than the set threshold, the target is directly located and tracked, and the filter model is updated in the next frame. If one of the two does not meet the threshold condition, it is determined to be a semi-occluded state and the template update is suspended; if both meet the threshold condition, the tracking is determined to fail and the target is relocated. The relocation method is a particle filter re-detection method, which provides a rich candidate object through a particle resampling strategy to re-perceive the target state and determine the center position of the corresponding object.
Claims
1. An anti-occlusion tracking method based on kernel correlation filter, characterized in that: The steps include: Step 1: Initialize the relevant parameters of the KCF tracker, extract the target window features, generate positive and negative sample training classifiers through circulant matrix dense sampling, and transform them into the frequency domain for training and updating; the mathematical description of the training and updating process in the frequency domain is: Among them, (x i ,y i ) is a sequence corresponding to a training sample and a regression target, λ is a regularization parameter used to prevent overfitting, and w is a training weight; Step 2: Calculate the maximum response value F of the correlation filter max The average peak correlation energy APCE is compared with the set threshold to determine whether occlusion occurs in the current frame; Step 3: The tracking result is judged according to the two confidence indicators. If both confidence indicators meet the threshold condition, the filter model is continued to be updated; if both confidence indicators do not meet the threshold condition and are judged to be untrustworthy, go to step 4 to relocate the target; the tracking result is judged according to the two confidence indicators. If both confidence indicators meet the threshold condition, the filter model is continued to be updated. When one of the two confidence indicators does not meet the condition, it indicates that there is occlusion at this time, and the template of the current filter is saved, and the learning rate is set to 0 to stop updating the current filter parameters; The update model of the filter is: Where: t is the serial number of the current frame, η is the learning rate, is the target appearance model for the predicted position, are the filter parameters obtained through training; Step 4: When it is determined that tracking fails, the particle filter re-detection strategy is used to relocate the target, the response value corresponding to each particle is calculated respectively, the particle with the largest response value is selected as the new target area, and the relevant filter is updated; when it is determined that the target tracking fails, the particle filter re-detection is used to relocate the target, and the N particles generated by the particle filter are Corresponding to the image blocks of the same size as the search window in the N correlation filters, the response value of the correlation filter is calculated by extracting the feature information in the image block and converting it into the frequency domain. The optimal state of the target is estimated by the response value, and finally the position information of the target is detected. The corresponding response value of each particle is defined as: in, represents the i-th particle at time t Corresponding observations, x and α represent the appearance model and coefficients of the correlation filter, respectively. is the i-th particle at time t The response obtained by the correlation filter, φ is the kernel function.
2. The anti-occlusion tracking method based on kernel correlation filter according to claim 1, characterized in that: In step 1, extracting the target window features specifically includes the following steps: (1) Based on the given target frame, initialize the Hanning window to find the appropriate size of the detection area; (2) Extract Hog features, perform normalization and PCA dimensionality reduction; (3) Extract Lab features and connect the results with Hog features; (4) Weight all features according to the location of the cell.
3. The anti-occlusion tracking method based on kernel correlation filter according to claim 1, characterized in that: In step 2, the operation of calculating the maximum response value of the correlation filter specifically includes the following steps: (1) Obtain the actual position of the target frame in the image frame and extract features; (2) Perform convolution operation on the extracted features in the frequency domain; (3) The target response diagram is obtained by transforming it into the time domain through IFFT.
4. The anti-occlusion tracking method based on kernel correlation filter according to claim 1, characterized in that: In step 2, the highest point of the response graph reflects the confidence of the tracking result, and the location of the maximum response value is selected as the target of the new frame: F max =maxF(s,y;w) Among them, s represents the position of the target image block in the previous frame, y is the target position, w is the parameter of the classifier, and F is the possibility function of judging (s, y) as the target. max The larger it is, the higher the tracking accuracy; The average peak correlation energy APCE can reflect the sharpness and fluctuation of the response graph, and the calculation formula is: Among them: F max 、F min and F w，h Represents the maximum value, minimum value, and element in the wth row and hth column of F(s, y; w).