An Improved KCF Object Tracking Method Combining with Lightweight SSD

Abstract

The present invention discloses an improved KCF object tracking method combined with a lightweight SSD. The method uses the lightweight SSD object detection framework to extract features of the object in the initial frame, and sequentially passes the extracted features through a prediction module, a non-maximum suppression module, and an output detection module to transfer the initial frame object information to KCF for initialization; KCF constructs a circulant matrix for the object to be tracked, uses ridge regression to train a position filter, and for the scale problem, adopts a binary tree scale search strategy to select the best scale near the position determined by the position filter; for the occlusion problem that occurs in object tracking, the average peak energy is selected as a judgment index to detect whether the object is occluded, and whether to update the template is determined according to the judgment result. The present invention effectively solves the problem of low efficiency caused by the need for manual annotation of the tracking object in KCF object tracking, and improves the accuracy and precision of object tracking in the case of scale changes and occlusions of the object.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual object tracking, and particularly relates to an improved KCF object tracking method combined with lightweight SSD. Technical Background

[0002] The detection and tracking of moving objects are important research contents in the field of computer vision research, and have important applications in civil and military fields such as intelligent monitoring, military reconnaissance, and human-computer interaction. The Correlation Filtering (CF) tracking algorithm is a typical discriminant method. In 2010, Bolem introduced the correlation in the field of signal processing into the field of computer vision, proposed the least square error output and the MOSSE filter, and achieved an amazing 615 FPS in tracking speed. On this basis, the CSK algorithm extended the dense sampling and adopted the kernel trick to perform high-dimensional operations in the low-dimensional space, avoiding the curse of dimensionality. The KCF tracking algorithm based on kernel correlation filtering further extended the fHOG feature on the basis of CSK, and utilized the property that the circulant matrix can be diagonalized in the frequency domain space to greatly reduce the algorithm operation amount and improve the tracking speed. Although the object tracking algorithm based on kernel correlation filtering performs well in terms of speed, directly using it to process video sequences taken from the perspective of an unmanned aerial vehicle (UAV) is not satisfactory, especially when there are complex situations such as target scale changes, deformations, and occlusions in the pictures taken by the UAV, which often leads to the traditional kernel correlation filtering algorithm being prone to tracking failure. Martin proposed the DSST algorithm, which introduced a scale filter on the basis of the two-dimensional position filter, collected 33 scales at the target center position for similarity comparison, and selected the maximum response value as the best matching scale of the target. However, due to the brute-force matching method adopted, the algorithm operation complexity is relatively high; secondly, since the determination of the initial frame target depends on manual annotation, the real-time performance of the algorithm operation is reduced. Summary of the Invention

[0003] The purpose of the present invention is to provide an improved KCF object tracking method combined with lightweight SSD. This method first uses an SSD object detection algorithm with a lightweight structure to locate the initial position target in the sequence to be tracked; secondly, a scale binary tree search strategy is introduced to reduce the algorithm operation amount; finally, a new model update strategy is adopted to update the template when the target is occluded or the like, improving the tracking effect.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] An improved KCF object tracking method combined with lightweight SSD, comprising:

[0006] Step 1, initialize the network parameters, set the target monitoring network label as the object to be tracked, and input the initial label into the network;

[0007] Step 2: Input the first-frame picture of the image sequence to be tracked into the target detection network part, and use the depthwise separable convolution block to extract features from the initial-frame picture;

[0008] Step 3: Pass the picture after feature extraction through the prediction module and the non-maximum suppression module, compare with the initial label, output the detected target coordinate values, and transfer them to the target tracking network;

[0009] Step 4: Receive the initial value transferred in Step 3, frame the corresponding target position, initialize the KCF filter template, construct a circulant matrix, and train the position filter;

[0010] Step 5: Construct a scale filter, and use the scale binary tree search strategy to select the best scale of the target near the position generated by the position filter;

[0011] Step 6: Obtain the Gaussian response of the target at the best scale, and use the APCE criterion for judgment. If the template update condition is satisfied, perform template update, increment the frame number and return to Step 5 until the tracking ends; if the template update condition is not satisfied, return to Step 2 to re-detect the target until the tracking ends;

[0012] Step 7: Output the tracking result, and the tracking ends.

[0013] The present invention has the following advantages:

[0014] 1. Aiming at the problem of manual annotation of the first-frame target in target tracking, the present invention uses a detection network to locate the initial-frame target, and combines it with the target tracking network by utilizing the characteristic of the low computational complexity of the depthwise separable convolution structure, avoiding the time loss of manual annotation.

[0015] 2. Aiming at the problems of scale change and target occlusion during the tracking process, the present invention respectively adopts the binary tree scale search strategy and the average peak energy criterion for improvement, enabling the target to be accurately tracked and located when complex situations such as scale change and occlusion occur during the tracking process. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the overall framework diagram of the method of the present invention

[0017] Figure 2 is the schematic diagram of the depthwise separable convolution

[0018] Figure 3 is the schematic diagram of the scale selection binary tree

[0019] Figure 4 is the comparison of the tracking effects between the method of the present invention and the correlation filtering algorithms

[0020] Figure 5 Comparison of the success rate and accuracy between the method of the present invention and related filtering algorithms in the SRE mode

[0021] Figure 6 Comparison of the success rate and accuracy between the method of the present invention and related filtering algorithms under occlusion Detailed implementation manners

[0022] The technical solution of the present invention will be specifically described with reference to the attached drawings.

[0023] As Figure 1 shown, an improved KCF object tracking method combining lightweight SSD according to the present invention specifically includes the following steps:

[0024] Step 1: Initialize network parameters, set the target detection network label as the object to be tracked, input the initial label into the network, and save the corresponding label before each tracking;

[0025] Step 2: Input the first frame picture of the image sequence to be tracked into the target detection network part, and use the depthwise separable convolution block to extract features from the picture. The schematic diagram of the depthwise separable convolution is as Figure 2 shown. First, use a 3×3×n filter to extract features from the feature map to be processed, where n represents the number of channels of the input feature map, and then use a 1×1×m convolution to fuse the information between different channels, where m represents the number of channels of the output feature map;

[0026] Step 3: Pass the picture after feature extraction through the prediction module and the non-maximum suppression module. During the prediction process, determine the category and confidence of each prediction box according to its category confidence, save the prediction box with the highest confidence of the object that is the same as the label input in Step 1 as the initial value of the target coordinates, and transfer it to the target tracking network;

[0027] Step 4: Receive the initial value of the target coordinates transferred in Step 3, construct a circulant matrix for the target, and use the Gaussian kernel function to perform ridge regression operation on the samples in the kernel space to solve the classifier, specifically as follows:

[0028] α=(K + λI) -1 y (8)

[0029] In the formula, K = φ(X)φ(X) T is the kernel matrix in the kernel space, where φ(X) is the non-linear mapping function, y is the label vector, and λ is the regularization parameter. Since K is a circulant matrix, using the properties of the circulant matrix, the formula (8) can be solved to obtain:

[0030]

[0031] In the formula, For the kernel matrix \(K = \varphi(X)\varphi(X)^T\), the first row of data is the base sample in the circulant matrix. For the next frame sample \(z\), the sample set \(Z\) can be obtained through feature extraction and the circulant matrix, and together with the training sample set \(X\), they form the kernel circulant matrix \(K_Z\). The optimal target position is shown by the following formula: T For the first row of data of T , that is, the base sample in the circulant matrix. For the next frame sample \(z\), the sample set \(Z\) can be obtained through feature extraction and the circulant matrix, and together with the training sample set \(X\), they form the kernel circulant matrix \(K_Z\). The optimal target position is shown by the following formula:

[0032]

[0033] Step 5, construct a scale filter. Near the optimal position obtained in Step 4, use the binary tree scale search strategy to determine the optimal scale. The schematic diagram of binary tree scale selection is as shown in Near the optimal position obtained in step 4, use the binary tree scale search strategy to determine the optimal scale. The schematic diagram of binary tree scale selection is as shown in Figure 3 As shown, each node defines three attributes, namely the scale value, the upper node \(a\), and the lower node \(b\). Assume that the initial size of the tracked target in the \((t - 1)\)-th frame is \(S\). The maximum value of the target response in the \(t\)-th frame is calculated by Equation (10) as Subsequently, select \(S\) i，j \(=\sigma\) i，j \(\cdot S\), where \(i\) represents the \(i\)-th layer of the scale tree, and \(j\in\{a, b\}\) respectively represent the upper and lower sub-branches of the \(i\)-th layer of the scale tree. The selected scale value satisfies the following conditions:

[0034]

[0035] When expanding (shrinking) the candidate region, if the target response value is greater than the original response value, select the larger scale branch; if the target response value is less than the original response value, select the smaller branch until the best scale is selected;

[0036] Step 6, obtain the Gaussian response of the target at the best scale, and use the APCE criterion to judge the tracking effect of the target. APCE reflects the peak level and average fluctuation degree of the tracking response map. The specific calculation formula is:

[0037]

[0038] In the formula, \(F\) max 、\(F\) min respectively represent the maximum and minimum values on the response map, \(F\) W，H represents the response value at the position \((W, H)\), and mean[\(\cdot\)] is the operation of taking the average value.

[0039] To improve the template accuracy, a new template update method is adopted, specifically:

[0040]

[0041] In the formula, is the filter model at the previous moment, \(\eta\) is the model update rate, and \(\theta\) is the model update discrimination factor, and the value is as follows:

[0042]

[0043] In the formula, represents the historical mean of the average peak energy ratio, represents the historical mean of the maximum response value, λ 1 and λ 2 are preset proportionality coefficients.

[0044] If the template update condition is satisfied, the template is updated, the frame number is incremented by one, and step 5 is returned until the tracking ends. If the template update condition is not satisfied, step 2 is returned to re-detect the target;

[0045] Step 7, output the tracking result, and the tracking ends.

[0046] To verify the feasibility and effectiveness of the present invention, the present invention will be further described in detail below with reference to examples.

[0047] This experiment was completed on a desktop computer with an Intel(R) Core(TM) i5-9400F CPU @ 2.9 GHZ, 6 GB of memory, and an NVIDIA GeForce GTX 1660, and was implemented through Pycharm, CUDA 9.0, and CUDNN 7.0. The experiment was verified from three aspects: tracking effect evaluation, center position error, and overall tracking performance of the algorithm.

[0048] Figure 4 This is a comparison of the tracking effects of the method of the present invention and related filtering algorithms. In the figure, the red box represents the method of the present invention, the black box represents the KCF algorithm, the blue box represents the SAMF algorithm, the green box represents the LMCF algorithm, and the pink box represents the DSST algorithm. It can be seen that in Figure 4 (a) In the CarScale sequence, the black SUV car approaches from far to near, and the scale of the target in the image gradually increases. Since the KCF algorithm does not include the function of scale change, the size of the tracking box remains unchanged and the tracking effect is poor. DSST and SAMF can ensure scale update at the beginning, but the problem of inaccurate target scale prediction also appears in the second half of the tracking. The method of the present invention can better adapt to the scale change; in Figure 4 (b) In the SUV image sequence, the target is occluded. Since the occlusion time is short, each algorithm can track the target, but the target positioning is different. Around 900 frames, the positioning range of the method of the present invention includes the white roof, while the other algorithms do not. Therefore, the method of the present invention is more accurate; in Figure 4(c) In the MountainBike image sequence, the target performs an aerial rotation action and there is interference from a similar background. It can be seen that there are tracking fluctuations within a certain range in KCF, LMCF, DSST, and SAMF. The method of the present invention can locate details of the target, including the wheels, and is more accurate; in Figure 4 (d) In the Car14 image sequence, the target captured from the perspective of the drone occupies a relatively small proportion of the overall image. The information learned by the KCF and DSST algorithms is less, and it is easy to have tracking failures. The method of the present invention can accurately locate the target.

[0049] Figure 5 and Figure 6 respectively show the comparison of the success rate and accuracy between the method of the present invention and the correlation filtering algorithm in the SRE mode and under occlusion. As can be seen from Figure 5 , in the SRE mode, the success rate and accuracy of the method of the present invention are 0.801 and 0.807 respectively, which are 7.9% and 3% higher than those of the second-place LMCF algorithm. As can be seen from Figure 6 , under occlusion, the success rate and accuracy of the method of the present invention are 0.735 and 0.732 respectively, which are 6.8% and 7.5% higher than those of the second-place LMCF algorithm.

Claims

1. An improved KCF object tracking method combined with lightweight SSD, characterized in that, it includes the following steps: Step 1, initialize network parameters, set the target monitoring network label as the object to be tracked, and input the initial label into the network; Step 2, input the first frame picture of the image sequence to be tracked into the object detection network part, and use the depthwise separable convolution block to extract features from the initial frame picture; Step 3, pass the picture after feature extraction through the prediction module and the non-maximum suppression module, compare with the initial label, output the detected target coordinate values, and transmit them to the object tracking network; Step 4, receive the initial value transmitted in Step 3, frame the corresponding target position, initialize the KCF filter template, construct a circulant matrix, and train the position filter; Step 5, construct a scale filter, and use the scale binary tree search strategy to select the best scale of the target near the position generated by the position filter. Specifically: (5.1) In the non-linear case of the KCF algorithm, by introducing the Gaussian kernel function, ridge regression operation is performed on the samples in the kernel space to solve the classifier. Specifically: α = (K + λI) -1 y(1) where \(K = \varphi(X)\varphi(X)\) T is the kernel matrix of the kernel space, where \(\varphi(X)\) is a non-linear mapping function, \(y\) is the label vector, and \(\lambda\) is the regularization parameter; since \(K\) is a circulant matrix, using the properties of circulant matrices, solving Equation (1) gives: where * represents the complex conjugate, is the kernel matrix K = φ(X)φ(X) T the first row data of, that is, the base samples in the circulant matrix. For the next frame sample z, the sample set Z can be obtained through feature extraction and the circulant matrix, and together with the training sample set X, it constitutes the kernel circulant matrix KZ. The target optimal position is shown as follows: In the formula, is a kernel circulant matrix jointly composed of the next-frame test sample and the training sample, is the discrete Fourier transform of α; (5.2) Assume that the initial size of the tracked target in the (t - 1)-th frame is S, and the maximum value of the target response in the t-th frame is calculated by Equation (3) as Subsequently, select S i,j = σ i,j ·S, where i represents the i-th layer of the scale tree, j ∈ {a, b}, and a and b respectively represent the two sub-branches of the i-th layer of the scale tree. The selected scale value satisfies the following conditions: (5.3) When expanding or shrinking the candidate region, if the maximum value of the target response is greater than the original response value, select the larger scale branch; If the maximum value of the target response is less than the original response value, then select the smaller scale branch until the best scale is selected; Step 6, obtain the Gaussian response obtained by the target at the best scale, judge using the APCE criterion. If the template update condition is met, perform template update, increment the frame number and return to Step 5 until the tracking ends; if the template update condition is not met, return to Step 2 to re-detect the target until the tracking ends. The specific conditions of the APCE criterion are: where F max , F min represent the maximum and minimum values on the response map respectively, F W,H represents the response value at the position (W, H), and mean[·] is the operation of taking the average value; when the peak is sharper and the response fluctuation is smaller, the APCE is larger; when interference or occlusion occurs, the APCE value will decrease sharply; To improve the template accuracy, the following template update method is adopted: In the formula, is the filter model at the previous moment, η is the model update rate, and θ is the model update discrimination factor; Step 7, output the best scale of the target in Step 6, and the tracking ends.

2. An improved KCF object tracking method combined with lightweight SSD according to claim 1, characterized in that, in Step 2, the depthwise separable convolution block is used to extract features from the initial frame picture. Specifically: First, a 3*3*n filter is used to extract features from the input image, and then the information between different channels is fused through a 1*1*m convolution on the generated feature map after extraction, where n represents the number of channels of the input feature map, and m represents the number of channels of the output feature map; after performing operations using depthwise separable convolution, the computational complexity is: D K × D K × M × D F × D F + M × N × D F × D F (7) where D K is the size of the input image, M is the number of channels of the input image, D F is the size of the output feature map, and N is the number of output channels.

Images