Support device real-time tracking method based on correlation filtering and saliency detection

Abstract

The invention discloses a support device real-time tracking method based on correlation filtering and saliency detection, and the method comprises the steps: firstly inputting a first-frame image containing annotation information, and carrying out the separation operation of a foreground and a background through a saliency detection algorithm; initializing a tracking filter according to the saliency detection result and the image features, and performing feature dimension reduction operation on the filter to reduce the parameter quantity of the filter; carrying out feature image generation on a new frame of image; performing convolution operation on the tracking filter and the feature graph to obtain a response graph; the maximum value of the response diagram is a tracking target; classifying the positioned image into a filtering kernel training sample set, and compressing and simplifying a sample space by adopting a Gaussian mixture model for tracking the training of a filter; meanwhile, model updating efficiency is considered, and a mode of training and updating a tracking filter every frame is adopted, so that memory consumption is reduced, and model calculation efficiency is improved; and judging whether the current video ends or not, if so, ending the algorithm, and if not, repeating the steps.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a real-time tracking method for a support device based on correlation filtering and saliency detection. Background Art

[0002] The target tracking algorithm aims to automatically track the position and scale changes of a moving target from a video sequence. It can extract features from the video sequence according to the target's motion model, and then determine the position and scale changes of the target in the current frame by calculating indicators such as similarity or correlation. Accurate tracking of the support device by the drone is a key step in automated inspection and also provides necessary support for the next step of data collection. However, the scenes along the high-speed rail are complex and changeable, resulting in the diverse characteristics of the support device background. In addition, the shooting angle of the drone is constantly changing during the tracking process, resulting in deformation and physical occlusion of the target during the tracking process. This puts high demands on the tracking algorithm used by the drone: the tracking algorithm needs to be lightweight and real-time to meet the requirements of real-time operation on the drone airborne platform. In addition, the tracking algorithm also needs to be able to effectively deal with complex and changeable backgrounds and deformation of foreground targets. The target tracking algorithm is an important research direction in the field of computer vision, and has been rapidly developed and widely used in recent years. The target tracking algorithm has been widely used in video surveillance, autonomous driving, robot vision and other fields. In recent years, many excellent target algorithms have emerged, such as KCF, ECO, SiamRPN, ATOM, etc. These algorithms continue to promote the development and application of target tracking technology. Generally, target tracking algorithms are classified according to model types, including target tracking algorithms based on template matching, statistics, machine learning and deep learning. The target tracking algorithm based on template matching is simple and fast in calculation, and is suitable for some simple tracking scenes, but the tracking effect is poor in scenes with changing illumination and complex background. The target tracking algorithm based on machine learning can enrich the target feature representation through machine learning methods to improve the accuracy and robustness of tracking, but such algorithms have high computational complexity and require a lot of computing resources. In addition, for tracking some complex scenes, a large amount of labeled data is required for training. The target tracking algorithm based on deep learning can be trained on large-scale data sets to learn more complex target feature representations and motion models, but it has the disadvantages of very high computational complexity and requires a lot of computing resources. In addition, for fast-moving targets, its tracking effect is not as good as other categories of tracking algorithms, and it also requires a large amount of data to be labeled. For complex scenes along the high-speed railway, the target tracking algorithm based on template matching is not applicable; and the target tracking algorithm based on machine learning and deep learning cannot be deployed on the UAV platform in a lightweight manner due to its high computational complexity and the requirement of a large amount of computing resources. On the contrary, the target tracking algorithm based on statistics has the advantages of fast calculation speed, low requirements on computing resources, and does not require a large number of training samples. Therefore, it is the most suitable tracking algorithm for automated inspection of contact network support devices. Among them, the target tracking algorithm based on correlation filtering is a classic tracking algorithm based on statistics. It realizes target tracking by establishing a correlation filter by learning the relationship between target features and background.This type of algorithm has the advantages of simple calculation and fast speed, and shows good robustness in various tracking scenarios. Classic target tracking algorithms based on correlation filtering include MOSSE algorithm, KCF algorithm and ECO algorithm. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a real-time tracking method for a support device based on correlation filtering and saliency detection, which solves the problems of complex calculation and slow recognition speed in the prior art.

[0004] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0005] A method for real-time tracking of a support device based on correlation filtering and saliency detection is provided, comprising the following steps:

[0006] S1, obtaining a video to be tested, and annotating the first frame of the video to be tested to obtain a first frame image containing the annotated information, and performing a foreground-background separation operation on the first frame image using an image saliency detection algorithm to obtain a saliency detection map;

[0007] S2. Initialize the ECO tracking filter according to the saliency detection map;

[0008] S3, extracting the feature image of the next frame image;

[0009] S4, performing convolution operation on the ECO tracking filter and the feature image of the next frame image to obtain a response map, and based on the response map, taking the location of the maximum response value as the target position, obtaining a positioning completion image of the current frame;

[0010] S5, classify the positioned image into the training sample set of the ECO tracking filter, and use the Gaussian mixture model to compress and simplify the training sample set, and determine whether the ECO tracking filter is updated in the previous frame tracking. If so, directly go to step S6, otherwise, update the ECO tracking filter based on the training sample set;

[0011] S6. Determine whether the video to be tested has ended. If so, the algorithm terminates. If not, jump to step S3.

[0012] Furthermore, the S1 is specifically:

[0013] S101, obtaining a video to be tested, and annotating the first frame of the video to be tested to obtain a first frame image containing the annotated information, and dividing the first frame image into a plurality of image blocks;

[0014] S102, performing DCT transformation on each image block to obtain a frequency domain representation of each image block;

[0015] S103, based on the frequency domain representation of each image block, respectively calculate the sum of the squares of the DCT coefficients of each image block to obtain the energy value of each image block in the frequency domain, that is, the significance value of each image block;

[0016] S104, normalizing the saliency value of each image block, and extracting the image blocks whose normalized saliency value is greater than the saliency threshold, to obtain the sparse salient region of the first frame image;

[0017] S105, perform IDCT inverse transformation on the sparse salient area of ​​the first frame image to obtain a saliency detection map. Further, the expression of the saliency detection map in S1 is:

[0018] M(x)=IDCT(S(x)) 2

[0019] S(x)=sign(DCT(x))

[0020] Among them, M(x) is the reconstructed image obtained by inverse discrete cosine transform, that is, the saliency detection map; IDCT() is the inverse IDCT transform; S(x) is the image saliency feature obtained by discrete cosine transform; sign is the sign function; DCT() is the discrete cosine transform; x is the image frame.

[0021] Furthermore, the objective function of the ECO tracking filter in S2 is:

[0022]

[0023] f=(f 1 ,…,f D )

[0024] P=[P d,c ]

[0025] Where E(f,P) is the objective function of the ECO tracking filter; f is the D-channel correlation filter; P is the linear combination coefficient matrix; is the square of the L2 norm; It is the Fourier transform result of the high-dimensional feature after the feature extraction operation; is the Fourier transform result of the filter; is the Fourier transform result of the expected response value; c is the filter number; C is the number of filters; is the Fourier transform result of the spatial regularization penalty term; is the Fourier transform result of the cth filter; λ is the regularization parameter; is the square of the Frobenius norm; f D is the Dth channel correlation filter; p d,cis the linear combination coefficient; T is the transpose.

[0026] Furthermore, S3 specifically adopts different feature extraction methods for the next frame image to obtain feature images with different feature image resolutions.

[0027] Furthermore, the expression of the target response score of the response graph in S4 is:

[0028]

[0029] Among them, S pf {x} is the target response score of the response map; x is the image frame; P is the linear combination coefficient matrix; f is the D-channel correlation filter; J{x} is the interpolation result; c is the filter number; d is the d-th dimension channel; pd,c is the linear combination coefficient; f c is the cth filter; Jd{xd} is the interpolation result of the weighted value of each translation function; x d is the weighted value of each translation function; T is the transpose.

[0030] Furthermore, S4 is specifically as follows: performing convolution operations on the ECO tracking filter and feature images with different resolutions of each feature map respectively to obtain a number of response maps with different scales; and performing weighted summation on each response map, taking the location of the maximum response value as the target position, to obtain a positioning completion image of the current frame.

[0031] The beneficial effects of the present invention are: a tracking method for a research target object for real-time tracking of a contact network support device based on a target tracking algorithm based on correlation filtering; on the one hand, it effectively improves the tracking accuracy, and on the other hand, it meets the requirements of deployment and real-time operation on a UAV airborne platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The present invention is a flow chart of the method.

[0033] Figure 2 Schematic diagram of the saliency detection process.

[0034] Figure 3 This is a C-COT principle flow chart of the present invention.

[0035] Figure 4 Schematic diagram comparing the number of filters in the C-COT algorithm and the ECO algorithm.

[0036] in, Figure 4 (a) is C-COT filtering, Figure 4 (b) is ECO filtering. DETAILED DESCRIPTION

[0037] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0038] like Figure 1 As shown, a real-time tracking method for a supporting device based on correlation filtering and saliency detection comprises the following steps:

[0039] S1, obtaining a video to be tested, and annotating the first frame of the video to be tested to obtain a first frame image containing the annotated information, and performing a foreground-background separation operation on the first frame image using an image saliency detection algorithm to obtain a saliency detection map;

[0040] S2. Initialize the ECO tracking filter according to the saliency detection map;

[0041] S3, extracting the feature image of the next frame image;

[0042] S4, performing convolution operation on the ECO tracking filter and the feature image of the next frame image to obtain a response map, and based on the response map, taking the location of the maximum response value as the target position, obtaining a positioning completion image of the current frame;

[0043] S5, classify the positioned image into the training sample set of the ECO tracking filter, and use the Gaussian mixture model to compress and simplify the training sample set, and determine whether the ECO tracking filter is updated in the previous frame tracking. If so, directly go to step S6, otherwise, update the ECO tracking filter based on the training sample set;

[0044] S6. Determine whether the video to be tested has ended. If so, the algorithm terminates. If not, jump to step S3.

[0045] like Figure 2 As shown, the S1 is specifically:

[0046] S101, obtaining a video to be tested, and annotating the first frame of the video to be tested to obtain a first frame image containing the annotated information, and dividing the first frame image into a plurality of image blocks;

[0047] S102, performing DCT transformation on each image block to obtain a frequency domain representation of each image block;

[0048] S103, based on the frequency domain representation of each image block, respectively calculate the sum of the squares of the DCT coefficients of each image block to obtain the energy value of each image block in the frequency domain, that is, the significance value of each image block;

[0049] S104, normalizing the saliency value of each image block, and extracting the image blocks whose normalized saliency value is greater than the saliency threshold, to obtain the sparse salient region of the first frame image;

[0050] S105, perform IDCT inverse transformation on the sparse salient area of ​​the first frame image to obtain a saliency detection map. The expression of the saliency detection map in S1 is:

[0051] M(x)=IDCT(S(x)) 2

[0052] S(x)=sign(DCT(x))

[0053] Among them, M(x) is the reconstructed image obtained by inverse discrete cosine transform, that is, the saliency detection map; IDCT() is the inverse IDCT transform; S(x) is the image saliency feature obtained by discrete cosine transform; sign is the sign function; DCT() is the discrete cosine transform; x is the image frame.

[0054] The objective function of the ECO tracking filter in S2 is:

[0055]

[0056] f=(f 1 ,…,f D )

[0057] P=[P d,c ]

[0058] Where E(f,P) is the objective function of the ECO tracking filter; f is the D-channel correlation filter; P is the linear combination coefficient matrix; is the square of the L2 norm; It is the Fourier transform result of the high-dimensional feature after the feature extraction operation; is the Fourier transform result of the filter; is the Fourier transform result of the expected response value; c is the filter number; C is the number of filters; is the Fourier transform result of the spatial regularization penalty term; is the Fourier transform result of the cth filter; λ is the regularization parameter; is the square of the Frobenius norm; f D is the Dth channel correlation filter; p d,c is the linear combination coefficient; T is the transpose.

[0059] In this embodiment, the basic algorithm used in the present invention is the ECO (Efficient Convolution Operator) algorithm, and the ECO algorithm is mainly improved based on the C-COT algorithm (Continuous Convolution Operator Tracking). Figure 3 Shown is the schematic diagram of the C-COT algorithm.

[0060] The S3 specifically adopts different feature extraction methods for the next frame image to obtain feature images with different feature image resolutions.

[0061] The expression for the target response score of the response graph in S4 is:

[0062]

[0063] Among them, S pf {x} is the target response score of the response map; x is the image frame; P is the linear combination coefficient matrix; f is the D-channel correlation filter; J{x} is the interpolation result; c is the filter number; d is the d-th dimension channel; pd,c is the linear combination coefficient; f c is the cth filter; Jd{xd} is the interpolation result of the weighted value of each translation function; x d is the weighted value of each translation function; T is the transpose.

[0064] The S4 is specifically as follows: performing convolution operations on the ECO tracking filter and feature images with different resolutions of each feature map to obtain a number of response maps with different scales; and performing weighted summation on each response map, taking the location of the maximum response value as the target position, to obtain a positioning completion image of the current frame.

[0065] In this embodiment, in the previous tracking algorithm, the model will save all previous tracking results for model training and filter update. As the tracking progresses, the number of training sets used for model update increases, and there is no shortage of repeated and outdated information. Although retaining each frame of tracking results is conducive to improving the expressiveness of the model, the data redundancy caused by too many training sets will cause the model to slow down and even affect the tracking accuracy.

[0066] Therefore, the ECO algorithm uses a probabilistic generation model to update the training set. The probabilistic generation model used by the algorithm is a Gaussian mixture model. The model reduces the number of training sets by deleting samples with too small weights and fusing similar samples, while ensuring that the training set contains all states of the target, thus ensuring the diversity of the training set samples. Figure 4As shown in the figure, the second line is the training sample set of the traditional model, which includes the tracking results of each frame, and the training samples are too comprehensive; the first line is the training sample set of the ECO algorithm, and the samples include data of various angles and postures. While streamlining the number, the sample categories are more comprehensive.

Claims

1. A real-time tracking method for supporting devices based on correlation filtering and saliency detection, It is characterized in that The following steps are involved: S1, obtaining a video to be tested, and annotating the first frame of the video to be tested to obtain a first frame image containing the annotated information, and performing a foreground-background separation operation on the first frame image using an image saliency detection algorithm to obtain a saliency detection map; S2. Initialize the ECO tracking filter based on the saliency detection map; S3, extracting the feature image of the next frame image; S4, performing convolution operation on the ECO tracking filter and the feature image of the next frame image to obtain a response map, and based on the response map, taking the location of the maximum response value as the target position, obtaining a positioning completion image of the current frame; S5, classify the positioned image into the training sample set of the ECO tracking filter, and use the Gaussian mixture model to compress and simplify the training sample set, and determine whether the ECO tracking filter is updated in the previous frame tracking. If so, directly go to step S6, otherwise, update the ECO tracking filter based on the training sample set; S6. Determine whether the video to be tested has ended. If so, the algorithm terminates. If not, jump to step S3.

2. The method for real-time tracking of a supporting device based on correlation filtering and saliency detection according to claim 1, It is characterized in that The S1 is specifically: S101, obtaining a video to be tested, and annotating the first frame of the video to be tested to obtain a first frame image containing the annotated information, and dividing the first frame image into a plurality of image blocks; S102, performing DCT transformation on each image block to obtain a frequency domain representation of each image block; S103, based on the frequency domain representation of each image block, respectively calculate the sum of the squares of the DCT coefficients of each image block to obtain the energy value of each image block in the frequency domain, that is, the significance value of each image block; S104, normalizing the saliency value of each image block, and extracting the image blocks whose normalized saliency value is greater than the saliency threshold, to obtain the sparse salient region of the first frame image; S105 , performing an IDCT inverse transform on the sparse salient region of the first frame image to obtain a saliency detection map.

3. The method for real-time tracking of a supporting device based on correlation filtering and saliency detection according to claim 1, It is characterized in that The expression of the saliency detection map in S1 is: M(x)=IDCT(S(x)) 2 S(x)=sign(DCT(x)) Among them, M(x) is the reconstructed image obtained by inverse discrete cosine transform, that is, the saliency detection map; IDCT() is the inverse IDCT transform; S(x) is the image saliency feature obtained by discrete cosine transform; sign is the sign function; DCT() is the discrete cosine transform; x is the image frame.

4. The method for real-time tracking of a supporting device based on correlation filtering and saliency detection according to claim 1, It is characterized in that The objective function of the ECO tracking filter in S2 is: f=(f 1 ,…,f D ) P=[P d,c ] Where E(f,P) is the objective function of the ECO tracking filter; f is the D-channel correlation filter; P is the linear combination coefficient matrix; is the square of the L2 norm; It is the Fourier transform result of the high-dimensional feature after the feature extraction operation; is the Fourier transform result of the filter; is the Fourier transform result of the expected response value; c is the filter number; C is the number of filters; is the Fourier transform result of the spatial regularization penalty term; is the Fourier transform result of the cth filter; λ is the regularization parameter; is the square of the Frobenius norm; f D is the Dth channel correlation filter; p d,c is the linear combination coefficient; T is the transpose.

5. The method for real-time tracking of a supporting device based on correlation filtering and saliency detection according to claim 1, It is characterized in that The S3 specifically adopts different feature extraction methods for the next frame image to obtain feature images with different feature image resolutions.

6. The method for real-time tracking of a supporting device based on correlation filtering and saliency detection according to claim 1, It is characterized in that The expression for the target response score of the response graph in S4 is: Among them, S pf {x} is the target response score of the response map; x is the image frame; P is the linear combination coefficient matrix; f is the D-channel correlation filter; J{x} is the interpolation result; c is the filter number; d is the d-th dimension channel; pd,c is the linear combination coefficient; f c is the cth filter; Jd{xd} is the interpolation result of the weighted value of each translation function; x d is the weighted value of each translation function; T is the transpose.

7. The method for real-time tracking of supporting devices for correlation filtering and saliency detection according to claim 5, It is characterized in that The S4 is specifically as follows: performing convolution operations on the ECO tracking filter and feature images with different resolutions of each feature map to obtain a number of response maps with different scales; and performing weighted summation on each response map, taking the location of the maximum response value as the target position, to obtain a positioning completion image of the current frame.

Images