A small target tracking method and system based on correlation filtering improvement

Abstract

The application provides a small target tracking method and system based on improved correlation filtering, which comprises the following steps: cropping an initial target area and pre-processing an image, performing two-dimensional Fourier transform and making a correlation filtering tracking template, tracking in the target area, judging whether the target is lost according to the statistical result, inputting the tracking coordinates into a Kalman filter to predict the position of the next frame template, and updating the tracking template using a new target area. The method improves the original correlation filtering tracker by taking advantage of the feature of the Kalman filter that can predict the future position of the target. The system can improve the problems of target occlusion, too fast target motion, complex target background and the like in the small target tracking task under the premise of realizing high robustness of small target transformation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target tracking, and particularly relates to a small target tracking method and system based on improved correlation filtering. BACKGROUND

[0002] The target tracking technology based on correlation filtering is a technology for tracking a known target in continuous videos. A frequency domain image is obtained by performing two-dimensional Fourier transform on a target region, time domain correlation operation of traditional template tracking is converted into frequency domain operation, processing speed is accelerated, and real-time tracking is ensured. In addition, a learning rate is set to calculate and update a filter template, and tracking capability when a target to be measured is morphologically transformed is improved. The technology is widely used in the field of real-time target tracking.

[0003] Traditional target tracking algorithms mainly adopt a method of modeling a target or tracking a specific feature of the target. Such methods include region matching, feature point matching, and optical flow method. A commonly used method is to use target features to perform feature matching in adjacent frames. Such features include SIFT features, SURF features, and Harris corner features. However, traditional target tracking methods still face challenges in actual applications, especially in small target tracking tasks. First, the traditional algorithms cannot well cope with the morphological changes of the tracked target. When the target is morphologically changed, the tracking effect of the original observation model and the features will be greatly reduced. In addition, when the tracking task is performed, the target may be partially occluded or temporarily disappeared. At this time, the tracker will fail to find the corresponding model, resulting in tracking failure.

[0004] The target tracker based on correlation filtering trains an arbitrary affine transformation of a target to be tracked, and updates the filter template at each tracking time by setting a learning rate, so that it has high robustness to target changes. Although the target tracking technology based on correlation filtering is quite mature, there are still problems in small target tracking tasks. First, in order to reduce the influence of Fourier transform spectrum leakage on tracking effect, the correlation filtering method performs windowing on the measured region, which causes the target to be on the edge of the measured window when moving fast, thereby losing features. In addition, when performing a small target tracking task, the target is easily partially occluded or temporarily disappeared, resulting in tracking failure. Therefore, how to solve these problems under the premise of ensuring the tracking effect of the correlation filtering tracker is one of the important technical problems in the field. SUMMARY

[0005] The present application aims to provide a small target tracking method and system based on improved correlation filtering, which can be used to improve the tracking capability of traditional correlation filtering in the case of fast target motion, target partial occlusion, and temporary disappearance.

[0006] The technical solution for achieving the object of the present application is:

[0007] A small target tracking method based on correlation filtering improvement, comprising:

[0008] Step 1, crop the target area and pre-process the collected target image, if the target image is the first frame, execute step 2, otherwise execute step 3;

[0009] Step 2, perform two-dimensional Fourier transform on the pre-processed first frame target image and determine the filter tracking template, and execute step 7;

[0010] Step 3, multiply the pre-processed target image by the filter tracking template to obtain a response map;

[0011] Step 4, calculate the peak-to-sidelobe ratio of the response map, if the peak-to-sidelobe ratio is lower than a set value, execute step 5, otherwise execute step 6;

[0012] Step 5, input the tracking coordinates in the response map into the Kalman filter to predict the target position of the next frame, and execute step 7;

[0013] Step 6, update the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response map;

[0014] Step 7, repeat steps 1-6 to perform real-time tracking in the target area.

[0015] Further, the step 1 of cropping the initial target area and pre-processing the collected target image specifically comprises:

[0016] Step 1-1, convert the collected original target image into a gray image;

[0017] Step 1-2, logarithmically transform the gray image, which is

[0018] s=clog(1+r)

[0019] Wherein, c is a constant, used to adjust the intensity of the transformation, r is the input gray value, and s is the output gray value;

[0020] Step 1-3, multiply the target area by the Hanning window, and gradually change the edge of the transformed gray image to zero.

[0021] Further, the Hanning window is:

[0022]

[0023] Wherein, n is the serial number of the discrete sequence, and N represents the length of the sequence.

[0024] Further, the step 2 of performing two-dimensional Fourier transform on the preprocessed first frame target image and determining a filter tracking template comprises:

[0025] Step 2-1, performing random affine transformation on the target image to construct a series of training images;

[0026] Step 2-2, based on the training images, using a two-dimensional Gaussian function to make an expected response map, and generating a Gaussian matrix at the target center; the two-dimensional Gaussian function is:

[0027]

[0028] Step 2-3, based on a minimization problem, solving a frequency domain H* of the filter tracking template to obtain the filter tracking template.

[0029] Further, the minimization problem is described as:

[0030]

[0031] Wherein F i is a frequency domain image of an input image, G i is a frequency domain image of a two-dimensional Gaussian distribution;

[0032] The H * is:

[0033]

[0034] Wherein F i * is a frequency domain image of an input image.

[0035] Further, the peak-to-sidelobe ratio in the step 4 is:

[0036]

[0037] Wherein g max is a peak value of a response map region, μ sl and σ sl are the mean and standard deviation of the sidelobe, respectively.

[0038] Further, the Kalman filter prediction formula of the Kalman filter in the step 5 is:

[0039]

[0040] Wherein, is a current predicted estimation value, A represents a state transition matrix during prediction, u k represents a control input vector at time k, B represents a control matrix, represents a prediction covariance matrix at time k, and Q represents a prediction noise covariance matrix.

[0041] Further, the Kalman filter parameter update in step 6 is:

[0042]

[0043] In the formula, z k represents an observation value, H is an observation matrix, R represents a measurement noise matrix, K k represents a Kalman gain at time k, P k respectively represent the corrected estimated value and the covariance matrix.

[0044] Further, the filter tracking template update in step 6 is:

[0045]

[0046] A i =ηG i ⊙F i * +(1-η)A i-1

[0047] B i =ηF i ηF i * +B i-1

[0048] Wherein, η represents a learning rate, F i is a frequency domain image of an input image, G i is a frequency domain image of a two-dimensional Gaussian distribution.

[0049] An improved small target tracking system based on correlation filtering, comprising:

[0050] An image preprocessing unit for cropping a target region and preprocessing the collected target image;

[0051] A filter tracking template determination unit for performing two-dimensional Fourier transform on the preprocessed first frame target image and making a filter tracking template;

[0052] A response map generation unit for multiplying the preprocessed target image by the filter tracking template to obtain a response map;

[0053] A sidelobe ratio calculation unit for calculating the peak sidelobe ratio of the response map;

[0054] A Kalman filter for predicting the position of the next frame target;

[0055] A filter tracking template and Kalman filter parameter update unit for updating the filter tracking template and the Kalman filter parameters according to the tracking coordinates in the response map.

[0056] Compared with the prior art, the present application has the beneficial effects that the present application improves the original correlation filter tracker by taking advantage of the feature of Kalman filter that can predict the future position of the target; the present application can improve the problems of target occlusion, too fast target motion speed and complex target background in the small target tracking task under the premise of realizing high robustness of small target transformation. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The contrastive image of the preprocessed image provided for the example of the present application, Figure 1 (a) is the original image, and (b) is the image after preprocessing.

[0058] Figure 2 The contrastive image of the target image and the image after superimposing the Hann window provided for the example of the present application, Figure 2 (a) is the target image, and (b) is the target image after superimposing the Hann window.

[0059] Figure 3 The schematic diagram of two-dimensional Gaussian distribution as an ideal response provided for the example of the present application.

[0060] Figure 4 The time-domain image of the tracking template provided for the example of the present application.

[0061] Figure 5 The contrastive image of the tracking target and the response result provided for the example of the present application, Figure 5 (a) is the tracking target image, and (b) is the response image.

[0062] Figure 6 The contrastive image of the tracking target and the response result when the PSR value is lower than the threshold provided for the example of the present application, Figure 6 (a) is the tracking target image, and (b) is the response image.

[0063] Figure 7 The result image of using Kalman filter to predict under low PSR value provided for the example of the present application, Figure 7 (a), (b), (c) and (d) are respectively the result images of the 1st frame, the 2nd frame, the n-th frame and the n+1-th frame.

[0064] Figure 8 The flowchart provided for the example of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0066] The embodiments of the present application will be first introduced below with reference to the drawings.

[0067] In this embodiment, the small target is a small unmanned aerial vehicle, combined with Figure 8 An improved small unmanned aerial vehicle tracking method based on correlation filtering, comprising the following steps:

[0068] Step 1, crop the target area and pre-process the collected target image, if the target image is the first frame, execute step 2, otherwise execute step 3;

[0069] Specifically, the pre-processing is divided into the following steps:

[0070] Step 1-1, if the input is a color image, convert it to a grayscale image.

[0071] Step 1-2, log transform the grayscale image to increase the contrast of the image. The image processing result is shown in Figure 1 The log transform of the image is as follows:

[0072] s = c log (1 + r)

[0073] Where c is a constant used to adjust the strength of the transformation. r is the input grayscale value, and s is the output grayscale value.

[0074] Step 1-3, multiply the target area by the Hanning window, gradually change the image edge to zero, and reduce the influence of Fourier transform leakage on the result. The Hanning window is as follows:

[0075]

[0076] Where n is the sequence number of the discrete sequence, and N represents the length of the sequence. The Hanning window function has the characteristic of reducing the amplitude of the sequence at both ends in the time domain, thereby reducing the influence of spectral leakage. Compared with other window functions, it has a narrower main lobe width and lower sidelobe leakage. The comparison before and after multiplying the target area by the Hanning window is shown in Figure 2 .

[0077] Step 2, perform two-dimensional Fourier transform on the pre-processed first frame target image and make a filter tracking template, execute step 7;

[0078] Specifically, the calculation of the correlation filter template is divided into the following steps:

[0079] Step 2-1, make a series of training images by randomly affine transforming the target image;

[0080] Step 2-2, use a two-dimensional Gaussian function to make an ideal response map, generate a Gaussian matrix with the target center, and the two-dimensional Gaussian distribution when σ = 2 is shown in Figure 3 The two-dimensional Gaussian function is as follows:

[0081]

[0082] Step 2-3, solve H* to get the filter tracking template, in order to find the filter that will map the training input to the desired training output, by finding a filter H that minimizes the sum of the squared error between the actual output of the convolution and the desired output of the convolution. This minimization problem takes the following form:

[0083]

[0084] Where F i is the frequency domain image of the input image, G i is the frequency domain image of a two-dimensional Gaussian distribution, by solving H * The closed form expression for the tracking filter is given below, where H * The time domain image is shown in Figure 4 .

[0085]

[0086] Step 3, multiply the tracking region frequency domain image by the filter template to get the response result, the tracking target and the response map are shown in Figure 5 , the peak value of the response map is the tracking position.

[0087] Step 4, determine whether the target is lost by counting the results, the basis for the judgment is the PSR (peak side lobe ratio) value of the response map, calculate the peak side lobe ratio value of the response map. When the PSR value meets the requirements, it is considered that the target is tracked, and step 6 is executed; if the PSR value is lower than the set value, step 5 is executed, and the Kalman filter prediction value is used for tracking again until the target is locked.

[0088] Specifically, determining whether the target is lost and subsequent tracking includes the following steps:

[0089] Step 4-1, inverse Fourier transform the response map to get the time domain image.

[0090] Step 4-2, find the response peak value and calculate the average and standard deviation of the corresponding map, calculate the PSR value, the PSR value formula is shown below, where g max is the region peak value, μ sl and σ sl are the mean and standard deviation of the side lobe, respectively:

[0091]

[0092] Step 4-3, determine whether the target is correctly tracked according to the PSR value, if the PSR value is lower than the set value as shown in Figure 6 , step 5 is executed, and the Kalman filter prediction value is used for tracking again until the target is locked.

[0093] Step 5, input the tracking coordinates into the Kalman filter to predict the target position of the next frame;

[0094] Specifically, the Kalman filter prediction formula is as follows:

[0095]

[0096] wherein, is the current predicted estimate, A represents the state transition matrix at the time of prediction, u k represents the control input vector at time k, B represents the control matrix, represents the prediction covariance matrix at time k, Q represents the prediction noise covariance matrix.

[0097] Step 6, update the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response map;

[0098] Specifically, the relevant filter template update formula is as follows:

[0099]

[0100] A i = ηG i ⊙F i * +(1-η)A i-1

[0101] B i = ηF i ⊙F i * +B i-1

[0102] wherein η represents the learning rate of the tracker, the higher the value of the learning rate represents that the tracker pays more attention to the target state of the recent frame, and makes the influence of the previous frame decay exponentially. F i is the frequency domain image of the new input image, G i is the frequency domain image of the two-dimensional Gaussian distribution;

[0103] Finally, update the tracking template according to the calculation result, and continue to track the target in the next frame. Specifically, it includes:

[0104] Kalman filter parameter update, if the PSR value is lower than the set value, do not update, but predict again through the prediction value of the last Kalman filter. In the case as shown in Figure 7 , in Figure 7 (b) start target loss, use Kalman filter prediction until Figure 7 (d) the PSR value is higher than the threshold value and the tracking is successful. The Kalman filter update formula is as follows:

[0105]

[0106] wherein z k represents the observation value transmitted by the detection algorithm, H is an observation matrix, R represents a measurement noise matrix of the detector, K k represents the Kalman gain at time k, P k respectively represent the corrected estimated value and the covariance matrix, which are used for the next prediction.

[0107] Step 7, repeat steps 1-6 to perform real-time tracking in the target area.

[0108] The embodiment also provides a small target tracking system based on the improved correlation filtering, comprising:

[0109] an image preprocessing unit configured to crop a target area and pre-process the collected target image;

[0110] a filter tracking template determination unit configured to perform two-dimensional Fourier transform on the pre-processed first frame of target image and make a filter tracking template;

[0111] a response map generation unit configured to multiply the pre-processed target image by the filter tracking template to obtain a response map;

[0112] a sidelobe ratio calculation unit configured to calculate a peak sidelobe ratio of the response map;

[0113] a Kalman filter configured to predict a next frame of target position;

[0114] a filter tracking template and Kalman filter parameter updating unit configured to update the filter tracking template and the Kalman filter parameter according to the tracking coordinates in the response map.

[0115] For the purpose of illustration, the above specific embodiments are only exemplary, and are intended to enable those skilled in the art to better understand the patent, and cannot be understood as a limitation on the scope of the patent; any technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the patent.

Claims

1. A method for tracking small targets based on correlation filtering improvement, characterized in that, The method comprises the following steps: Step 1, cropping a target region and pre-processing a collected target image, if the target image is a first frame, executing step 2, otherwise executing step 3; Step 2, performing two-dimensional Fourier transform on the pre-processed first frame target image and determining a filter tracking template, and executing step 7; Step 3, multiplying the pre-processed target image by the filter tracking template to obtain a response map; Step 4, calculating a peak-to-sidelobe ratio of the response map, if the peak-to-sidelobe ratio is lower than a set value, executing step 5, otherwise executing step 6; Step 5, inputting tracking coordinates in the response map into a Kalman filter to predict a target position of a next frame, and executing step 7; Step 6, updating the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response map; Step 7, repeating steps 1-6 to perform real-time tracking in the target region. In the step 2, the two-dimensional Fourier transform on the pre-processed first frame target image and the determination of the filter tracking template comprise the following steps: Step 2-1, performing random affine transformation on the target image to construct a series of training images; Step 2-2, based on the training images, using a two-dimensional Gaussian function to make an expected response map, and generating a Gaussian matrix from a target center; the two-dimensional Gaussian function is: Step 2-3, based on a minimization problem, solving a frequency domain H* of the filter tracking template to obtain the filter tracking template; The minimization problem is described as: where F i is a frequency domain image of the input image, G i is a frequency domain image of a two-dimensional Gaussian distribution; The H * is: where F i * is a frequency domain image of the input image.

2. The improved small target tracking method based on correlation filtering according to claim 1, characterized in that, In the step 1, the cropping of the initial target region and the pre-processing of the collected target image specifically comprise the following steps: Step 1-1, converting the collected original target image into a gray image; Step 1-2, performing logarithmic transformation on the gray image, s=clog(1+r) Wherein, c is a constant, used for adjusting the intensity of the transformation, r is an input gray value, and s is an output gray value; Step 1-3, multiplying the target region by a Hanning window, and gradually changing the edge of the transformed gray image to zero.

3. The improved small target tracking method based on correlation filtering according to claim 2, characterized in that, The Hanning window is: Wherein, n is the serial number of the discrete sequence, and N represents the length of the sequence.

4. The improved small target tracking method based on correlation filtering according to claim 1, characterized in that, In the step 4, the peak-to-sidelobe ratio is: where g max is the response map region peak value, μ sl and σ sl are the mean and standard deviation of the sidelobes, respectively.

5. The improved small target tracking method based on correlation filtering according to claim 1, characterized in that, In the step 5, the Kalman filter prediction formula of the Kalman filter is: wherein, is an estimated value of the current prediction, A denotes a state transition matrix at the time of prediction, u k denotes a control input vector at time k, B denotes a control matrix, denotes a prediction covariance matrix at time k, Q denotes a prediction noise covariance matrix.

6. The improved small target tracking method based on correlation filtering according to claim 5, characterized in that, In the step 6, the Kalman filter parameter update is: In the formula, z k represents an observation value, H is an observation matrix, R represents a measurement noise matrix, K k represents a Kalman gain at time k, P k respectively represent a corrected estimated value and a covariance matrix.

7. The improved small target tracking method based on correlation filtering according to claim 5, characterized in that, In the step 6, the filter tracking template update is: A i = ηG i ⊙F i * +(1-η)A i-1 B i = ηF i ⊙F i * +B i-1 wherein η represents a learning rate, F i is a frequency domain image of the input image, G i is a frequency domain image of a two-dimensional Gaussian distribution.

8. An improved small target tracking system based on correlation filtering, which implements the method of any one of claims 1-7. The method comprises the following steps: An image pre-processing unit, configured to crop a target region and pre-process a collected target image; A filter tracking template determination unit, configured to perform two-dimensional Fourier transform on the pre-processed first frame target image and make a filter tracking template; A response map generation unit, configured to multiply the pre-processed target image by the filter tracking template to obtain a response map; A sidelobe ratio calculation unit, configured to calculate a peak-to-sidelobe ratio of the response map; A Kalman filter, configured to predict a target position of a next frame; A filter tracking template and Kalman filter parameter update unit, configured to update the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response map.

Images