Small target tracking method and system based on correlation filtering improvement

Abstract

The invention provides a small target tracking method and system based on correlation filtering improvement. The method comprises the steps of cutting an initial target area and preprocessing an image, performing two-dimensional Fourier transform and making a correlation filtering tracking template, tracking in the target area, counting a result to judge whether a target is lost, inputting a tracking coordinate into a Kalman filter to predict the position of a next frame of template, and obtaining the position of the next frame of template. And updating the tracking template by using the new target area. According to the method, the characteristic that a Kalman filter can predict the future position of a target is utilized, and an original correlation filtering tracker is improved. The system can improve the problems that the target is shielded, the target movement speed is too high, the target background is complex and the like in a small target tracking task on the premise of realizing high robustness of small target transformation.

Description

Technical Field

[0001] The present application relates to the field of target tracking, and in particular to a small target tracking method and system based on improved correlation filtering. Background Art

[0002] Target tracking technology based on correlation filtering is a technology used to track known targets in continuous videos. It obtains frequency domain images by performing two-dimensional Fourier transform on the target area, converts the time domain correlation operation of traditional template tracking into frequency domain operation, speeds up the processing speed, and ensures the real-time tracking. In addition, the learning rate is set to calculate and update the filter template, which improves the tracking ability when the target undergoes morphological transformation. This technology is widely used in the field of real-time target tracking.

[0003] Traditional target tracking algorithms mainly use methods to model the target or track specific features of the target, such as region matching, feature point matching, optical flow method, etc. The more commonly used method is to use target features to match features in adjacent frames, such features include SIFT features, SURF features, Harris corner features, etc. However, traditional target tracking methods still face challenges in practical applications, especially in small target tracking tasks. First of all, traditional algorithms cannot cope well with the morphological changes of the tracked target. When the target changes its posture, the tracking effect of the original observation model and features will be greatly reduced. In addition, when the tracking task is in progress, the target may be partially obscured or disappear temporarily. At this time, the tracker will fail to track because it cannot find the corresponding model.

[0004] The target tracker based on correlation filtering trains the target to be tracked by any affine transformation, and updates the filter template each time it is tracked by setting the learning rate, so that it has high robustness against target changes. Although the target tracking technology based on correlation filtering is quite mature, there are still problems in the small target tracking task. First, in order to reduce the impact of Fourier transform spectrum leakage on the tracking effect, the correlation filtering method performs windowing on the measured area, which causes the target to be at the edge of the measured window when it moves quickly, thereby losing features. In addition, when performing small target tracking tasks, it is very easy for the target to be partially blocked or disappear temporarily, resulting in tracking failure. Therefore, how to solve these problems while ensuring the tracking effect of the correlation filter tracker is one of the important technical issues in this field. Summary of the invention

[0005] The purpose of the present application is to provide a small target tracking method and system based on improved correlation filtering, which can be used to improve the tracking ability of traditional correlation filtering when encountering fast target motion, partial occlusion and temporary disappearance of the target.

[0006] The technical solution to achieve the purpose of the present invention is:

[0007] A small target tracking method based on improved correlation filtering, comprising:

[0008] Step 1, crop the target area and pre-process the acquired target image. If the target image is the first frame, execute step 2, otherwise execute step 3;

[0009] Step 2, performing a two-dimensional Fourier transform on the preprocessed first frame target image and determining a filter tracking template, and executing step 7;

[0010] Step 3, multiply the preprocessed target image by the filtered tracking template to obtain a response image;

[0011] Step 4, calculating the peak-to-sidelobe ratio of the response graph, if the peak-to-sidelobe ratio is lower than the set value, executing step 5, otherwise executing step 6;

[0012] Step 5, input the tracking coordinates in the response graph into the Kalman filter to predict the target position of the next frame, and execute step 7;

[0013] Step 6, updating the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response graph;

[0014] Step 7, repeat steps 1-6 to perform real-time tracking in the target area.

[0015] Furthermore, in step 1, the initial target area is cropped and the acquired target image is preprocessed, specifically including:

[0016] Step 1-1, converting the collected original target image into a grayscale image;

[0017] Step 1-2, transform the grayscale image logarithmically to

[0018] s=clog(1+r)

[0019] Among them, c is a constant used to adjust the intensity of the transformation, r is the input grayscale value, and s is the output grayscale value;

[0020] Step 1-3, multiply the target area by the Hanning window to gradually change the edge of the transformed grayscale image to zero.

[0021] Furthermore, the Hanning window is:

[0022]

[0023] Among them, n is the serial number of the discrete sequence, and N represents the length of the sequence.

[0024] Furthermore, in step 2, performing a two-dimensional Fourier transform on the preprocessed first frame target image and determining a filtering tracking template includes:

[0025] Step 2-1, perform random affine transformation on the target image to construct a series of training images;

[0026] Step 2-2, based on the training image, use a two-dimensional Gaussian function to make an expected response map, and generate a Gaussian matrix with the target center; the two-dimensional Gaussian function is:

[0027]

[0028] Step 2-3, solving the frequency domain H* of the filtered tracking template based on the minimization problem to obtain the filtered tracking template.

[0029] Further, the minimization problem is described as:

[0030]

[0031] where F i is the frequency domain image of the input image, G i is the frequency domain image of the two-dimensional Gaussian distribution;

[0032] The H * for:

[0033]

[0034] Among them, F i * is the frequency domain image of the input image.

[0035] Furthermore, the peak sidelobe ratio in step 4 is:

[0036]

[0037] Among them, g max is the peak value of the response graph area, μ sl and σ sl are the mean and standard deviation of the side lobes, respectively.

[0038] Furthermore, the Kalman filter prediction formula of the Kalman filter in step 5 is:

[0039]

[0040] in, is the estimated value of the current prediction, A represents the state transfer matrix during prediction, and u k represents the control input vector at time k, B represents the control matrix, represents the prediction covariance matrix at time k, and Q represents the prediction noise covariance matrix.

[0041] Furthermore, in step 6, the Kalman filter parameters are updated as follows:

[0042]

[0043] In the formula, z k represents the observation value, H is the observation matrix, R represents the measurement noise matrix, K k represents the Kalman gain at time k, P k denote the corrected estimates and covariance matrix, respectively.

[0044] Furthermore, in step 6, the filter tracking template is updated as:

[0045]

[0046] A i =ηG i ⊙F i * +(1-η)A i-1

[0047] B i =ηF i ηF i * +B i-1

[0048] Among them, η represents the learning rate, F i is the frequency domain image of the input image, G i is the frequency domain image of the two-dimensional Gaussian distribution.

[0049] A small target tracking system based on improved correlation filtering, comprising:

[0050] An image preprocessing unit, used for cropping a target area and preprocessing the acquired target image;

[0051] A filter tracking template determination unit performs a two-dimensional Fourier transform on the preprocessed first frame target image and creates a filter tracking template;

[0052] A response map generating unit, which multiplies the preprocessed target image by the filter tracking template to obtain a response map;

[0053] A sidelobe ratio calculation unit, used for calculating a peak sidelobe ratio of a response graph;

[0054] Kalman filter, used to predict the target position in the next frame;

[0055] The wave tracking template and Kalman filter parameter updating unit updates the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response graph.

[0056] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention utilizes the characteristic of the Kalman filter that can predict the future position of the target to improve the original correlation filter tracker; the present invention can improve the problems of target occlusion, target movement speed being too fast, and target background complexity in small target tracking tasks while achieving high robustness to small target transformations. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A comparison chart of the preprocessed images provided in this application example. Figure 1 (a) is the original image, and (b) is the preprocessed image.

[0058] Figure 2 The target image provided in this application example and the comparison image after superimposing the Hanning window, Figure 2 (a) is the target image, and (b) is the target image with the Hanning window superimposed.

[0059] Figure 3 Schematic diagram of a two-dimensional Gaussian distribution as an ideal response provided in this application example.

[0060] Figure 4 This is the time domain image of the tracking template provided for this application example.

[0061] Figure 5 A comparison chart of the tracking targets and response results provided for this application example. Figure 5 (a) is the tracking target map, and (b) is the response map.

[0062] Figure 6 A comparison chart of the tracking target and response results when the PSR value provided for this application example is lower than the threshold. Figure 6 (a) is the tracking target map, and (b) is the response map.

[0063] Figure 7 The result diagram of using Kalman filtering to predict at low PSR values ​​provided in this application example, Figure 7 (a), (b), (c), and (d) are the result images of the 1st frame, the 2nd frame, the nth frame, and the n+1th frame, respectively.

[0064] Figure 8 Flowchart provided for this application example. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0066] The following first introduces the embodiments of the present application in conjunction with the accompanying drawings.

[0067] In this embodiment, the small target is a small drone. Figure 8 , a small UAV tracking method based on correlation filtering improvement includes the following steps:

[0068] Step 1, crop the target area and pre-process the acquired target image. If the target image is the first frame, execute step 2, otherwise execute step 3;

[0069] Specifically, the preprocessing is divided into the following steps:

[0070] Step 1-1: If the input is a color image, convert it to a grayscale image.

[0071] Step 1-2, logarithmically transform the grayscale image to increase the contrast of the image. The image processing result is as follows Figure 1 As shown, the image logarithmic transformation is as follows:

[0072] s=clog(1+r)

[0073] Where c is a constant used to adjust the intensity of the transformation, r is the input grayscale value, and s is the output grayscale value.

[0074] Steps 1-3, multiply the target area by the Hanning window, gradually change the image edge to zero, and reduce the impact of Fourier transform leakage on the result. The Hanning window is as follows:

[0075]

[0076] Where n is the number of the discrete sequence, and N is the length of the sequence. The characteristic of the Hanning window function is that it can reduce the amplitude at both ends of the sequence in the time domain, thereby reducing the impact of spectrum leakage. Compared with other window functions, it has a narrower main lobe width and lower side lobe leakage. The comparison before and after the point multiplication of the Hanning window in the tracking target area is as follows: Figure 2 shown.

[0077] Step 2, perform a two-dimensional Fourier transform on the preprocessed first frame target image and create a filter tracking template, and then execute step 7;

[0078] Specifically, the calculation of the relevant filter template is divided into the following steps:

[0079] Step 2-1, perform random affine transformation on the target image to construct a series of training images;

[0080] Step 2-2, use the two-dimensional Gaussian function to make an ideal response diagram, generate a Gaussian matrix with the target center, and the two-dimensional Gaussian distribution when σ = 2 is as follows Figure 3 As shown, the two-dimensional Gaussian function is as follows:

[0081]

[0082] Step 2-3, solve H* to obtain the filtered tracking template, in order to find the filter that maps the training input to the desired training output, by finding a filter H that minimizes the sum of the squared errors between the actual output of the convolution and the desired output of the convolution. This minimization problem takes the following form:

[0083]

[0084] where F i is the frequency domain image of the input image, G i is the frequency domain image of the two-dimensional Gaussian distribution. By solving H * The closed-form expression for the tracking filter is found as follows, where H * Time domain images such as Figure 4 shown.

[0085]

[0086] Step 3: Multiply the frequency domain image of the tracking area by the filter template to obtain the response result. The tracking target and the response are shown in the figure below. Figure 5 As shown, the peak of the corresponding graph is the tracking position.

[0087] Step 4, statistical results to determine whether the target is lost; the basis for judgment is the PSR (peak sidelobe ratio) value of the response graph, and the peak sidelobe ratio of the response graph is calculated. When the PSR value meets the requirements, it is considered that the target is tracked, and step 6 is executed; if the PSR value is lower than the set value, step 5 is executed, and the Kalman filter prediction value is used to track again until the target is locked.

[0088] Specifically, determining whether the target is lost and subsequent tracking includes the following steps:

[0089] Step 4-1, perform inverse Fourier transform on the response graph to obtain a time domain image.

[0090] Step 4-2, find the response peak and calculate the mean and standard deviation of the corresponding graph, and calculate the PSR value. The PSR value formula is as follows, where g max is the regional peak value, μ sl and σ sl are the mean and standard deviation of the side lobes respectively:

[0091]

[0092] Step 4-3, determine whether the target is tracked correctly based on the PSR value. Figure 6 In the case shown, if the PSR value is lower than the set value, execute step 5 and use the Kalman filter prediction value to track again until the target is locked.

[0093] Step 5, input the tracking coordinates into the Kalman filter to predict the target position of the next frame;

[0094] Specifically, the Kalman filter prediction formula is as follows:

[0095]

[0096] in, is the estimated value of the current prediction, A represents the state transfer matrix during prediction, and u k represents the control input vector at time k, B represents the control matrix, represents the prediction covariance matrix at time k, and Q represents the prediction noise covariance matrix.

[0097] Step 6, updating the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response graph;

[0098] Specifically, the relevant filter template update formula is as follows:

[0099]

[0100] A i =ηG i ⊙F i * +(1-η)A i-1

[0101] B i =ηF i ⊙F i * +B i-1

[0102] Where η represents the learning rate of the tracker. A higher learning rate means that the tracker pays more attention to the target state of the most recent frame and exponentially decays the influence of the previous frame. i is the frequency domain image of the new input image, G i is the frequency domain image of the two-dimensional Gaussian distribution;

[0103] Finally, the tracking template is updated according to the calculation results, and the target is tracked in the next frame. Specifically, it includes:

[0104] Kalman filter parameters are updated. If the PSR value is lower than the set value, no update is performed and only the prediction value of the last Kalman filter is used for prediction. Figure 7 In the case shown Figure 7 (b) The target starts to be lost and the Kalman filter is used to predict until Figure 7 (d) The PSR value is higher than the threshold and the tracking is successful. The Kalman filter update formula is as follows:

[0105]

[0106] In the formula, z k represents the observation value transmitted by the detection algorithm, H is the observation matrix, R represents the measurement noise matrix of the detector, K k represents the Kalman gain at time k, P k They represent the revised estimated value and covariance matrix, which are used for the next prediction.

[0107] Step 7, repeat steps 1-6 to perform real-time tracking in the target area.

[0108] This embodiment also provides a small target tracking system based on correlation filtering improvement, including:

[0109] An image preprocessing unit, used for cropping a target area and preprocessing the acquired target image;

[0110] A filter tracking template determination unit performs a two-dimensional Fourier transform on the preprocessed first frame target image and creates a filter tracking template;

[0111] A response map generating unit, which multiplies the preprocessed target image by the filter tracking template to obtain a response map;

[0112] A sidelobe ratio calculation unit, used for calculating a peak sidelobe ratio of a response graph;

[0113] Kalman filter, used to predict the target position in the next frame;

[0114] The wave tracking template and Kalman filter parameter updating unit updates the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response graph.

[0115] For the purpose of explanation, the above-mentioned specific implementation methods are only exemplary, in order to better enable those skilled in the art to understand this patent, and should not be understood as limiting the scope of this patent; any technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A small target tracking method based on improved correlation filtering, characterized in that: include: Step 1, crop the target area and pre-process the acquired target image. If the target image is the first frame, execute step 2, otherwise execute step 3; Step 2, performing a two-dimensional Fourier transform on the preprocessed first frame target image and determining a filter tracking template, and executing step 7; Step 3, multiply the preprocessed target image by the filtered tracking template to obtain a response image; Step 4, calculating the peak-to-sidelobe ratio of the response graph, if the peak-to-sidelobe ratio is lower than the set value, executing step 5, otherwise executing step 6; Step 5, input the tracking coordinates in the response graph into the Kalman filter to predict the target position of the next frame, and execute step 7; Step 6, updating the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response graph; Step 7, repeat steps 1-6 to perform real-time tracking in the target area.

2. According to claim 1, a small target tracking method based on improved correlation filtering is characterized in that: In step 1, the initial target area is cut and the acquired target image is preprocessed, specifically including: Step 1-1, converting the collected original target image into a grayscale image; Step 1-2, transform the grayscale image logarithmically to s = clog (1 + r) Among them, c is a constant used to adjust the intensity of the transformation, r is the input grayscale value, and s is the output grayscale value; Step 1-3, multiply the target area by the Hanning window to gradually change the edge of the transformed grayscale image to zero.

3. The small target tracking method based on improved correlation filtering according to claim 2 is characterized in that: The Hanning window is: Among them, n is the serial number of the discrete sequence, and N represents the length of the sequence.

4. The small target tracking method based on improved correlation filtering according to claim 1 is characterized in that: In the step 2, performing a two-dimensional Fourier transform on the preprocessed first frame target image and determining a filtering tracking template includes: Step 2-1, perform random affine transformation on the target image to construct a series of training images; Step 2-2, based on the training image, use a two-dimensional Gaussian function to make an expected response map, and generate a Gaussian matrix with the target center; the two-dimensional Gaussian function is: Step 2-3, solving the frequency domain H* of the filtered tracking template based on the minimization problem to obtain the filtered tracking template.

5. The small target tracking method based on improved correlation filtering according to claim 4 is characterized in that: The minimization problem is described as: where F i is the frequency domain image of the input image, G i is the frequency domain image of the two-dimensional Gaussian distribution; The H * for: Among them, F i * is the frequency domain image of the input image.

6. The small target tracking method based on improved correlation filtering according to claim 1 is characterized in that: The peak sidelobe ratio in step 4 is: Among them, g max is the peak value of the response graph area, μ sl and σ sl are the mean and standard deviation of the side lobes, respectively.

7. The small target tracking method based on improved correlation filtering according to claim 1 is characterized in that: The Kalman filter prediction formula of the Kalman filter in step 5 is: in, is the estimated value of the current prediction, A represents the state transfer matrix during prediction, uk represents the control input vector at time k, B represents the control matrix, represents the prediction covariance matrix at time k, and Q represents the prediction noise covariance matrix.

8. The small target tracking method based on improved correlation filtering according to claim 7 is characterized in that: In step 6, the Kalman filter parameters are updated as follows: In the formula, z k represents the observation value, H is the observation matrix, R represents the measurement noise matrix, K k represents the Kalman gain at time k, P k denote the corrected estimates and covariance matrix, respectively.

9. The small target tracking method based on improved correlation filtering according to claim 7 is characterized in that: In step 6, the filter tracking template is updated to: Among them, η represents the learning rate, F i is the frequency domain image of the input image, G i is the frequency domain image of the two-dimensional Gaussian distribution.

10. A small target tracking system based on improved correlation filtering for implementing any of the methods described in claims 1-9, characterized in that: include: An image preprocessing unit, used for cropping a target area and preprocessing the acquired target image; A filter tracking template determination unit performs a two-dimensional Fourier transform on the preprocessed first frame target image and creates a filter tracking template; A response map generating unit, which multiplies the preprocessed target image by the filter tracking template to obtain a response map; A sidelobe ratio calculation unit, used for calculating a peak sidelobe ratio of a response graph; Kalman filter, used to predict the target position in the next frame; The wave tracking template and Kalman filter parameter updating unit updates the filter tracking template and Kalman filter parameters according to the tracking coordinates in the response graph.

Images