A real-time target tracking method with anti-occlusion

Abstract

A method for real-time target tracking with anti-occlusion belongs to the technical field of target tracking. The present invention solves the problem that the existing methods do not have universal adaptability, resulting in low tracking accuracy when using the existing methods in some scenarios. The present invention improves the kernel correlation filtering algorithm, and introduces a "protected scale pool" and "multi-tracker fusion" to ensure the high real-time and robustness of the target tracking algorithm. By referencing an adaptive link, the method of the present invention can be applied to target tracking and target scale estimation in any scenario, overcoming the problem of low tracking accuracy in some scenarios caused by the use of fixed parameters in the existing methods. The present invention can be applied to target tracking.

Description

Technical Field

[0001] The invention belongs to the technical field of target tracking, and in particular relates to an anti-occlusion real-time target tracking algorithm based on a protection scale pool and multi-tracker fusion. Background Art

[0002] Kernel correlation filtering algorithm by Henriques F et al. proposed in 2015 that this method is based on fast Fourier transform, maps complex time domain calculations to the frequency domain, transforms the convolution problem into a correlation problem, and uses the property that the circulant matrix can be diagonalized in the frequency domain, which greatly simplifies the operation, making it possible to quickly calculate a large number of samples, which had a very good effect at the time. However, some inherent defects of the kernel correlation algorithm have not been well solved, such as scale estimation problems, target occlusion problems, etc. In recent years, some researchers have combined the kernel correlation filter algorithm with the scale pooling algorithm and the Kalman filter algorithm, which has improved the tracking effect of the algorithm to a certain extent, but they can only achieve accurate tracking in specific scenarios and do not have universal adaptability. Therefore, how to improve the robustness and adaptability of the algorithm while ensuring the real-time performance of the kernel correlation filter algorithm has become the focus and difficulty of research in the field of target tracking based on online learning. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that the existing methods do not have universal adaptability, resulting in low tracking accuracy when using the existing methods in some scenarios, and propose an anti-occlusion real-time target tracking method.

[0004] The technical solution adopted by the present invention to solve the above technical problems is: an anti-occlusion real-time target tracking method, the method specifically comprises the following steps:

[0005] Step 1: preprocessing the input first frame image by using the Laplace operator to obtain the preprocessed first frame image; framing the target in the preprocessed first frame image to obtain the target frame image;

[0006] And extract the HOG features and CN features of the image in the target frame, obtain the tracker template based on the HOG features according to the extracted HOG features, obtain the tracker template based on the CN features according to the extracted CN features, and initialize the fusion coefficient β′ HOG and β′ CN ;

[0007] Step 2: preprocessing the input second frame image using the Laplace operator to obtain a preprocessed second frame image;

[0008] Determine the position of the candidate area in the second frame image after preprocessing, and extract HOG features and CN features of the candidate area image, perform correlation operation on the HOG features of the candidate area image and the tracker template based on the HOG features to obtain the tracker response value based on the HOG features, and perform correlation operation on the CN features of the candidate area image and the tracker template based on the CN features to obtain the tracker response value based on the CN features;

[0009] Step 3: Improved peak intensity sidelobe ratio SPSR using HOG features of the candidate region image HOG Improved peak intensity sidelobe ratio (SPSR) of CN features of candidate region images CN β′ HOG and β′ CN Fine-tune and obtain the fine-tuned fusion coefficient β HOG and β CN ;

[0010] According to the fine-tuned fusion coefficient, the response value of the tracker based on the HOG feature and the response value of the tracker based on the CN feature are fused to obtain a fused response value; the position of the maximum response value in the fused response value is used as the target position in the second frame image;

[0011] Step 4: According to SPSR HOG and SPSR CN , and calculate the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image in combination with the fine-tuned fusion coefficient;

[0012] Step 5: Use the protected scale pool algorithm to estimate the target size in the second frame image; and update the tracker template based on HOG features and the tracker template based on CN features according to the target position and size in the second frame image;

[0013] Step 6, setting a threshold T, wherein the value of the threshold T is one quarter of the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image;

[0014] Step 7, calculating the total improved peak intensity sidelobe ratio SPSR corresponding to the input third frame image, if the total improved peak intensity sidelobe ratio SPSR corresponding to the third frame image is greater than or equal to a threshold T, then using the protected scale pool algorithm to estimate the target size in the third frame image;

[0015] Otherwise, if the total improved peak intensity sidelobe ratio SPSR corresponding to the third frame image is less than the threshold value T, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fourth frame image is continued to be calculated. If the total improved peak intensity sidelobe ratio corresponding to the fourth frame image is greater than or equal to the threshold value T, the protected scale pool algorithm is used to estimate the target size in the fourth frame image; otherwise, if the total improved peak intensity sidelobe ratio SPSR in the fourth frame image is still less than the threshold value T, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fifth frame image is continued to be calculated;

[0016] If the total improved peak intensity sidelobe ratio SPSR in 5 consecutive frames of images is less than the threshold T, it is considered that the target has indeed disappeared due to complete occlusion, and the target position is re-detected in the entire image range for the fifth frame of the 5 consecutive frames of images;

[0017] If the maximum response value of the fused response value of the fifth frame of the five consecutive frames in the whole image range is greater than or equal to the preset value T1, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fifth frame of the five consecutive frames detected in the whole image range is verified. If it is greater than or equal to the threshold T, it is considered that the target is re-detected, and the target size in the fifth frame of the five consecutive frames is estimated;

[0018] Otherwise, if the maximum response value of the fused response value of the fifth frame of the five consecutive frames of images whose corresponding total improved peak intensity sidelobe ratios SPSR are all less than the threshold value T in the whole image range is less than the preset value T1, or the maximum response value in the fused response value is greater than or equal to the preset value T1, but the total improved peak intensity sidelobe ratio corresponding to the detection of the fifth frame of the image in the whole image range is less than the threshold value T, then continue to re-detect the target position in the whole image range for the sixth frame of the image until the target is re-detected, and estimate the size of the re-detected target;

[0019] Step 8: Use the detected target features to update the tracker template based on HOG features and the tracker template based on CN features;

[0020] Step 9: Repeat the process of steps 7 to 8 for subsequent input images to achieve real-time tracking of the target.

[0021] The beneficial effects of the present invention are as follows: the present invention proposes an anti-occlusion real-time target tracking method, the present invention improves the kernel correlation filtering algorithm, and introduces a "protection scale pool" and "multi-tracker fusion" to ensure the high real-time and robustness of the target tracking algorithm. By referencing an adaptive link, the method of the present invention can be applied to target tracking and target scale estimation in any scenario, overcoming the problem of low tracking accuracy in some scenarios caused by the use of fixed parameters in existing methods.

[0022] When the Intel Core i7-6500U low-voltage CPU is used as the main processor, the method of the present invention can achieve an operating speed of 80FPS / or above for an image of 640×480 size, and can also maintain an operating speed of around 40FPS for an image of 1080×720 size, thus meeting the real-time requirement well. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of the protection scale pool algorithm of the present invention;

[0024] Figure 2 is a flow chart of the method of the present invention;

[0025] Figure 3a ) is the original image;

[0026] Figure 3b ) is the Laplace sharpening effect diagram;

[0027] Figure 4 This is the SPSR measured picture;

[0028] Figure 5a ) is the scale estimation effect Figure 1 ;

[0029] Figure 5b ) is the scale estimation effect Figure 2 . DETAILED DESCRIPTION

[0030] Specific implementation method 1: Combination Figure 2 The present embodiment describes an anti-occlusion real-time target tracking method, which specifically includes the following steps:

[0031] Step 1: preprocessing the input first frame image by using the Laplace operator to obtain the preprocessed first frame image; framing the target in the preprocessed first frame image to obtain the target frame image;

[0032] And extract the HOG features and CN features of the image in the target frame, obtain the tracker template based on the HOG features according to the extracted HOG features, obtain the tracker template based on the CN features according to the extracted CN features, and initialize the fusion coefficient β′ HOG and β′ CN ;

[0033] Step 2: preprocessing the input second frame image using the Laplace operator to obtain a preprocessed second frame image;

[0034] Determine the position of the candidate area in the second frame image after preprocessing (in a set of continuous video sequences, the acquisition interval between two adjacent frames is very short, so in two adjacent frames, the relative position of the target in the entire image will not change much. Based on this idea, we take the center of the tracking frame of the previous frame as the center point, and take a rectangular range with a length and width of 2.5 times the size of the tracking frame of the previous frame as the candidate area of ​​the target in the subsequent frame), and extract HOG features and CN features from the candidate area image, and perform correlation operation on the HOG features of the candidate area image and the tracker template based on the HOG features (correlation operation is a mathematical operation for comparing the similarity of two sequences. In the kernel correlation filtering algorithm, it is used to calculate the similarity between two different image blocks. The kernel function and the correlation operation method constitute the main part of the kernel correlation filtering algorithm), and obtain the tracker response value based on the HOG feature, and perform correlation operation on the CN features of the candidate area image and the tracker template based on the CN features to obtain the tracker response value based on the CN features;

[0035] Step 3: Improved peak intensity sidelobe ratio SPSR using HOG features of the candidate region image HOG Improved peak intensity sidelobe ratio (SPSR) of CN features of candidate region images CN β′ HOG and β′ CN Fine-tune and obtain the fine-tuned fusion coefficient β HOG and β CN ;

[0036] According to the fine-tuned fusion coefficient, the response value of the tracker based on the HOG feature and the response value of the tracker based on the CN feature are fused to obtain a fused response value; the position of the maximum response value in the fused response value is used as the target position in the second frame image;

[0037] Step 4: According to SPSR HOG and SPSR CN , and calculate the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image in combination with the fine-tuned fusion coefficient;

[0038] Step 5: Use the protected scale pool algorithm to estimate the target size in the second frame image; and update the tracker template based on HOG features and the tracker template based on CN features according to the target position and size in the second frame image;

[0039] Step 6, setting a threshold T, the value of which is one quarter of the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image (the value of the threshold T remains unchanged during the entire detection process);

[0040] Step 7: Calculate the total improved peak intensity sidelobe ratio SPSR corresponding to the input third frame image (the fusion coefficient of the second frame image needs to be updated during the calculation). If the total improved peak intensity sidelobe ratio SPSR corresponding to the third frame image is greater than or equal to the threshold T, use the protected scale pool algorithm to estimate the target size in the third frame image.

[0041] The total improved peak intensity sidelobe ratio corresponding to the input third frame image is calculated based on the updated template and the fusion coefficient;

[0042] Otherwise, if the total improved peak intensity sidelobe ratio SPSR corresponding to the third frame image is less than the threshold value T, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fourth frame image is continued to be calculated. If the total improved peak intensity sidelobe ratio corresponding to the fourth frame image is greater than or equal to the threshold value T, the protected scale pool algorithm is used to estimate the target size in the fourth frame image; otherwise, if the total improved peak intensity sidelobe ratio SPSR in the fourth frame image is still less than the threshold value T, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fifth frame image is continued to be calculated;

[0043] If the total improved peak intensity sidelobe ratio SPSR in 5 consecutive frames of images is less than the threshold T, it is considered that the target has indeed disappeared due to complete occlusion, and the target position is re-detected in the entire image range for the fifth frame of the 5 consecutive frames of images;

[0044] If the maximum response value of the fused response value of the fifth frame of the five consecutive frames of images within the entire image range is greater than or equal to a preset value T1 (the corresponding critical value T1 preset in the present invention is 0.20), then the total improved peak intensity sidelobe ratio SPSR corresponding to the fifth frame of the five consecutive frames of images is verified within the entire image range. If it is greater than or equal to the threshold value T, it is considered that the target is re-detected, and the target size in the fifth frame of the five consecutive frames of images is estimated;

[0045] Otherwise, if the maximum response value of the fused response value of the fifth frame of the five consecutive frames of images whose corresponding total improved peak intensity sidelobe ratios SPSR are all less than the threshold value T in the whole image range is less than the preset value T1, or the maximum response value in the fused response value is greater than or equal to the preset value T1, but the total improved peak intensity sidelobe ratio corresponding to the detection of the fifth frame of the image in the whole image range is less than the threshold value T, then continue to re-detect the target position in the whole image range for the sixth frame of the image until the target is re-detected, and estimate the size of the re-detected target;

[0046] When re-detecting the target position within the full image range, the process of steps 2 and 3 needs to be repeated. The only difference is that the range of the candidate area needs to be set to the full image range.

[0047] Step 8: Use the detected target features (after detecting the position and size of the target, the HOG features and CN features of the target can be extracted) to update the tracker template based on the HOG feature and the tracker template based on the CN feature (update based on the template updated last time);

[0048] Step 9. Repeat the process of steps 7 to 8 for subsequent input images (in the next iteration, the next frame of image is processed using the updated template, detected target size and target position, and the fusion coefficient of the previous frame is fine-tuned according to the improved peak intensity sidelobe ratio SPSR based on HOG features and the improved peak intensity sidelobe ratio SPSR based on CN features of the new frame) to achieve real-time tracking of the target.

[0049] The present invention adaptively adjusts the tracker fusion coefficient during the tracking process, so that our tracking algorithm is well adapted to some test data sets with unclear target CN features (such as black and white or night data sets) or test data sets with unclear target HOG features (such as data sets with mixed background and foreground). It appropriately reduces the proportion of feature types that cannot effectively characterize the target during tracker fusion, and increases the proportion of feature types that can effectively characterize the target during tracker fusion, so that the tracking algorithm can show better adaptability to different tracking situations. In addition, the degree of distinction between the target and the background is different in different test data sets, which leads to a large difference in the value of the total improved peak intensity sidelobe ratio SPSR in tracking. Therefore, adaptively adopting different occlusion judgment thresholds T for different test data sets can also help us to make more effective judgments on complete occlusion situations.

[0050] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that: in step 1, the input first frame image is preprocessed using the Laplace operator, and the specific process is as follows:

[0051]

[0052]

[0053] g(x,y)=5f(x,y)-[f(x+1,y)+f(x-1,y)+f(x,y+1)+f(x,y-1)]

[0054] Among them, f(x,y) is the pixel value of the point (x,y) in the first frame image collected, f(x+1,y) is the pixel value of the point (x+1,y) in the first frame image collected, f(x-1,y) is the pixel value of the point (x-1,y) in the first frame image collected, f(x,y+1) is the pixel value of the point (x,y+1) in the first frame image collected, f(x,y-1) is the pixel value of the point (x,y-1) in the first frame image collected, and g(x,y) is the pixel value of the point (x,y) in the first frame image after preprocessing.

[0055] Preprocessing the image based on Laplace image sharpening can enhance the image contrast and better deal with image blur and other situations.

[0056] Specific implementation method three: This implementation method is different from the specific implementation method one in that: in step one, a tracker template based on HOG features is obtained according to the extracted HOG features, and a tracker template based on CN features is obtained according to the extracted CN features. The specific process is as follows:

[0057] The extracted HOG features are used to train a single feature tracker based on HOG features to obtain a tracker template based on HOG features. The extracted CN features are used to train a single feature tracker based on CN features to obtain a tracker template based on CN features.

[0058] HOG feature is also called directional gradient histogram feature. It is a feature that describes the local texture information of an image and is often used in target tracking, target detection, etc. The directional gradient histogram feature first calculates the gradient value of a part of the image using a sliding window strategy, and then classifies and counts these gradients according to their directions to form a gradient histogram. These histograms represent the direction and size of the image gradient in this area, forming what we call HOG features.

[0059] The CN feature was proposed by M. Danelljan et al. in 2014. The core idea is to map RGB color attributes to 11 predefined colors (black, blue, brown, gray, green, orange, pink, purple, red, white and yellow). The mapped color features will better represent the color information of the target than the initial RGB features, are more robust, and are medium-significant features.

[0060] The HOG feature has good adaptability to the geometric and optical changes of the target, while the CN feature has good adaptability to the geometric deformation and rotation changes of the target. To this extent, the two are complementary to each other.

[0061] It is not uncommon to fuse HOG features and CN features to improve KCF. However, most of the fusion in the literature connects the HOG feature matrix and the CN feature matrix end to end to form a comprehensive feature matrix, and then brings it into the correlation filter for solution. The feature fusion method adopted by the present invention is to train the kernel correlation tracker based on HOG features and the kernel correlation filter tracker based on CN features separately. The two do not interfere with each other but follow the same tracking principle and are updated at the same time.

[0062] The improved algorithm of the previous kernel correlation filter algorithm for multi-feature fusion is to first fuse the features to form a large feature matrix and then use this new comprehensive feature as the feature to train the tracker. This is simple to operate but will cause an imbalance in the feature ratio. Therefore, the present invention first trains multiple independent single feature trackers based on different features, then selects a suitable ratio according to the actual effect, combines the response graphs obtained by tracking each single feature tracker to form an overall response graph, and then selects the maximum response position from the overall response graph as the target position of the next frame image.

[0063] In addition to the two features of this implementation, the three features of HOG+CN+LAB can also be used. Based on these three features, the trackers are trained and fused respectively, and the weighting coefficients of each tracker are appropriately adjusted. If the computer performance allows, this implementation can improve the tracking effect, but the algorithm operation rate will be slightly reduced. Assuming that the kernel correlation filter response values ​​obtained from a candidate block are f HOG (z), f CN (z) and f LAB (z), the total response is:

[0064] f(z)=β HOG f HOG (z)+β CN f CN (z)+β LAB f LAB (z)

[0065] Specific implementation method 4: This implementation method is different from specific implementation method 3 in that: in step 3, according to the fine-tuned fusion coefficient, the tracker response value based on the HOG feature and the tracker response value based on the CN feature are fused to obtain the fused response value; the specific process is:

[0066] f(z)=β HOG f1(z)+β CN f2(z)

[0067] Among them, f(z) is the response value after fusion, f1(z) is the response value of the tracker based on HOG features, f2(z) is the response value of the tracker based on CN features, β HOG is the fine-tuned fusion coefficient corresponding to f1(z), β CN is the fine-tuned fusion coefficient corresponding to f2(z), satisfying β HOG +β CN =1.

[0068] Specific implementation method 5: This implementation method is different from the specific implementation method 4 in that: the fine-tuned fusion coefficient β HOG and β CN The calculation method is:

[0069]

[0070] Where: β′ HOG is the initialization fusion coefficient corresponding to f1(z), β′ CN is the initialization fusion coefficient corresponding to f2(z), SPSR HOG The improved peak intensity sidelobe ratio, SPSR, of the HOG feature of the candidate region image CN is the improved peak intensity sidelobe ratio of the CN feature of the candidate region image.

[0071] Taking the fusion of HOG features and CN features as an example, after a lot of experiments, we believe that β HOG When the value is 0.7-0.8, tracking can achieve better results, so we set β′ in initialization. HOG is defined as 0.75, correspondingly β′ CN It is defined as 0.25, and then the improved peak intensity sidelobe ratio SPSR of the two trackers is solved in real time during the tracking process. When the SPSR of one tracker is very large and the SPSR of the other tracker is very small, it is considered that the target feature is very obvious in one tracker but not in the other tracker. At this time, the fusion weight of the tracker corresponding to the larger SPSR can be appropriately increased, but the updated fusion weight finally needs to fall within the preset range.

[0072] Specific implementation method 6: This implementation method is different from the specific implementation method 5 in that: the improved peak intensity sidelobe ratio SPSR of the HOG feature of the candidate region image HOG Improved peak intensity sidelobe ratio (SPSR) of CN features of candidate region images CN The calculation method is:

[0073] Improved peak intensity sidelobe ratio (SPSR) of HOG feature of candidate region image HOG :

[0074]

[0075] Among them, g max Represents the maximum response value among the tracker response values ​​based on HOG features, with the maximum response value g max Centered on the left and right, the tracker template width based on the HOG feature is expanded The length of the tracker template based on the HOG feature is extended upward and downward The maximum response value g max The circumscribed rectangle is the center; μ represents the mean of all response values ​​contained in the circumscribed rectangle, and σ represents the variance of all response values ​​contained in the circumscribed rectangle;

[0076] Similarly, the improved peak intensity sidelobe ratio SPSR of the CN feature of the candidate region image is calculated CN The value of .

[0077] Specific implementation method 7: This implementation method is different from specific implementation method 6 in that: in step 4, according to SPSR HOG and SPSR CN , and the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image is calculated in combination with the fine-tuned fusion coefficient; the specific process is:

[0078] SPSR=β HOG SPSR HOG +β CN SPSR CN

[0079] The present invention adopts a seven-scale pool structure, and the scale coefficients in the scale pool are 1.15, 1.10, 1.05, 1, 1 / 1.05, 1 / 1.10, and 1 / 1.15, respectively. According to the principle of the scale pool method, the scale coefficient will be multiplied by the original tracking frame size at each detection to separate 7 tracking frames of different sizes to track the target. The tracker will calculate the target response at the current size based on these 7 scales (the same as the calculation method in step 2), and then calculate the response value of each scale, and consider the tracking frame size corresponding to the maximum response value as the new target size. In actual experiments, although this method is simple and effective, in complex tracking environments, there will be a problem of frequent jumping of the tracking frame size, that is, the problem of unstable tracking frame scale. The present invention improves this problem and proposes the concept of "protective scale pool". The operating block diagram of the protective scale pool algorithm is shown as follows. Figure 1 shown.

[0080] Specific implementation method eight: Combination Figure 5a )and Figure 5b) This embodiment is different from the specific embodiment seven in that: in step five, the protection scale pool algorithm is used to estimate the target size in the second frame image, and the specific process is as follows:

[0081] When estimating the target size, a seven-scale pool structure is used to obtain tracking frames of seven different scales; the sizes of the seven tracking frames of different scales are 1.15, 1.10, 1.05, 1, 1 / 1.05, 1 / 1.10, and 1 / 1.15 times the size of the target frame in step 1, respectively;

[0082] In the next iteration, the target size obtained in step 6 will be used as the original target frame size, and the sizes of the tracking frames of the seven different scales will be 1.15, 1.10, 1.05, 1, 1 / 1.05, 1 / 1.10, and 1 / 1.15 times the target size obtained in step 6 respectively;

[0083] After calculating the maximum response value of each size tracking frame respectively, multiply the maximum response value of the tracking frame whose size is 1.15 times the target frame by the protection factor 1 / 1.10, multiply the maximum response value of the tracking frame whose size is 1.10 times the target frame by the protection factor 1 / 1.05, multiply the maximum response value of the tracking frame whose size is 1.05 times the target frame by the protection factor 1, multiply the maximum response value of the tracking frame whose size is 1 times the target frame by the protection factor 1, multiply the maximum response value of the tracking frame whose size is 1 / 1.05 times the target frame by the protection factor 1, multiply the maximum response value of the tracking frame whose size is 1 / 1.10 times the target frame by the protection factor 1 / 1.05, and multiply the maximum response value of the tracking frame whose size is 1 / 1.15 times the target frame by the protection factor 1 / 1.05;

[0084] After multiplying by the protection coefficient, the maximum response value of each size tracking frame in the protection scale pool is obtained, and the size of the tracking frame corresponding to the maximum response value in the protection scale pool is used as the target size.

[0085] This implementation adds a protection factor when comparing the target responses of each scale. The protection factor is less than 1, and the greater the difference between the tracking frame size in the scale pool and 1, the smaller the protection factor. Then the protection factor is multiplied by the target response value of each scale before comparison. The core idea of ​​this method is that when the target response value of the new scale is much larger than the current scale, it is considered that there is indeed a scale change, and the scale will be updated. When the target response value of the new scale is close to the current scale, it is considered to be clutter caused by background interference or motion blur, and the current scale is maintained. At the same time, when the difference between the new scale and the current scale is greater, the scale update is more cautious. Such a scale protection mechanism maintains the stability of the tracking frame to the greatest extent and effectively avoids the introduction of scale estimation errors.

[0086] Specific implementation method 9: This implementation method is different from specific implementation method 8 in that: in step 5, the tracker template based on HOG features and the tracker template based on CN features are updated according to the target position and size in the second frame image, and the specific process is as follows:

[0087] For the tracker template based on HOG features:

[0088] α=j*z+(1-j)*α0

[0089] Where: α represents the updated tracker template based on HOG features, j is a constant, z represents the HOG features of the target in the second frame image, and α0 represents the tracker template based on HOG features obtained in step 1;

[0090] As the target tracking process continues, α0 represents the tracker template based on HOG features after the last update, and z represents the HOG features of the target extracted in the current frame;

[0091] Similarly, the tracker template based on the CN feature is updated to obtain an updated tracker template based on the CN feature.

[0092] Specific implementation method ten: This implementation method is different from specific implementation method nine in that: the value of the fixed value j is:

[0093]

[0094] If the total improved peak intensity sidelobe ratio SPSR corresponding to the current frame image is greater than or equal to the threshold T, the target size of the current frame image is estimated, and j is calculated using the total improved peak intensity sidelobe ratio corresponding to the current frame image. When the target is completely occluded, j is set to 0, that is, the template update is stopped.

[0095] Template updating is a very important part of the KCF algorithm. If the template is not updated in real time with the tracking situation, the tracking system will not be able to adapt to changes in the target's geometric shape and other aspects, which will make the algorithm less robust during long-term tracking and ultimately lead to tracking failure. If the target template is completely replaced by the current frame's target every time it is updated, the tracking performance of the algorithm will also be reduced.

[0096] In the present invention, the value of j is adjusted according to the real-time tracking situation. When the tracking situation is ideal and the target is clear, the confidence of the template change is high, and it can be considered that the target feature has indeed changed slightly. In order to facilitate subsequent tracking, the update coefficient should be appropriately increased. On the contrary, when blurring, occlusion and other situations occur, the target feature becomes less obvious. At this time, the extracted target template is likely to contain some unnecessary information such as background clutter and occlusion. If the template is still updated in the original way, it is easy to cause the template drift phenomenon. Therefore, the update coefficient should be appropriately reduced.

[0097] Example

[0098] Step 1: Process the first frame of image;

[0099] Step 1.1: Preprocess the image using the Laplacian operator;

[0100] The Laplace operator calculation formula is used to differentiate the image in the x and y directions. The Laplace operator is very sensitive to the sudden change area in the image, but tends to ignore the area with slow changes in the image. Therefore, we only need to superimpose the original image and the output image after Laplace operation to restore the area with slow changes in the image while maintaining the Laplace sharpening result. The original image and the Laplace sharpening effect are shown as follows: Figure 3a) and 3b) shown.

[0101] Step 1.2: The operator frames the target through the human-computer interaction interface;

[0102] Step 1.3: Initialize the tracker fusion weights, extract HOG and CN features from the framed image, and train the tracker templates separately;

[0103] Step 2: Subsequent image processing;

[0104] Step 2.1: Preprocess the image using the Laplacian operator;

[0105] Step 2.2: Determine the location of the candidate area, extract the HOG and CN features of the candidate area, and perform correlation operations with the templates of their respective features to obtain the target response value;

[0106] After extracting HOG features and CN features, the principal component analysis method is used to reduce the dimension of the CN features, and the 11-dimensional CN features are converted into 2 dimensions. At the same time, the weighting coefficients during tracker fusion are adjusted. This can greatly improve the algorithm operation rate, and the performance of the CN tracker can still reach about 80% of implementation plan one. When the computer performance is insufficient, this implementation method can enhance the real-time performance of the algorithm while ensuring good tracking effects.

[0107] Step 2.5: According to the weight coefficient, the response values ​​of the HOG tracker and the CN tracker are fused to obtain the final response value;

[0108] Step 2.6: Obtain the target position of the current frame from the response value, and calculate the SPSR of the two trackers and the total SPSR respectively;

[0109] Step 2.7: Fine-tune the weighting coefficient β according to the value of SPSR;

[0110] Step 2.8: Perform occlusion discrimination and target re-detection based on the total SPSR;

[0111] Based on the peak intensity sidelobe ratio, SPSR is proposed to judge occlusion. When it is judged to be completely occluded, the model update is stopped immediately, and the target is re-detected within a certain image range. When the response value of a part of the image exceeds the preset value and the corresponding total improved peak intensity sidelobe ratio is greater than the threshold T, it is considered that the target has reappeared. When the target is detected to reappear, it is tracked again.

[0112] Step 2.9: Add the protection scale pool algorithm for scale estimation;

[0113] During the tracking process, the scale of the target may change as the target approaches or moves away. In the initial and related filtering algorithms, the size of the tracking box is fixed. Therefore, it is necessary to add a scale estimation strategy based on the protection scale pool proposed in this design to estimate the target scale in real time, adjust the size of the tracking box according to the actual tracking situation, and improve the adaptability of the tracking algorithm.

[0114] Step 2.10: Update the template according to SPSR and output the target position;

[0115] Step 3: Testing and evaluation;

[0116] Step 3.1: Algorithm real-time test;

[0117] For target tracking algorithms, the real-time performance of the algorithm is of paramount importance. Otherwise, if the algorithm runs too slowly, the target displacement between frames will be too large in actual operation, which can easily cause target loss and affect the robustness of the algorithm. Therefore, it is necessary to evaluate the real-time performance of the algorithm first. It is generally believed that the algorithm running speed above 25FPS / S reaches real-time status.

[0118] Step 3.2: Algorithm robustness test;

[0119] This step requires measuring the tracking accuracy and tracking success rate of the tracking algorithm based on the target standard position provided in the data set. The better the tracking accuracy and tracking success rate, the better the robustness of the algorithm.

[0120] Figure 4 It is the numerical change of the improved peak intensity sidelobe ratio (denoted as SPSR) measured when testing a certain OTB50 dataset. In the video sequence, the target is occluded around the 100th frame, and it can be seen that the value of SPSR around the 100th frame is significantly reduced.

[0121] At present, many improved algorithms only focus on improving tracking accuracy and enhancing algorithm robustness. Therefore, a large number of operations are added in the improvement, resulting in the algorithm no longer being able to meet real-time requirements and losing the inherent advantages of the kernel correlation filtering algorithm. The present invention has always attached great importance to the problem of calculation speed in the design and added a frame-by-frame processing mechanism at necessary positions, so it has good real-time performance. The present invention can achieve a running speed of more than 80FPS / S for images of 640×480 size, and can also maintain a running speed of about 40FPS / S for images of 1080×720 size, which well meets the real-time requirements.

[0122] According to the actual application scenarios of the present invention, we selected some representative typical data sets involving multiple interferences from the OTB data set and the VOT data set for testing and compared them with the original algorithm. As shown in Table 1 and Table 2, they are the test results of the OTB data set and the test results of the VOT data set respectively:

[0123] Table 1 Test results of 11 typical datasets selected from the OTB dataset (tracking success rate / tracking accuracy)

[0124]

[0125]

[0126] Table 2 Test results of 5 typical data sets selected from the VOT-boat data set (tracking success rate / tracking accuracy)

[0127] Algorithm\Dataset boat-1 boat-3 boat-8 boat-13 boat-14 average KCF 0.649 / 0.561 0.216 / 0.606 0.458 / 0.124 0.571 / 0.618 0.140 / 0.285 0.407 / 0.539 Improve KCF 0.726 / 0.741 0.773 / 0.929 0.930 / 0.680 0.733 / 0.818 0.797 / 0.889 0.791 / 0.811

[0128] Judging from the experimental results on the OTB dataset and the VOT dataset, the algorithm of the present invention has very obvious improvements in tracking robustness and accuracy. Compared with the current mainstream non-deep learning improved algorithms (for the OTB dataset, the improved tracking accuracy and tracking success rate are generally around 0.85), the effect is also very outstanding.

[0129] The above calculation examples of the present invention are only used to explain the calculation model and calculation process of the present invention in detail, and are not intended to limit the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A real-time target tracking method with anti-occlusion, characterized in that: The method comprises the following steps: Step 1: preprocessing the input first frame image by using the Laplace operator to obtain the preprocessed first frame image; framing the target in the preprocessed first frame image to obtain the target frame image; And extract the HOG features and CN features of the image in the target frame, obtain the tracker template based on the HOG features according to the extracted HOG features, obtain the tracker template based on the CN features according to the extracted CN features, and initialize the fusion coefficient β′ HOG and β′ CN ; Step 2: preprocessing the input second frame image using the Laplace operator to obtain a preprocessed second frame image; Determine the position of the candidate area in the second frame image after preprocessing, and extract HOG features and CN features of the candidate area image, perform correlation operation on the HOG features of the candidate area image and the tracker template based on the HOG features to obtain the tracker response value based on the HOG features, and perform correlation operation on the CN features of the candidate area image and the tracker template based on the CN features to obtain the tracker response value based on the CN features; Step 3: Improved peak intensity sidelobe ratio SPSR using HOG features of the candidate region image HOG Improved peak intensity sidelobe ratio (SPSR) of CN features of candidate region images CN β′ HOG and β′ CN Fine-tune and obtain the fine-tuned fusion coefficient β HOG and β CN ; The fine-tuned fusion coefficient β HOG and β CN The calculation method is: Where: β′ HOG is the initialization fusion coefficient corresponding to f1(z), β′ CN is the initialization fusion coefficient corresponding to f2(z), SPSR HOG The improved peak intensity sidelobe ratio, SPSR, of the HOG feature of the candidate region image CN is the improved peak intensity sidelobe ratio of the CN feature of the candidate region image; According to the fine-tuned fusion coefficient, the response value of the tracker based on the HOG feature and the response value of the tracker based on the CN feature are fused to obtain a fused response value; the position of the maximum response value in the fused response value is used as the target position in the second frame image; According to the fine-tuned fusion coefficient, the tracker response value based on the HOG feature and the tracker response value based on the CN feature are fused to obtain the fused response value; the specific process is: f(z)=β HOG f1(z)+β CN f2(z) Among them, f(z) is the response value after fusion, f1(z) is the response value of the tracker based on HOG features, f2(z) is the response value of the tracker based on CN features, β HOG is the fine-tuned fusion coefficient corresponding to f1(z), β CN is the fine-tuned fusion coefficient corresponding to f2(z), satisfying β HOG +β CN =1; Step 4: According to SPSR HOG and SPSR CN , and calculate the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image in combination with the fine-tuned fusion coefficient; Step 5: Use the protected scale pool algorithm to estimate the target size in the second frame image; and update the tracker template based on HOG features and the tracker template based on CN features according to the target position and size in the second frame image; Step 6, setting a threshold T, wherein the value of the threshold T is one quarter of the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image; Step 7, calculating the total improved peak intensity sidelobe ratio SPSR corresponding to the input third frame image, if the total improved peak intensity sidelobe ratio SPSR corresponding to the third frame image is greater than or equal to a threshold T, using the protected scale pool algorithm to estimate the target size in the third frame image; Otherwise, if the total improved peak intensity sidelobe ratio SPSR corresponding to the third frame image is less than the threshold value T, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fourth frame image is continued to be calculated. If the total improved peak intensity sidelobe ratio corresponding to the fourth frame image is greater than or equal to the threshold value T, the protected scale pool algorithm is used to estimate the target size in the fourth frame image; otherwise, if the total improved peak intensity sidelobe ratio SPSR in the fourth frame image is still less than the threshold value T, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fifth frame image is continued to be calculated; If the total improved peak intensity sidelobe ratio SPSR in 5 consecutive image frames is less than the threshold T, it is considered that the target has indeed disappeared due to complete occlusion, and the target position is re-detected in the entire image range for the fifth frame of the 5 consecutive image frames; If the maximum response value of the fused response value of the fifth frame of the five consecutive frames in the whole image range is greater than or equal to the preset value T1, then the total improved peak intensity sidelobe ratio SPSR corresponding to the fifth frame of the five consecutive frames detected in the whole image range is verified. If it is greater than or equal to the threshold T, it is considered that the target is re-detected, and the target size in the fifth frame of the five consecutive frames is estimated; Otherwise, if the maximum response value of the fused response value of the fifth frame of the five consecutive frames of images whose corresponding total improved peak intensity sidelobe ratios SPSR are all less than the threshold value T in the whole image range is less than the preset value T1, or the maximum response value in the fused response value is greater than or equal to the preset value T1, but the total improved peak intensity sidelobe ratio corresponding to the detection of the fifth frame of the image in the whole image range is less than the threshold value T, then continue to re-detect the target position in the whole image range for the sixth frame of the image until the target is re-detected, and estimate the size of the re-detected target; Step 8: Use the detected target features to update the tracker template based on HOG features and the tracker template based on CN features; Step 9: Repeat the process of steps 7 to 8 for subsequent input images to achieve real-time tracking of the target.

2. The anti-occlusion real-time target tracking method according to claim 1, characterized in that: In the step 1, the first frame of the input image is preprocessed using the Laplace operator, and the specific process is as follows: g(x,y)=5f(x,y)-[f(x+1,y)+f(x-1,y)+f(x,y+1)+f(x,y-1)] Among them, f(x,y) is the pixel value of the point (x,y) in the first frame image collected, f(x+1,y) is the pixel value of the point (x+1,y) in the first frame image collected, f(x-1,y) is the pixel value of the point (x-1,y) in the first frame image collected, f(x,y+1) is the pixel value of the point (x,y+1) in the first frame image collected, f(x,y-1) is the pixel value of the point (x,y-1) in the first frame image collected, and g(x,y) is the pixel value of the point (x,y) in the first frame image after preprocessing.

3. The anti-occlusion real-time target tracking method according to claim 2, characterized in that: In the step 1, a tracker template based on HOG features is obtained according to the extracted HOG features, and a tracker template based on CN features is obtained according to the extracted CN features. The specific process is as follows: The extracted HOG features are used to train a single feature tracker based on HOG features to obtain a tracker template based on HOG features. The extracted CN features are used to train a single feature tracker based on CN features to obtain a tracker template based on CN features.

4. The anti-occlusion real-time target tracking method according to claim 3, characterized in that: Improved peak intensity sidelobe ratio SPSR of the HOG feature of the candidate region image HOG Improved peak intensity sidelobe ratio (SPSR) of CN features of candidate region images CN The calculation method is: Improved peak intensity sidelobe ratio (SPSR) of HOG feature of candidate region image HOG : Among them, g max Represents the maximum response value among the tracker response values ​​based on HOG features, with the maximum response value g max Centered on the left and right, the tracker template width based on the HOG feature is expanded The length of the tracker template based on the HOG feature is extended upward and downward The maximum response value g max The circumscribed rectangle is the center; μ represents the mean of all response values ​​contained in the circumscribed rectangle, and σ represents the variance of all response values ​​contained in the circumscribed rectangle; Similarly, the improved peak intensity sidelobe ratio SPSR of the CN feature of the candidate region image is calculated CN The value of .

5. The anti-occlusion real-time target tracking method according to claim 4, characterized in that: In step 4, according to SPSR HOG and SPSR CN , and the total improved peak intensity sidelobe ratio SPSR corresponding to the second frame image is calculated in combination with the fine-tuned fusion coefficient; the specific process is: SPSR=β HOG SPSR HOG +β CN SPSR CN 。 6. The anti-occlusion real-time target tracking method according to claim 5, characterized in that: In step 5, the protected scale pool algorithm is used to estimate the target size in the second frame image, and the specific process is as follows: When estimating the target size, a seven-scale pool structure is used to obtain tracking frames of seven different scales; the sizes of the seven tracking frames of different scales are 1.15, 1.10, 1.05, 1, 1 / 1.05, 1 / 1.10, and 1 / 1.15 times the size of the target frame in step 1, respectively; After calculating the maximum response value of each size tracking frame respectively, multiply the maximum response value of the tracking frame whose size is 1.15 times the target frame by the protection factor 1 / 1.10, multiply the maximum response value of the tracking frame whose size is 1.10 times the target frame by the protection factor 1 / 1.05, multiply the maximum response value of the tracking frame whose size is 1.05 times the target frame by the protection factor 1, multiply the maximum response value of the tracking frame whose size is 1 times the target frame by the protection factor 1, multiply the maximum response value of the tracking frame whose size is 1 / 1.05 times the target frame by the protection factor 1, multiply the maximum response value of the tracking frame whose size is 1 / 1.10 times the target frame by the protection factor 1 / 1.05, and multiply the maximum response value of the tracking frame whose size is 1 / 1.15 times the target frame by the protection factor 1 / 1.05; After multiplying by the protection coefficient, the maximum response value of each size tracking frame in the protection scale pool is obtained, and the size of the tracking frame corresponding to the maximum response value in the protection scale pool is used as the target size.

7. The anti-occlusion real-time target tracking method according to claim 6, characterized in that: In the step 5, the tracker template based on the HOG feature and the tracker template based on the CN feature are updated according to the target position and size in the second frame image, and the specific process is as follows: For the tracker template based on HOG features: α=j*z+(1-j)*α0 Where: α represents the updated tracker template based on HOG features, j is a constant, z represents the HOG features of the target in the second frame image, and α0 represents the tracker template based on HOG features obtained in step 1; Similarly, the tracker template based on the CN feature is updated to obtain an updated tracker template based on the CN feature.

8. The anti-occlusion real-time target tracking method according to claim 7, characterized in that: The value of the constant j is:

Images