A multi-feature fusion target tracking and positioning method based on response peak

Through multi-feature fusion and response graph weighted fusion technology, the problem of insufficient tracking accuracy of a single feature in complex environments is solved, and the target positioning accuracy and robustness of complex environments is improved.

CN114972437BActive Publication Date: 2024-11-26XIDIAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210687130.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-16
Publication Date
2024-11-26
Estimated Expiration
2042-06-16

AI Technical Summary

Technical Problem

Existing target tracking algorithms are prone to offset and loss of targets in complex environments, and a single depth feature is difficult to effectively distinguish interferers, resulting in insufficient tracking accuracy and robustness.

Method used

The multi-feature fusion method is adopted to generate a response map by extracting multiple features, perform point-multiple fusion and weighted fusion, find the maximum position in the response map to determine the target position, reduce noise interference, and improve tracking accuracy and robustness.

Benefits of technology

Effectively reduce noise interference, improve tracking accuracy and robustness, so that targets can be accurately tracked in complex environments, and enhance the stability of the tracker.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114972437B_ABST
    Figure CN114972437B_ABST
Patent Text Reader

Abstract

The present invention proposes a multi-feature fusion target tracking and positioning method based on response peak value, which mainly solves the problem that the existing tracker based on single feature is prone to offset and loss of target when facing complex tracking environment. It includes: 1) determining the search area of ​​the input image and extracting different types of features; 2) generating corresponding response maps for different types of features in the search area through the correlation filter tracking algorithm; 3) performing point multiplication fusion between the response maps corresponding to different types of features to eliminate the influence of the noise part; 4) performing weighted fusion on the response maps after pairwise fusion to obtain the final response map; 5) determining the target position by finding the maximum value position in the final response map; 6) using the target position to update the filter parameters, continue the subsequent tracking, and complete the target tracking and positioning task. The present invention can effectively reduce the interference of noise in the tracking environment, thereby improving the tracking accuracy and robustness of the tracking algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of target tracking, and further relates to target positioning and feature fusion, and specifically is a multi-feature fusion target tracking and positioning method based on response peak value, which can be used for target tracking and positioning of unmanned aerial vehicles. Background Art

[0002] Target tracking technology plays an important role in the field of computer vision; it has a wide range of applications in video surveillance, human-computer interaction, unmanned driving and other fields. When faced with problems such as occlusion, illumination changes, target rotation, and scale changes in real scenes, using different image features to describe the tracked target can achieve different tracking effects. Faced with complex tracking environments and target interference, a single feature description of the target is completely unable to achieve the ideal tracking accuracy and robustness.

[0003] In the patent document with the publication number CN108805909B and the name “Target tracking method based on particle filter re-detection under correlation filter framework”, a technical solution is provided that mainly determines whether to perform re-detection operation by judging whether the response graph is credible through decision function. However, this method uses a single feature to represent the target and cannot accurately track and locate the position of the target in a complex tracking environment. In the patent document with the publication number CN106846364B and the name “A target tracking method and device based on convolutional neural network”, a target tracking solution is proposed to extract features of the target through convolutional neural network. Although it overcomes the shortcomings of artificial features and improves the robustness of the target tracking algorithm, this extraction method that only uses deep features is easily interfered by noise in the tracking environment, affecting the tracking accuracy of the tracker.

[0004] Although some target tracking algorithms try to use the powerful representation ability of convolutional neural networks to improve the performance of target tracking, most target tracking algorithms based on deep convolutional neural networks only use a single deep global semantic feature to represent the target. However, a single global semantic feature is difficult to effectively distinguish interference objects similar to the target, causing the tracker to drift frequently during the tracking process. Although by utilizing the complementarity between convolutional features at different layers, constructing weak tracking on different convolutional layer features and returning a final tracking result can improve the tracking accuracy to a certain extent. However, a single deep feature still cannot track the target robustly. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and to propose a multi-feature fusion target tracking and positioning method based on response peak value, so as to solve the problem that the existing tracker based on single feature is prone to deviation and loss of target in the face of complex tracking environment.

[0006] The basic idea of ​​implementing the present invention is: first, robustly represent the target by extracting a variety of different features; then generate corresponding response maps for different types of features through the correlation filter tracking algorithm; then perform point multiplication fusion between the response maps corresponding to different types of features to eliminate the influence of the noise part; finally, perform weighted fusion on the response maps after pairwise fusion to obtain the final response map, and determine the position of the target by finding the position of the maximum value in the response map. The present invention can effectively reduce the interference of noise in the tracking environment, thereby improving the tracking accuracy and robustness of the tracking algorithm.

[0007] The present invention achieves the above-mentioned purpose by the following specific steps:

[0008] (1) Input an image, determine the target search area, and extract different features from the target search area;

[0009] (2) Calculate the response map corresponding to different features in the search area, which is implemented as follows:

[0010] (2.1) According to the target position information in the first frame of the input image video sequence, the correlation filter trackers corresponding to different features are trained respectively:

[0011]

[0012] Among them, x represents the training sample, y represents the label corresponding to the sample, λ represents the regularization parameter weight, * represents the convolution operation, f(w C,H,D ) represents the loss function, w C,H,D Represents the correlation filters corresponding to different features;

[0013] (2.2) In the target search area z of any frame image, the correlation filters corresponding to different types of features after training are used to compare the corresponding types of features z in the area z. C,H,D Perform convolution operation to obtain the target response map

[0014]

[0015] in, represents the Fourier transform, represents the inverse Fourier transform, ⊙ represents the convolution operation;

[0016] (3) By performing point multiplication fusion on the response maps corresponding to different features, the response maps after pairwise fusion are obtained. Then, the importance of each feature in the final fusion feature is obtained according to the peak value of the response maps after pairwise fusion. The response maps after pairwise fusion are weighted fused to obtain the final response map. The implementation steps are as follows:

[0017] (3.1) The target response maps corresponding to any two features are fused to obtain the response map R after pairwise fusion. ij :

[0018] R ij =R i ⊙R j ,

[0019] Among them, i and j are two features extracted in step 1, R i , R j Respectively represent the target response graphs corresponding to the first feature i and the second feature j;

[0020] (3.2) Calculate R according to the following formula ij The weight k in the final response graph ij :

[0021]

[0022] (3.3) Construct a multi-feature fusion model and obtain the final feature map R:

[0023] R=∑k ij R ij ;

[0024] (4) Find the position of the maximum value maxR in the final feature map R and obtain the target tracking result;

[0025] (5) Update the correlation filter tracker parameters according to the target tracking results to obtain the updated correlation filter tracker Continue follow-up tracking until the tracking task is completed.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The method of the present invention uses a correlation filter tracker to give response maps corresponding to different features, performs point multiplication on different response maps to eliminate noise interference, and performs weighted fusion on the response maps with noise interference eliminated, thereby improving the tracking accuracy and robustness of the tracker.

[0028] First, because the present invention adopts a multi-feature fusion method, the tracker can well represent the target when facing different tracking environments, thereby achieving the purpose of accurately tracking and locating the target;

[0029] Second, since the present invention uses the response graphs corresponding to different features to perform point multiplication, the uninterrupted parts of the response graph are strengthened and the interfered parts of the response graph are weakened, thereby reducing the influence of noise interference on target tracking and positioning;

[0030] Third, since the present invention adopts a weighted fusion strategy to fuse the response graphs that eliminate noise interference, the robustness of the tracking algorithm is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart for implementing the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and specific examples.

[0033] Reference Figure 1 The present invention proposes a multi-feature fusion target tracking and positioning method based on response peak, which mainly includes: determining the search area of ​​the input image and extracting different types of features; calculating the response map corresponding to different features, and then fusing the features; tracking and positioning the target according to the fused response map; updating the filter parameters using the target position, and repeating the above operations until the tracking task is completed. The specific implementation steps include the following:

[0034] Step 1, input an image, determine the target search area, and extract different features from the target search area; the features include: color features, HOG features, depth features, grayscale features, etc. This embodiment is described by extracting three different features: color feature C, HOG feature H, and depth feature D.

[0035] Step 2, calculate the response map corresponding to different features in the search area, which is implemented as follows:

[0036] (2.1) According to the target position information in the first frame of the input image video sequence, the correlation filter trackers corresponding to different features are trained respectively:

[0037]

[0038] Among them, x represents the training sample, y represents the label corresponding to the sample, λ represents the regularization parameter weight, * represents the convolution operation, f(w C,H,D ) represents the loss function, w C,H,D Represents the correlation filter corresponding to different features; let w C 、w H 、w D They represent the relevant filters corresponding to color features, HOG features, and depth features, and are trained separately using the following formulas:

[0039]

[0040]

[0041]

[0042] (2.2) In the target search area z of any frame image, the correlation filters corresponding to different types of features after training are used to compare the corresponding types of features z in the area z. C,H,D , perform convolution operation and get the target response map

[0043]

[0044] in, represents the Fourier transform, represents the inverse Fourier transform, ⊙ represents the convolution operation;

[0045] The features included in this embodiment include color feature C, HOG feature H and depth feature D, respectively obtaining target response maps

[0046] Step 3: By performing point multiplication fusion on the response maps corresponding to different features, the response maps after pairwise fusion are obtained. Then, the importance of each feature in the final fusion feature is obtained according to the peak value of the response maps after pairwise fusion. The response maps after pairwise fusion are weighted fused to obtain the final response map. The implementation steps are as follows:

[0047] (3.1) The target response maps corresponding to any two features are fused to obtain the response map R after pairwise fusion. ij :

[0048] R ij =R i ⊙R j ,

[0049] Among them, i and j are two features extracted in step 1, R i , R j Respectively represent the target response maps corresponding to the first feature i and the second feature j.

[0050] In this embodiment, the response graphs corresponding to any two features i, j∈{C, H, D} are fused to eliminate the influence of the noise part.

[0051] (3.2) Calculate R according to the following formula ij The weight k in the final response graph ij :

[0052]

[0053] (3.3) Construct a multi-feature fusion model and obtain the final feature map R:

[0054] R=∑k ij R ij ;

[0055] Step 4, find the position of the maximum value maxR in the final feature map R to obtain the target tracking result;

[0056] Step 5: Update the parameters of the correlation filter tracker according to the target tracking result to obtain the updated correlation filter tracker Continue follow-up tracking until the tracking task is completed.

[0057] Here, the parameters of the correlation filter tracker are updated to enable the model to better adapt to the changes of the target during the motion process; the update is to retrain the correlation filter trackers corresponding to different features using the target tracking results after obtaining the target tracking results, and then use the trained correlation filter tracker w C,H,D Correlation filter tracker used in the previous frame Linear combination, get the correlation filter required for the current t-th frame, that is, the updated correlation filter tracker

[0058] Update the model as follows:

[0059]

[0060] in, represents the correlation filter updated after the target position information of the t-th frame image is determined, represents the correlation filter updated after the target position information of the t-1th frame image is determined, and η represents the smoothing coefficient.

[0061] The method of the present invention calculates the response graphs corresponding to different features, and adopts a tracker based on correlation filtering to respectively give the response graphs corresponding to different features in the search box; uses multi-feature fusion technology to determine the importance of each feature in the fused feature through the peak values ​​of the response graphs corresponding to different features, and fuses the response graphs corresponding to different features to give a final response graph; finally, the final position of the target is determined through the fused response graph, and the filter is updated to repeat the above process, and the target tracking and positioning task is finally completed. Using a correlation filtering tracking framework to give the response graphs corresponding to different features, and adopting a multi-feature fusion technology based on the response peak value to fuse the response graphs corresponding to different features to obtain the final response graph, this can effectively avoid the situation where a single feature is easily interfered, thereby improving the robustness and tracking accuracy of the tracker.

[0062] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A multi-feature fusion target tracking and positioning method based on response peak, characterized in that: The steps include: (1) Input an image, determine the target search area, and extract different features from the target search area, including color features, HOG features, and depth features; (2) Calculate the response map corresponding to different features in the search area, which is implemented as follows: (2.1) According to the target position information in the first frame of the input image video sequence, the correlation filter trackers corresponding to different features are trained respectively: Among them, x represents the training sample, y represents the label corresponding to the sample, λ represents the regularization parameter weight, * represents the convolution operation, f(w C,H,D ) represents the loss function, w C,H,D Represents the correlation filter corresponding to different features; let w C 、w H 、w D They represent the relevant filters corresponding to color features, HOG features, and depth features, and are trained separately using the following formulas: (2.2) In the target search area z of any frame image, the correlation filters corresponding to different types of features after training are used to compare the corresponding types of features z in the area z. C,H,D Perform convolution operation to obtain the target response map in, represents the Fourier transform, represents the inverse Fourier transform, ⊙ represents the convolution operation; the different types of features include color feature C, HOG feature H and depth feature D, and the target response map is obtained by the above formula. (3) By performing point multiplication fusion on the response maps corresponding to different features, the response maps after pairwise fusion are obtained. Then, the importance of each feature in the final fusion feature is obtained according to the peak value of the response maps after pairwise fusion. The response maps after pairwise fusion are weighted fused to obtain the final response map. The implementation steps are as follows: (3.1) The target response maps corresponding to any two features are fused to obtain the response map R after pairwise fusion. ij : R ij =R i ☉R j , Among them, i and j are two features extracted in step (1), R i , R j Respectively represent the target response graphs corresponding to the first feature i and the second feature j; (3.2) Calculate R according to the following formula ij The weight k in the final response graph ij : (3.3) Construct a multi-feature fusion model and obtain the final feature map R: R=∑k ij R ij ; (4) Find the position of the maximum value maxR in the final feature map R and obtain the target tracking result; (5) Update the correlation filter tracker parameters according to the target tracking results to obtain the updated correlation filter tracker Continue follow-up tracking until the tracking task is completed.

2. The method according to claim 1, characterized in that: In step (5), the parameters of the correlation filter tracker are updated. After obtaining the target tracking result, the correlation filter trackers corresponding to different features are retrained using the target tracking result, and the trained correlation filter tracker w is obtained. C,H,D Correlation filter tracker used in the previous frame Linear combination, get the correlation filter required for the current t-th frame, that is, the updated correlation filter tracker 3. The method according to claim 2, characterized in that: The correlation filter required by the current t frame According to the following update model: in, represents the correlation filter updated after the target position information of the t-th frame image is determined, represents the correlation filter updated after the target position information of the t-1th frame image is determined, and η represents the smoothing coefficient.