reciprocal inter-layer temporal discriminative target model for robust visual tracking

Abstract

The application discloses a reciprocal interlayer-time discriminant target model for robust visual tracking, and belongs to the technical field of target tracking, and is used for solving the problem that traditional twin tracking algorithms rarely consider information interaction of templates and search regions, and that cumulative error of similar target interference influences tracking results. The application firstly constructs an interlayer target perception enhancement model, realizes interlayer feature information interaction by establishing pixel-by-pixel correlation between templates and search regions in a feature extraction process, reduces cumulative error caused by invisible target to search regions, and enhances target perception; meanwhile, in order to weaken the influence of interference, a time interference evaluation strategy is designed, similarity between multiple candidate positions in adjacent frames is established by using an interframe candidate propagation module, similar interference is eliminated according to the determined similarity score, a more reliable target position is obtained, and robust tracking is realized.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a robust visual tracking reciprocal interlayer-time discriminative target model. Background Technology

[0002] Visual tracking, as a crucial research topic in computer vision, has wide applications in video surveillance, autonomous driving, and road traffic monitoring. It refers to identifying a target in the first frame of a given video sequence and then tracking that target in subsequent frames. In recent years, the Siamese tracking algorithm has gained significant popularity among researchers due to its superior tracking accuracy and speed, and visual tracking technology has made great progress. However, achieving robust tracking remains a pressing issue, facing challenges such as interference from similar targets and target deformation during actual tracking.

[0003] Common Siamese tracking algorithms determine the target's location by calculating the similarity between the template and the search region. However, during feature extraction, the target is blind to the search region, which can lead to the cumulative error of similar target interference, adversely affecting the final features and thus the tracking results. To address these issues, some algorithms believe that tracking the most salient target in the search region yields superior performance, thus focusing on designing better feature extraction methods; some algorithms consider enhancing the target's representation equally important, hence the development of attention mechanisms to further focus on the target's features; and some algorithms directly address the information interaction between the two branches during feature extraction, establishing a connection between the template and the search region to improve tracking accuracy. While these algorithms, which improve tracking performance from different perspectives, have greatly promoted the development of visual tracking, relying solely on appearance-based tracking frameworks cannot adequately solve the problems caused by interference.

[0004] To improve the discriminative ability of trackers, some algorithms attempt to more effectively distinguish between targets and background by establishing interference models. Some algorithms design a learning interference-aware module to capture changes in the target's appearance during tracking; others suggest actively tracking interference by designing an association network to propagate the target and interference between different frames, using the interference to further infer the target. It is important to note that background-aware tracking algorithms often neglect further target analysis; therefore, background interference can also negatively impact the tracking results. To address these issues, it is necessary to design a tracking algorithm that enhances target awareness while suppressing background interference, aiming to further mine target features while also utilizing background discrimination capabilities. Summary of the Invention

[0005] To address the above shortcomings, this invention proposes a robust visual tracking reciprocal inter-layer-temporal discriminative target model. This method enhances target perception by establishing information interaction between the template and the search region during the feature extraction process. At the same time, it utilizes the similarity relationship between candidate objects in adjacent frames to eliminate similarity interference, achieving reciprocity between the two and improving the robustness of tracking.

[0006] To achieve the above technical objectives, the technical solution of the present invention is as follows:

[0007] A robust visual tracking reciprocal inter-layer-temporal discriminative target model includes the following steps:

[0008] Step 1: Using a pre-trained network, feature extraction is performed on the template and the search region, and then the response map M1 is obtained through the matching module.

[0009] Step 2: Through the inter-layer target perception enhancement model, during the feature extraction process, establish the pixel-wise correlation between the template and the search region, perform information interaction between the first and fourth stages of the network, and then obtain the response map M2 through the matching module.

[0010] Step 3: Weight and fuse M1 and M2 to obtain a response map M that better highlights the target;

[0011] Step 4: Retain multiple candidates in the response map M using a time-disruption evaluation strategy;

[0012] Step 5: Establish connections between multiple candidates in adjacent frames through the inter-frame candidate propagation module to obtain similarity scores;

[0013] Step 6: Eliminate similar interfering candidates based on the obtained similarity scores to obtain a more reliable target location;

[0014] Step 7: Follow up with the above steps until the video ends.

[0015] Furthermore, the network described in step one is an FBNet network.

[0016] Furthermore, the information exchange between the first and fourth stages in step two is established as follows:

[0017] First, the input features are reshaped using 1*1 convolution and maximum downsampling. Then, the similarity between each pixel between the template and the search region features is obtained using a pixel-wise correlation module to obtain similarity features. Finally, these features are aggregated using 1*1 convolution and upsampling to obtain more discriminative features.

[0018] Furthermore, the response graph M in step three is represented as follows:

[0019] M = ω*M1 + (1-ω)*M2

[0020] M is the response map that highlights the target more after weighted fusion, ω is the weighting factor used to adjust the proportion of M1 and M2, M1 is the original response map without information interaction, and M2 is the response map obtained after information interaction between the template and the search area.

[0021] Furthermore, based on the time interference assessment strategy in step four, the influence of similar interference candidate objects in the response map is eliminated to obtain a more reliable target location. The selection of candidate objects takes the following form:

[0022] C i ={(s i p i f i )}, i∈[1,5]

[0023] C i Let s represent the candidate set. i p represents the corresponding fraction. i f represents the position corresponding to the fraction. i This represents the feature extracted at this location, where i is the independent variable, indicating that it contains 5 elements.

[0024] Furthermore, s i The methods for obtaining it are as follows:

[0025] s i =Top i (s), i∈[1,5)

[0026] Where Top represents selecting the position with the highest score in the response graph as the candidate set C. i In the array, i is the independent variable, taking values ​​from 1 to 5, representing the selection of the top 5 positions p based on the scores. i .

[0027] Furthermore, feature f i The representation of is as follows:

[0028]

[0029] This represents a feature extraction network, based on the score s. i and position p i The feature f at the corresponding position is obtained. i And stored in the candidate set C i middle.

[0030] Furthermore, in step four, candidates include the object's score, location, and features.

[0031] Furthermore, the final result is represented as follows:

[0032]

[0033] Result indicates the position of the final result, p i p1 is the position of the i-th ranked result in the current frame score, p1 is the position of the result with the highest score in the current frame, and S ij This represents the similarity score between the predicted position of the current frame and the predicted position of the previous frame, determined by the inter-frame candidate propagation module.

[0034] The beneficial effects of this invention are:

[0035] (1) This invention establishes the information interaction relationship between the template and the search area during the feature extraction process through the interlayer target perception enhancement model, thereby reducing the cumulative error caused by the invisibility of the target to the search area and enhancing target perception.

[0036] (2) The present invention adopts a time interference evaluation strategy to establish the similarity relationship between multiple candidate objects in adjacent frames, and eliminates similar interference through the obtained similarity score to obtain a more reliable target location.

[0037] (3) Integrating the above two into the same tracking framework achieves mutual benefit, enhancing target perception while eliminating similar interference and improving tracking robustness. Attached Figure Description

[0038] To more concisely and clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0039] Figure 1 This is a flowchart illustrating the overall structure of the robust visual tracking reciprocal interlayer-time discriminative target model of the present invention.

[0040] Figure 2 A heatmap comparison diagram showing the results of the interlayer target perception enhancement model designed in this invention and the original results;

[0041] Figure 3 This is a comparison chart of the overall evaluation results of this invention with other algorithms on the OTB-100 dataset;

[0042] Figure 4 This is a comparison chart of the present invention with other algorithms under the deformation challenge of the OTB-100 dataset;

[0043] Figure 5 This diagram illustrates the tracking performance of the present invention and other algorithms for different tracking sequences. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be described more intuitively below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0045] like Figure 1 As shown, a robust visual tracking reciprocal inter-layer-temporal discriminative target model includes the following steps:

[0046] Step 1: Using a pre-trained FBNet network, feature extraction is performed on the template and the search region, and then the response map M1 is obtained through the matching module.

[0047] Step 2: Through the inter-layer target perception enhancement model, during the feature extraction process, establish the pixel-wise correlation between the template and the search region, perform information interaction between the first and fourth stages of FBNet, and then obtain the response map M2 through the matching module.

[0048] The information exchange between the first and fourth stages is established as follows:

[0049] First, the input features are reshaped using 1*1 convolution and maximum downsampling. Then, the similarity between each pixel between the template and the search region features is obtained using a pixel-wise correlation module to obtain similarity features. Finally, these features are aggregated using 1*1 convolution and upsampling to obtain more discriminative features.

[0050] Step 3: Weight and fuse M1 and M2 to obtain a response map M that better highlights the target;

[0051] The response graph M is represented as follows:

[0052] M = ω*M1 + (1-ω)*M2

[0053] M is the response map that highlights the target more after weighted fusion, ω is the weighting factor used to adjust the proportion of M1 and M2, M1 is the original response map without information interaction, and M2 is the response map obtained after information interaction between the template and the search area.

[0054] Step 4: Use a time-disruption evaluation strategy to retain multiple candidates in the response map M, including the candidate's score, location, and features;

[0055] Step 5: Establish connections between multiple candidates in adjacent frames through the inter-frame candidate propagation module to obtain similarity scores;

[0056] Step 6: Eliminate similar interfering candidates based on the obtained similarity scores to obtain a more reliable target location;

[0057] Step 7: Follow up with the above steps until the video ends.

[0058] By employing an inter-layer target perception enhancement model, information interaction between the template and the search region is established during feature extraction. This reduces the accumulated error caused by the target's invisibility within the search region, enhances target perception, and yields a response map that better highlights the target. To further mitigate interference, the temporal interference evaluation strategy from step four is used to eliminate the influence of similar interference candidate objects in the response map, resulting in a more reliable target location. The selection of candidate objects follows this format:

[0059] C i ={(s i p i f i )}, i∈[1,5]

[0060] C i Let s represent the candidate set. i p represents the corresponding fraction. i f represents the position corresponding to the fraction. i This represents the feature extracted at this location, where i is the independent variable, indicating that it contains 5 elements. i The methods for obtaining it are as follows:

[0061] s i =Top i (s), i∈[1,5)

[0062] Where Top represents selecting the position with the highest score in the response graph as the candidate set C. i In the array, i is the independent variable, taking values ​​from 1 to 5, representing the selection of the top 5 positions p based on the scores. i Feature f i The representation of is as follows:

[0063]

[0064] This represents a feature extraction network, based on the score s. i and position p i The feature f at the corresponding position is obtained. i And stored in the candidate set C i middle.

[0065] The final result is represented as follows:

[0066]

[0067] Result indicates the position of the final result, p ip1 is the position of the i-th ranked result in the current frame score, p1 is the position of the result with the highest score in the current frame, and S ij This represents the similarity score between the predicted position in the current frame and the predicted position in the previous frame, determined by the inter-frame candidate propagation module. The inter-frame candidate propagation module uses cosine similarity to obtain the similarity relationship between the five predicted positions in the current frame and the five stored positions in the previous frame, and based on this score relationship, removes similarity interference to obtain a more reliable target position.

[0068] The experiments of this invention were implemented using Python 3.7 on a computer with an Intel i7-10700 CPU (2.9GHz), 16GB RAM, and an NVIDIA GeForce RTX 2060.

[0069] like Figure 2 As shown, by comparing the response map generated by the inter-layer target perception enhancement model designed in this invention with the original response map, it can be found that the response map obtained after establishing the interaction between the template and the search area information is more focused on the target.

[0070] like Figure 3 As shown, the robust visual tracking reciprocal inter-layer-temporal discriminative target model (ours) provided in this invention is compared with tracking algorithms such as (SiamCAR, DaSiamRPN, ATOM, GradNet, TADT, SiamRPN, SRCF) on the OTB-100 dataset. The results show that the algorithm provided in this invention has better performance in terms of success rate and accuracy. Figure 4 As shown in the comparison of different algorithms under the deformation challenge on the OTB-100 dataset, the method provided by this invention can effectively cope with the challenge and improve the robustness of the tracking results.

[0071] like Figure 5 As shown in the figure, the robust visual tracking reciprocal inter-layer-temporal discriminative target model and other advanced tracking algorithms provided in this invention have tracking results on certain frames, namely the five videos: Biker, Bolt, Bolt2, MotorRolling, and Soccer. The tracking results all show that the tracking algorithm proposed in this invention can adapt well to complex tracking scenarios and achieve good tracking results.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A robust visual tracking reciprocal inter-layer-temporal discriminative target model, characterized in that, Includes the following steps: Step 1: Using a pre-trained network, extract features from the template and search region respectively, and then obtain the response map through the matching module. ; Step 2: Through the inter-layer target perception enhancement model, during the feature extraction process, pixel-wise correlation between the template and the search region is established, and information exchange is performed between the first and fourth stages of the network. The response map is then obtained through the matching module. ; Step 3, and Weighted fusion yields a response map that better highlights the objective. ; Step 4: Preserve the response map using a time-disruption assessment strategy. Multiple candidates; Step 5: Establish connections between multiple candidates in adjacent frames through the inter-frame candidate propagation module to obtain similarity scores; Step 6: Eliminate similar interfering candidates based on the obtained similarity scores to obtain a more reliable target location; Step 7: Follow up with the above steps until the video ends; The information exchange between the first and fourth stages in step two is established as follows: First, the input features are reshaped using 1*1 convolution and maximum downsampling. Then, the similarity between each pixel between the template and the search region features is obtained using a pixel-wise correlation module to obtain similarity features. Finally, these features are aggregated using 1*1 convolution and upsampling to obtain more discriminative features. The time-based interference assessment strategy in step four is used to eliminate the influence of similar interference candidate objects in the response map, resulting in a more reliable target location. The selection of candidate objects takes the following form: Denotes the candidate set, Indicates the corresponding score. Indicates the position corresponding to the fraction. This indicates the features extracted at this location. The independent variable is 5 elements; The methods for obtaining it are as follows: in This means selecting the position with the highest score in the response graph as the candidate set. The elements in The independent variable is denoted by 1-5, representing the five positions selected from the top five scores. ; feature The representation of is as follows: This represents a feature extraction network, based on scores. and location To obtain the features at the corresponding location and stored in the candidate set. middle.

2. The robust visual tracking reciprocal interlayer-temporal discriminative target model according to claim 1, characterized in that: The network mentioned in step one is an FBNet network.

3. The robust visual tracking reciprocal inter-layer-temporal discriminative target model according to claim 1, characterized in that: Response diagram in step three The representation is as follows: To obtain a response diagram that better highlights the objective after weighted fusion, As a weighting factor, used to adjust and The proportion, This is the original response diagram without any information exchange. The response graph obtained after information interaction between the template and the search area.

4. The robust visual tracking reciprocal inter-layer-temporal discriminative target model according to claim 1, characterized in that: In step four, candidates include the object's score, location, and features.

5. The robust visual tracking reciprocal inter-layer-temporal discriminative target model according to claim 1, characterized in that: The final result is represented as follows: Indicates the position of the final result. Ranked by current frame score The position of the result This is the position of the result with the highest score in the current frame. This represents the similarity score between the predicted position in the current frame and the predicted position in the previous frame, determined by the inter-frame candidate propagation module.

Images