MAV airborne target tracking method and system based on adaptive dynamic template

Abstract

The invention relates to a method and system for MAV airborne target tracking based on an adaptive dynamic template, belonging to the field of target tracking technology, and solving the problem of the difficulty in balancing tracking performance and computational complexity in MAV airborne tracking. The method of the invention comprises: setting an initial template, an adjacent template, and a memory template, inputting the current frame search feature, the initial template, the adjacent template, and the memory template into an adaptive template fusion STF module to generate a final template; performing correlation operations on the final template and the search template to obtain a response graph, judging the current tracking state, and updating the adjacent template and the memory template; the memory template generation module uses a temporal series method to extract and integrate key information from historical tracking results, so that the limited template feature memory can contain all historical information. The adaptive template fusion module uses the similarity matrix between the template and the search feature to realize dynamic adjustment of the template weight at different tracking stages. The invention is suitable for MAV airborne target tracking.

Description

Technical Field

[0001] The present application relates to the field of target tracking technology, and in particular to MAV airborne target tracking. Background Art

[0002] With the advancement of micro-nanotechnology and system integration, micro-unmanned aerial vehicles (MAVs) have made significant progress, with a series of new MAVs, such as the Black Hornet 3, MetaFly, and Drone40, being released. The Black Hornet 3, a modular drone, is equipped with infrared imaging and visible light cameras. While capable of real-time video and image data transmission, it still requires human intervention during mission execution, resulting in relatively low levels of autonomy and intelligence. This limitation can distract operators, reduce their decision-making ability, increase the risk of misoperation, and even lead to missed opportunities or dangerous situations. Therefore, improving the autonomy and intelligence of MAVs and enabling single-target tracking with onboard processing are crucial for improving execution efficiency and responsiveness. Consequently, a growing number of researchers are integrating drone systems with target tracking algorithms to enhance their perception capabilities in complex environments.

[0003] However, the highly maneuverable flight characteristics of MAVs, the complex flight environment, and the limited onboard computing resources present challenges for MAV airborne tracking. First, the MAV's high maneuverability often leads to drastic changes in the camera's perspective, resulting in significant changes in the target's shape, position, and projection in the image. When such visual changes occur, there is a serious mismatch between the target's current appearance model and the model learned by the tracking algorithm, leading to tracking failure. Second, in complex urban or wilderness environments, challenges such as interference and occlusion from similar objects arise. When the tracker identifies an interference object or encounters severe occlusion, erroneous information accumulates in the video stream, ultimately leading to tracking failure. Furthermore, current algorithms with strong anti-interference capabilities often also have a high number of parameters and computational complexity. MAVs have strict size and payload restrictions, and the computing devices they can carry are very limited in terms of computing power, storage space, and energy consumption, further restricting the algorithm's application on MAV platforms.

[0004] While research on target tracking methods has made significant progress, target tracking specifically for MAVs is very limited. Existing MAV airborne tracking methods remain at the stage of correlation filtering algorithms. These algorithms exhibit relatively poor robustness in complex tracking scenarios, such as background clutter and target occlusion. If features similar to the target appear in the background, or if the target is partially or completely occluded, the tracker struggles to distinguish the target from the background, resulting in tracking deviation or loss. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem of difficulty in balancing tracking performance and computational complexity in MAV airborne tracking, and to provide a MAV airborne target tracking method and system based on adaptive dynamic templates.

[0006] The present invention is achieved through the following technical solutions. On the one hand, the present invention provides a method for tracking an MAV airborne target based on an adaptive dynamic template, the method comprising:

[0007] Step 1: At the initial moment, extract features from the initial image and store the extracted features as the initial template, adjacent template, and memory template. The adjacent template refers to the features retained after state evaluation in the adjacent frames, and the memory template refers to the template that integrates all historical information through the temporal attention mechanism.

[0008] Step 2: With the current position as the center, intercept the search image, perform feature extraction on the search image, and obtain the current frame search feature;

[0009] Step 3: Input the current frame search features, initial template, adjacent template, and memory template in step 2 into the adaptive template fusion STF module to generate the final template;

[0010] Step 4: Perform correlation operations on the final template and the search template to obtain a response map, and infer the target position based on the response map;

[0011] Step 5: The evaluation module determines the current tracking status based on the response graph;

[0012] Step 6: If the tracking status is good, update the adjacent template and use the memory template generation module MTG to generate a memory template. Otherwise, proceed to the next frame tracking.

[0013] Step 7: Continue to perform steps 2 to 6 as the tracking progresses until the tracking ends.

[0014] Furthermore, in step 3, the adaptive template fusion STF module specifically includes:

[0015] Keep the three input template features and the number of search feature channels unchanged, and expand the two-dimensional feature map of each channel into one dimension;

[0016] The template features are used as query and search features as keys to calculate the similarity matrix;

[0017] Use global maximum pooling to filter the similarity matrix to obtain the fusion weight, and finally weight it to generate the final template.

[0018] Furthermore, the calculation formula of the adaptive template fusion STF module is:

[0019]

[0020] Among them, T0, T1, T m They represent the initial template, adjacent template, and memory template respectively, X represents the search feature, T represents the final template, and d k Indicates the number of channels, GMP indicates global maximum pooling, δ0, δ1, δ m Represents the fusion weights of the initial template, adjacent template, and memory template, μ represents the modulation coefficient, X T represents the transpose of X.

[0021] Furthermore, in step 5, the evaluation module includes: evaluating the current tracking state by averaging the peak correlation energy and the maximum value of the response.

[0022] Furthermore, the evaluation criteria of the evaluation module are as follows:

[0023]

[0024] Among them, R i,j 、R max 、R min They represent the response value, maximum response value, and minimum response value in the i-th row and j-th column of the response graph, respectively. avg(·) represents the average. α and β are the modulation coefficients, both of which are 0.9.

[0025] Furthermore, in step 6, the memory template generation module MTG fuses multiple frame features in a serial manner, specifically including: the input of the memory template generation module MTG includes the memory template T at the previous moment m t-1 and the current adjacent template T1 t ;

[0026] T1 t and (T m t-1 ) T Perform matrix multiplication to obtain the similarity matrix A m , A m Element a in i,j Represents T1 t The i-th row element in T m t-1 The inner product between the elements in the jth column in the Transformer is replaced by the linear normalization Linear(·) to replace the Softmax(·) of the QKV structure in the Transformer.

[0027] A m As a weighted pair T m t-1 Perform weighted summation and get the same as T1 t associated contextual information;

[0028] Use residual connection to weight the T before and after m t Add and multiply by the modulation coefficient γ, which is used to adjust T m t The value range of .

[0029] Furthermore, in step 6, the update formula of the memory template is:

[0030]

[0031] Among them, t represents the current frame and t-1 represents the previous frame.

[0032] In a second aspect, the present invention provides an MAV airborne target tracking system based on an adaptive dynamic template, the system comprising:

[0033] The template setting module is used to extract features from the initial image at the initial moment and store the extracted features as the initial template, adjacent template, and memory template. The adjacent template refers to the features retained after state evaluation in the adjacent frame, and the memory template refers to the template that integrates all historical information through the temporal attention mechanism.

[0034] The current frame search feature acquisition module is used to intercept the search image with the current position as the center, perform feature extraction on the search image, and obtain the current frame search feature;

[0035] The template fusion module is used to input the current frame search features, initial template, adjacent template, and memory template into the adaptive template fusion STF module to generate the final template;

[0036] The target position acquisition module is used to perform correlation operations on the final template and the search template to obtain a response map, and reversely infer the target position based on the response map;

[0037] Tracking state judgment module, used by the evaluation module to judge the current tracking state according to the response graph;

[0038] The template update module is used to update the adjacent template if the tracking status is good, and use the memory template generation module MTG to generate a memory template. Otherwise, it will track the next frame.

[0039] The tracking module is used to continuously execute the above modules as the tracking progresses until the tracking ends.

[0040] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the steps of the MAV airborne target tracking method based on an adaptive dynamic template as described above are executed.

[0041] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein a plurality of computer instructions are stored in the computer-readable storage medium, and the plurality of computer instructions are used to enable a computer to execute the MAV airborne target tracking method based on an adaptive dynamic template as described above.

[0042] Beneficial effects of the present invention:

[0043] This paper provides a lightweight airborne target tracking method based on adaptive dynamic templates. This method uses far fewer parameters and has a much smaller computational complexity than existing tracking methods, yet outperforms correlation filtering methods, achieving a good balance between tracking performance and computational complexity. Furthermore, this method incorporates a temporal attention mechanism and an adaptive template generation module, improving the model's adaptability and anti-interference capabilities without increasing the number of parameters.

[0044] This paper proposes a memory template generation module and an adaptive template fusion module. The memory template generation module uses a temporal tandem approach to extract and integrate key information from historical tracking results. Compared to conventional temporal fusion methods, this strategy effectively conserves storage space, allowing the limited template feature memory to contain all historical information. The adaptive template fusion module utilizes the similarity matrix between the template and the search feature to dynamically adjust the template weights at different tracking stages.

[0045] The present invention performs particularly well in dealing with complex situations such as two-dimensional plane rotation, three-dimensional rotation, and posture changes. The memory template generation module of the present invention can continuously fuse effective information of the target during the change process during tracking, thereby ensuring that stable and accurate tracking can be maintained when the target has a significant appearance difference from the initial state. The present invention also performs well in the face of occlusion problems. When the tracked object reappears after a period of partial occlusion, the adaptive fusion module dynamically adjusts the proportion of different templates, which not only enhances the adaptability of the target, but also retains the necessary anti-interference ability.

[0046] The present invention is applicable to the field of MAV airborne target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 Schematic diagram of the structure of an MAV airborne target tracking method based on an adaptive dynamic template of the present invention. DETAILED DESCRIPTION

[0049] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, but are not to be construed as limiting the present invention.

[0050] Implementation 1: A method for tracking an MAV airborne target based on an adaptive dynamic template. The method includes:

[0051] Step 1: At the initial moment, extract features from the initial image and store the extracted features as the initial template, adjacent template, and memory template. The adjacent template refers to the features retained after state evaluation in the adjacent frames, and the memory template refers to the template that integrates all historical information through the temporal attention mechanism.

[0052] Step 2: With the current position as the center, intercept the search image, perform feature extraction on the search image, and obtain the current frame search feature;

[0053] Step 3: Input the current frame search features, initial template, adjacent template, and memory template in step 2 into the adaptive template fusion STF module to generate the final template;

[0054] Step 4: Perform correlation operations on the final template and the search template to obtain a response map, and infer the target position based on the response map;

[0055] Step 5: The evaluation module determines the current tracking status based on the response graph;

[0056] Step 6: If the tracking status is good, update the adjacent template and memory template, and use the memory template generation module MTG to generate a memory template. Otherwise, proceed to the next frame tracking.

[0057] Step 7: Continue to perform steps 2 to 6 as the tracking progresses until the tracking ends.

[0058] This implementation proposes a method for tracking targets onboard MAVs based on adaptive dynamic templates. This method uses far fewer parameters and has a much lower computational complexity than existing advanced tracking methods. It outperforms correlation filtering methods and achieves a good balance between tracking performance and computational complexity. Furthermore, this method incorporates a temporal attention mechanism and an adaptive template generation module, improving the model's adaptability and anti-interference capabilities without increasing the number of parameters.

[0059] Implementation method 2: This implementation method further defines the MAV airborne target tracking method based on the adaptive dynamic template as described above. In this implementation method, the adaptive template fusion STF module in step 3 is further defined, specifically including:

[0060] In step 3, the adaptive template fusion STF module specifically includes:

[0061] Keep the three input template features and the number of search feature channels unchanged, and expand the two-dimensional feature map of each channel into one dimension;

[0062] The template features are used as query and search features as keys to calculate the similarity matrix;

[0063] Use global maximum pooling to filter the similarity matrix to obtain the fusion weight, and finally weight it to generate the final template.

[0064] This embodiment provides an adaptive template fusion module (STF). This module performs two main operations: calculating the similarity matrix between template features and search features, and filtering weights using global maximum pooling. Considering the inherent uncertainty of the tracking task and the inability to predict the fusion ratio of the three templates at a given tracking moment, this module dynamically adjusts the fusion weights based on the target's tracking status.

[0065] Implementation method three: This implementation method further limits the MAV airborne target tracking method based on the adaptive dynamic template as described above. In this implementation method, the formula for the calculation process of the adaptive template fusion STF module in step 3 is further limited, specifically including:

[0066] The calculation formula of the adaptive template fusion STF module is:

[0067]

[0068] Among them, T0, T1, T m They represent the initial template, adjacent template, and memory template respectively, X represents the search feature, T represents the final template, and d k Indicates the number of channels, GMP indicates global maximum pooling, δ0, δ1, δ m Represents the fusion weights of the initial template, adjacent template, and memory template, μ represents the modulation coefficient, X T represents the transpose of X.

[0069] Implementation 4: This implementation further defines the MAV airborne target tracking method based on an adaptive dynamic template as described above. In this implementation, the evaluation module is further defined, specifically including:

[0070] In step 5, the evaluation module includes: evaluating the current tracking state by averaging the peak correlation energy and the maximum response value.

[0071] The evaluation module of this embodiment is used to determine the tracking state. The module evaluates the current tracking state through the average peak-to-correlation energy (APCE) and the maximum response value, and filters the tracking template to reduce the introduction of erroneous information.

[0072] Implementation 5: This implementation further defines the MAV airborne target tracking method based on an adaptive dynamic template as described above. In this implementation, the evaluation criteria of the evaluation module are further defined, specifically including:

[0073] The evaluation criteria for the evaluation module are as follows:

[0074]

[0075] Among them, R i,j 、R max 、R min The values ​​are the response value, maximum response value, and minimum response value in the response graph, respectively. avg(·) represents the average, α and β are the modulation coefficients, both set to 0.9, and the t in the upper right corner represents the data of the tth frame.

[0076] This embodiment provides an evaluation criterion applicable to target tracking based on the adaptive dynamic template as described above, for determining the tracking state.

[0077] Implementation 6: This implementation further defines the MAV airborne target tracking method based on an adaptive dynamic template as described above. In this implementation, the memory template generation module MTG in step 6 is further defined, specifically including:

[0078] Step 6 involves the memory template generation module MTG, which fuses multi-frame features in series. The input of MTG includes the memory template T m t-1 and the current adjacent template T1 t .

[0079] T1 t and (T m t-1 ) T Perform matrix multiplication to obtain the similarity matrix A m , A m Element a in i,j Represents T1 t The i-th row element in Tm t-1 The inner product between the elements in the j-th column in the Transformer is replaced by the linear normalization Linear(·) to reduce nonlinear calculations and prepare for subsequent edge device deployment.

[0080] A m As a weighted pair T m t-1 Perform weighted summation to obtain the same value as T1 t Associated context information to achieve effective aggregation of information between historical frame templates and current frame templates.

[0081] Since MTG uses the time series dimension to aggregate information, it is necessary to ensure that the T generated at each moment is m t Expressing the same spatial attributes, the residual connection is used at the end to weight the T before and after m t Add and multiply by a modulation coefficient γ, where the modulation coefficient is used to keep T m t The value range does not fluctuate greatly.

[0082] Implementation method seven: This implementation method further limits the MAV airborne target tracking method based on the adaptive dynamic template as described above. In this implementation method, the update formula of the memory template in step 6 is further limited, specifically including:

[0083] In step 6, the update formula of the memory template is:

[0084]

[0085] Among them, t represents the current frame and t-1 represents the previous frame.

[0086] This embodiment provides an update formula for a memory template. In addition, the update of the adjacent template is performed by evaluating the current tracking state through the evaluation module. If the state is good, the search feature is directly used as the adjacent template.

[0087] Implementation 8: This implementation is an example of a MAV airborne target tracking method based on an adaptive dynamic template as described above. Figure 1 As shown, specifically including:

[0088] Aiming at the problem of balancing tracking performance and computational complexity in MAV airborne tracking, a MAV airborne target tracking method based on adaptive dynamic template is proposed. The specific scheme is as follows:

[0089] Step 1: At the initial moment, feature extraction is performed on the initial image and the extracted features are stored as the initial template, adjacent template, and memory template. The adjacent template refers to the features retained after state evaluation in adjacent frames. During the tracking process, only one frame of features is retained and continuously updated as the tracking state changes. The memory template is a template that integrates all historical information through the temporal attention mechanism, which will be described in detail in the subsequent MTG module.

[0090] Step 2: Take the current position as the center, intercept the search image, extract the features of the search image, and obtain the search features of the current frame;

[0091] Step 3: Input the current frame search features, initial template, adjacent template, and memory template in step 2 into the adaptive template fusion STF module to generate the final template;

[0092] The adaptive template fusion module (STF) takes into account the inherent uncertainty of the tracking task and the inability to predict the fusion ratio of the three templates at a given moment. Therefore, it dynamically adjusts the fusion weights based on the target's tracking status. This module mainly performs two steps: calculating the similarity matrix between the template features and the search features, and filtering the weights using global maximum pooling. The specific calculation process is as follows:

[0093] Keep the three input template features and the number of search feature channels unchanged, and expand the two-dimensional feature map of each channel into one dimension;

[0094] The template features are used as query and search features as keys to calculate the similarity matrix;

[0095] Use global maximum pooling to filter the similarity matrix to obtain the fusion weight, and finally weight it to generate the final template.

[0096] The above process can be formed into the following formula:

[0097]

[0098] Among them, T0, T1, T m They represent the initial template, adjacent template, and memory template respectively, X represents the search feature, T represents the final template, and d k Indicates the number of channels, GMP indicates global maximum pooling, δ0, δ1, δ m Represent the fusion weights of the initial template, adjacent template, and memory template, μ represents the modulation coefficient, and X T represents the transpose of X.

[0099] Step 4: Perform correlation operations on the final template and the search template to obtain a response map, and infer the target position based on the response map to infer the current target position based on the response map;

[0100] Step 5: The evaluation module determines the current tracking status based on the response graph; if the current tracking status is good, this frame is saved as a nearby template.

[0101] The tracking status evaluation module evaluates the current tracking status using the average peak-to-correlation energy (APCE) and the maximum response value, and filters the tracking template to reduce the introduction of erroneous information. The evaluation criteria of the evaluation module are as follows:

[0102]

[0103] R max t ≥α·R max 0

[0104]

[0105] Among them, R i,j 、R max 、R min They represent the response value, maximum response value, and minimum response value in the i-th row and j-th column of the response graph, respectively. avg(·) represents the average. α and β are the modulation coefficients, both of which are 0.9.

[0106] Step 6: If the tracking status is good, update the adjacent template and memory template, and use the memory template generation module MTG to generate a memory template. Otherwise, proceed to tracking the next frame.

[0107] The memory template generation module MTG fuses multiple frame features in a serial manner. The input of MTG includes the memory template T m t-1 and the current adjacent template T1 t .

[0108] T1 t and (T m t-1 ) T Perform matrix multiplication to obtain the similarity matrix A m , A m Element a in i,j Represents T1 t The i-th row element in T m t-1 The inner product between the elements in the j-th column in the Transformer is replaced by the linear normalization Linear(·) to reduce nonlinear calculations and prepare for subsequent edge device deployment.

[0109] A mAs a weighted pair T m t-1 Perform weighted summation to obtain the same value as T1 t Associated context information to achieve effective aggregation of information between historical frame templates and current frame templates.

[0110] Since MTG uses the time series dimension to aggregate information, it is necessary to ensure that the T generated at each moment is m t Expressing the same spatial attributes, the residual connection is used at the end to weight the T before and after m t Add and multiply by a modulation coefficient γ, where the modulation coefficient is used to keep T m t The value range does not fluctuate greatly.

[0111] The updated memory template calculation formula is as follows:

[0112]

[0113] Among them, t represents the current frame and t-1 represents the previous frame.

[0114] The evaluation module evaluates the current tracking status and directly uses the search features as adjacent templates if the status is good.

[0115] Step 7: Continue to execute steps 2 to 6 as the tracking progresses until the tracking is completed.

[0116] The method of the present invention is compared with several tracking methods in the prior art using the OTB100 and UAV123 datasets, as shown in the following table:

[0117]

[0118] As can be seen, the proposed method outperforms the baseline SiamFC algorithm and advanced correlation filtering methods. Compared with SiamFC, the proposed method improves accuracy by 4.1% and success rate by 2.1% on the OTB100 dataset, and improves accuracy by 4.8% and success rate by 1.2% on the UAV123 dataset.

Claims

1. A method for tracking airborne targets in a MAV based on an adaptive dynamic template, characterized in that: The method comprises: Step 1: At the initial moment, extract features from the initial image and store the extracted features as the initial template, adjacent template, and memory template. The adjacent template refers to the features retained after state evaluation in the adjacent frames, and the memory template refers to the template that integrates all historical information through the temporal attention mechanism. Step 2: With the current position as the center, intercept the search image, perform feature extraction on the search image, and obtain the current frame search feature; Step 3: Input the current frame search feature, initial template, adjacent template, and memory template in step 2 into the adaptive template fusion STF module to generate the final template. The calculation formula of the adaptive template fusion STF module is: in, 、 、 Represent the initial template, adjacent template, and memory template respectively. Represents the search feature, Represents the final template, Indicates the number of channels, GMP indicates global maximum pooling, 、 、 Represent the fusion weights of the initial template, adjacent template, and memory template respectively, represents the modulation coefficient, represents the transpose of X; Step 4: Perform correlation operations on the final template and the search template to obtain a response map, and infer the target position based on the response map; Step 5: The evaluation module determines the current tracking status based on the response graph; Step 6: If the tracking status is good, update the adjacent template and use the memory template generation module MTG to generate a memory template. Otherwise, proceed to the next frame tracking. The memory template generation module MTG fuses multiple frame features in series. Specifically, the input of the memory template generation module MTG includes the memory template of the previous moment. and the current time's proximity template ; and Perform matrix multiplication to obtain the similarity matrix , Elements in represent Middle Row elements and The inner product between the elements in the jth column is linearly normalized. Replace the QKV structure in Transformer ; by As a weighted pair Perform weighted summation and get associated contextual information; Use residual connection to weight the front and back Add and multiply by the modulation coefficient , modulation coefficient For adjustment The value range of The update formula of the memory template is: Among them, t represents the current frame, and t-1 represents the previous frame; Step 7: Continue to perform steps 2 to 6 as the tracking progresses until the tracking ends.

2. The MAV airborne target tracking method based on adaptive dynamic template according to claim 1, characterized in that: In step 3, the adaptive template fusion STF module specifically includes: Keep the three input template features and the number of search feature channels unchanged, and expand the two-dimensional feature map of each channel into one dimension; The template features are used as query and search features as keys to calculate the similarity matrix; Use global maximum pooling to filter the similarity matrix to obtain the fusion weight, and finally weight it to generate the final template.

3. The MAV airborne target tracking method based on adaptive dynamic template according to claim 1, characterized in that: In step 5, the evaluation module includes: evaluating the current tracking state by averaging the peak correlation energy and the maximum response value.

4. The method for tracking an MAV airborne target based on an adaptive dynamic template according to claim 3, wherein: The evaluation criteria for the evaluation module are as follows: in, 、 、 Respectively represent the response value, maximum response value, and minimum response value in the i-th row and j-th column of the response graph. It means taking the average, α and β are the modulation coefficients, both of which are 0.

9.

5. A MAV airborne target tracking system based on adaptive dynamic template, characterized in that: The system comprises: The template setting module is used to extract features from the initial image at the initial moment and store the extracted features as the initial template, adjacent template, and memory template. The adjacent template refers to the features retained after state evaluation in the adjacent frame, and the memory template refers to the template that integrates all historical information through the temporal attention mechanism. The current frame search feature acquisition module is used to intercept the search image with the current position as the center, perform feature extraction on the search image, and obtain the current frame search feature; The template fusion module is used to input the current frame search feature, initial template, adjacent template, and memory template into the adaptive template fusion STF module to generate the final template. The calculation formula of the adaptive template fusion STF module is: in, 、 、 Represent the initial template, adjacent template, and memory template respectively. Represents the search feature, Represents the final template, Indicates the number of channels, GMP indicates global maximum pooling, 、 、 Represent the fusion weights of the initial template, adjacent template, and memory template respectively, represents the modulation coefficient, represents the transpose of X; The target position acquisition module is used to perform correlation operations on the final template and the search template to obtain a response map, and reversely infer the target position based on the response map; Tracking state judgment module, used by the evaluation module to judge the current tracking state according to the response graph; The template update module is used to update the adjacent template if the tracking status is good, and use the memory template generation module MTG to generate a memory template. Otherwise, it tracks the next frame. The memory template generation module MTG fuses multiple frame features in series, specifically including: the input of the memory template generation module MTG includes the memory template of the previous moment and the current time's proximity template ; and Perform matrix multiplication to obtain the similarity matrix , Elements in represent Middle Row elements and The inner product between the elements in the jth column is linearly normalized. Replace the QKV structure in Transformer ; by As a weighted pair Perform weighted summation and get associated contextual information; Use residual connection to weight the front and back Add and multiply by the modulation coefficient , modulation coefficient For adjustment The value range of The update formula of the memory template is: Among them, t represents the current frame, and t-1 represents the previous frame; The tracking module is used to continuously execute the above modules as the tracking progresses until the tracking ends.

6. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor runs the computer program stored in the memory, the steps of the method according to any one of claims 1 to 4 are performed.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of computer instructions, and the plurality of computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 4.

Images