An unmanned aerial vehicle target tracking method and system based on bionic vision double memory collaborative perception

Abstract

The present application belongs to the field of unmanned aerial vehicle intelligent perception, and provides an unmanned aerial vehicle target tracking method and system based on bionic vision double memory collaborative perception. Deeply draw lessons from the cognitive principle of biological vision system: in the short-term memory perception module, simulate the biological foveal vision mechanism, quickly capture the significant features in the current search frame and make immediate response; in the long-term memory perception module, construct a learnable long-term memory library to maintain the stable semantic representation of the target; the dynamic memory gating fusion process simulates the dynamic regulation mechanism of the biological brain to different memory types, and adaptively adjusts the contribution proportion of the short-term and long-term memory features according to the scene change. The unmanned aerial vehicle target tracking solution of the present application improves the target positioning accuracy of the unmanned aerial vehicle in the small target and high dynamic scene, and has significant practical value and application prospect in the fields of unmanned aerial vehicle dynamic tracking and complex environment perception such as military reconnaissance, traffic monitoring and disaster rescue.

Description

Technical Field

[0001] This invention relates to the fields of unmanned aerial vehicle (UAV) target tracking, computer vision, and intelligent perception, and particularly to a UAV target tracking method and system based on biomimetic visual dual-memory collaborative perception. Background Technology

[0002] Unmanned aerial vehicle (UAV) target tracking, as a core technology of aerial intelligent sensing systems, plays an irreplaceable role in many fields. In recent years, thanks to the development of deep convolutional neural networks and the Transformer architecture, tracking algorithms have made progress in feature representation and discrimination capabilities. However, existing mainstream methods still have significant limitations, such as:

[0003] Insufficient stability over time: Existing methods are mostly limited to data-driven pattern matching paradigms, lacking cognitive priors in spatiotemporal continuous perception, making it difficult to reproduce the coherent understanding of targets by biological vision. In scenarios where UAVs move rapidly at low altitudes and frequently switch perspectives, targets are easily affected by violent movements, occlusion, and background interference, leading to fluctuations in visual cues and feature drift. It is difficult to maintain a stable and consistent target representation over long time, resulting in poor tracking stability.

[0004] Limited temporal information modeling: Although existing methods attempt to introduce information from the previous few frames of the target to assist in tracking and localization, they only use historical information as an additional supplement. They do not differentiate the functional characteristics of information at different time scales, do not establish a hierarchical temporal information allocation and dynamic calling mechanism, and lack an effective mechanism to distinguish between the functional requirements of short-term rapid response and long-term feature consolidation. As a result, the model cannot achieve a dynamic balance between agile response and stable maintenance.

[0005] It is evident that existing technologies largely rely on purely engineering-based network optimization (such as single-memory storage and full-map uniform perception), resulting in weak anti-interference capabilities and insufficient adaptive capabilities in complex dynamic scenarios. Therefore, there is an urgent need for a UAV target tracking method that can balance rapid response capabilities with cross-temporal stability and effectively address the challenges of complex dynamic environments. Summary of the Invention

[0006] To address the shortcomings of existing UAV target tracking methods in complex dynamic environments, such as insufficient cross-temporal stability and limited temporal information modeling, this invention provides a UAV target tracking method and system based on biomimetic visual dual-memory collaborative perception. By deeply integrating the multi-layered memory collaboration and spatial perception mechanisms of biological vision systems, and simulating the short-term rapid response, long-term stable representation, and dynamic memory regulation characteristics of biological vision, differentiated modeling and dynamic fusion of information at different time scales are achieved, thereby improving the robustness and generalization ability of the tracking system.

[0007] On the one hand, according to the design scheme provided by this invention, a UAV target tracking method based on biomimetic visual dual-memory collaborative perception is provided, including the following:

[0008] Get the current search frame and extract its feature map;

[0009] A dual-memory collaborative perception and tracking model is constructed and trained; the dual-memory collaborative perception and tracking model includes a short-term memory perception module, a long-term memory perception module, a dynamic memory gating fusion module, and a prediction head;

[0010] The short-term memory perception module generates a saliency mask for a given video frame based on its feature map, and uses the saliency mask to obtain short-term memory features. The long-term memory perception module performs a linear mapping on the feature map of a given video frame to generate a query key and a query value; calculates the similarity between the query key and all memory keys stored in a preset long-term memory bank, and calculates long-term memory features based on the similarity, all memory values ​​stored in the long-term memory bank, and the query value. The dynamic memory gating fusion module constructs an initial fusion feature based on the short-term memory features and the long-term memory features, and adjusts the initial fusion feature through a channel gating mechanism and a spatial gating mechanism to obtain the final fusion feature; the final fusion feature is input to the prediction head, and finally the target's position prediction result and confidence level are obtained from the given video frame.

[0011] The feature map of the current search frame is input into the trained dual-memory collaborative sensing and tracking model to obtain the position prediction result and confidence of the target in the current search frame.

[0012] Furthermore, a saliency mask for a given video frame is generated based on its feature map, specifically including:

[0013] Global average pooling is used to extract the overall information of the feature map. The overall information is then used to predict multiple sets of scale weights through multi-head mapping. The feature map is then weighted and fused based on the scale weights. The weighted and fused features are then subjected to function activation processing to generate a saliency mask for the current search frame.

[0014] Furthermore, the short-term memory features are obtained using the saliency mask, specifically including:

[0015] Using the saliency mask, the feature map is divided into salient features and residual features, and the salient features and residual features are fused by depthwise separable convolution to obtain short-term memory features.

[0016] Furthermore, it also includes updating the long-term memory, specifically including:

[0017] The feature map is linearly mapped to generate candidate memory keys and memory values. If the confidence result of the location prediction result in the current search frame is higher than a preset threshold, the candidate memory keys and memory values ​​are written into the long-term memory. When the capacity of the long-term memory reaches the upper limit, a time decay strategy is used to remove expired or low-contribution memory keys and memory values.

[0018] Furthermore, a linear mapping is performed on the feature map to generate candidate memory keys and memory values, specifically including:

[0019] The feature map is modulated based on the confidence level of the target location prediction result in the current search frame. The modulated feature map is then processed by a ResNet network and further processed by a module consisting of depthwise separable convolution, channel attention mechanism, spatial attention mechanism, batch normalization and nonlinear activation function to obtain processed features. Candidate memory keys and memory values ​​are generated from the processed features.

[0020] Furthermore, the computational process for obtaining long-term memory features specifically includes:

[0021] A global similarity matrix is ​​obtained by performing a global similarity calculation between the query key and all memory keys stored in the long-term memory:

[0022]

[0023] in, Represents the global affinity matrix. The number of channels for the memory keys. This refers to the query key. This refers to the memory key in the long-term memory bank.

[0024] Next, the global affinity matrix is ​​used to perform a weighted summation of all memory values ​​stored in the long-term memory bank to obtain the long-term memory semantic compensation features of the current search frame. The long-term memory semantic compensation features are then concatenated with the query value to obtain the long-term memory features.

[0025] Furthermore, an initial fusion feature is constructed based on the short-term memory features and the long-term memory features, specifically including:

[0026] The short-term memory features and the long-term memory features are reshaped into a two-dimensional spatial mapping form through linear mapping, and the initial fusion features are constructed using the two-dimensional spatial mapping form. :

[0027]

[0028] in, For lightweight convolutional fusion blocks, This is a characteristic of short-term memory. It is a two-dimensional spatial mapping form. Specifically, the method of jointly adjusting the initial fusion features through channel gating and spatial gating mechanisms to obtain the final fusion features includes:

[0029] The initial fusion features are modulated using channel gating and spatial gating to obtain the final fusion features. :

[0030]

[0031] in, For initial fusion features, For channel gating, For spatial gating.

[0032] Furthermore, the total loss function used in the training process of the dual-memory collaborative sensing tracking model is:

[0033]

[0034] in, For weighted focus loss, For GIoU loss, For L1 loss, and This is the regularization parameter.

[0035] On the other hand, the present invention also provides a UAV target tracking system based on biomimetic visual dual-memory collaborative perception, comprising:

[0036] Feature extraction unit: used to acquire the current search frame and extract the feature map of the current search frame;

[0037] Model building and training unit: used to build and train the dual-memory collaborative perception tracking model; the dual-memory collaborative perception tracking model includes a short-term memory perception module, a long-term memory perception module, a dynamic memory gating fusion module, and a prediction head;

[0038] The short-term memory perception module generates a saliency mask for a given video frame based on its feature map, and uses the saliency mask to obtain short-term memory features. The long-term memory perception module performs a linear mapping on the feature map of a given video frame to generate a query key and a query value; calculates the similarity between the query key and all memory keys stored in a preset long-term memory bank, and calculates long-term memory features based on the similarity, all memory values ​​stored in the long-term memory bank, and the query value. The dynamic memory gating fusion module constructs an initial fusion feature based on the short-term memory features and the long-term memory features, and adjusts the initial fusion feature through a channel gating mechanism and a spatial gating mechanism to obtain the final fusion feature; the final fusion feature is input to the prediction head, and finally the target's position prediction result and confidence level are obtained from the given video frame.

[0039] The result prediction unit is used to input the feature map of the current search frame into the trained dual-memory collaborative sensing tracking model to obtain the position prediction result and confidence of the target in the current search frame.

[0040] The beneficial effects of this invention are:

[0041] (1) By using the dual-memory collaborative perception framework, the division of labor and cooperation mechanism of short-term memory and long-term memory in biological vision is simulated, and the functional requirements of short-term rapid response and long-term stable representation are explicitly distinguished. This can effectively alleviate the problem of cross-time instability, and take into account the rapid adaptability when the target appearance changes and the tracking stability under long time.

[0042] (2) The short-term memory sensing module quickly focuses on the instantaneous salient area of ​​the current search frame through the dynamic central fovea sensing mechanism, which can suppress background interference and improve the target positioning accuracy of UAV in small target and high dynamic scenarios.

[0043] (3) The long-term memory perception module provides stable semantic support across time scales through memory-guided modulation and memory retrieval mechanisms, enabling the model to maintain the consistency of the target identity in scenarios with occlusion, weak texture, and background interference.

[0044] (4) The dynamic memory gating fusion module simulates the dynamic recall of different memory types by the synergistic effect of channel gating and spatial gating. It enhances the weight of short-term memory when the target mutates and enhances the weight of long-term memory in occlusion and weak texture scenes, thus achieving flexible adaptation to complex dynamic scenes.

[0045] (5) The confidence-driven long-term memory update strategy effectively avoids the contamination of the memory by low-confidence samples, suppresses error accumulation and memory expansion, and improves the robustness of the model. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a UAV target tracking method based on biomimetic visual dual-memory collaborative perception in an embodiment.

[0047] Figure 2 This is a schematic diagram of the overall architecture of a UAV target tracking method based on biomimetic visual dual-memory collaborative perception in an embodiment.

[0048] Figure 3 This is a schematic diagram of the short-term memory perception module architecture in a UAV target tracking method based on biomimetic visual dual-memory collaborative perception in an embodiment.

[0049] Figure 4 This is a schematic diagram of the long-term memory perception module architecture in a UAV target tracking method based on biomimetic visual dual-memory collaborative perception in an embodiment.

[0050] Figure 5 This is a schematic diagram of the dynamic memory gating fusion module architecture in a UAV target tracking method based on biomimetic visual dual-memory collaborative perception in an embodiment.

[0051] Figure 6 This is a schematic diagram showing the success rate comparison curves of a UAV target tracking method based on biomimetic visual dual-memory collaborative perception and existing advanced methods on a typical dataset in this embodiment.

[0052] Figure 7 This is a schematic diagram comparing the accuracy of a UAV target tracking method based on biomimetic visual dual-memory collaborative perception with existing advanced methods on a typical dataset in this embodiment.

[0053] Figure 8 This is a schematic diagram comparing the normalized accuracy curves of a UAV target tracking method based on biomimetic visual dual-memory collaborative perception and existing advanced methods on a typical dataset in this embodiment.

[0054] Figure 9 This is a visualization comparing the tracking performance of a UAV target tracking method based on biomimetic visual dual-memory collaborative perception with other methods on a UAV target tracking dataset, as shown in the embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0056] like Figure 1As shown, this embodiment of the invention provides a UAV target tracking method based on biomimetic visual dual-memory collaborative perception, including:

[0057] S101. Obtain the current search frame and extract the feature map of the current search frame;

[0058] Input the current search frame into the backbone network to extract hierarchical features and obtain the feature map of the current search frame.

[0059] S102. Construct and train a dual-memory collaborative perception tracking model; the dual-memory collaborative perception tracking model includes a short-term memory perception module, a long-term memory perception module, a dynamic memory gating fusion module, and a prediction head;

[0060] The short-term memory perception module generates a saliency mask for a given video frame based on its feature map, and uses the saliency mask to obtain short-term memory features. The long-term memory perception module performs a linear mapping on the feature map of a given video frame to generate a query key and a query value; calculates the similarity between the query key and all memory keys stored in a preset long-term memory bank, and calculates long-term memory features based on the similarity, all memory values ​​stored in the long-term memory bank, and the query value. The dynamic memory gating fusion module constructs an initial fusion feature based on the short-term memory features and the long-term memory features, and adjusts the initial fusion feature through a channel gating mechanism and a spatial gating mechanism to obtain the final fusion feature; the final fusion feature is input to the prediction head, and finally the target's position prediction result and confidence level are obtained from the given video frame.

[0061] Furthermore, the short-term memory perception module generates a saliency mask for a given video frame based on the feature map of that video frame. Specifically, this includes:

[0062] Global average pooling is used to extract the overall information of the feature map. The overall information is then used to predict multiple sets of scale weights through multi-head mapping. The feature map is then weighted and fused based on the scale weights. The weighted and fused features are then subjected to function activation processing to generate a saliency mask for the current search frame.

[0063] Specifically, the feature map extracted from the backbone network of the current search frame Global average pooling (GAP) is used to extract overall statistical information, and multi-head mapping is used to predict the scale weights of three groups corresponding to different receptive fields (1×1, 3×3, 5×5). :

[0064]

[0065] The multi-scale convolutional features are weighted and fused using the aforementioned scale weights to obtain the weighted fused features. :

[0066]

[0067] in, , The scale weights are used for adaptive prediction in lightweight multi-head networks to dynamically balance the local and global receptive fields.

[0068] After multi-scale feature fusion, the SMP module further generates a saliency mask through Sigmoid function activation. :

[0069]

[0070] in, The foveal attention distribution in the feature space is represented by B, where B is the number of features, and H and W are the height and width of the features. The high-response region of the saliency mask corresponds to the short-term visual focus of the model in the current search frame.

[0071] Furthermore, the short-term memory perception module utilizes the saliency mask to obtain short-term memory features, including:

[0072] Using the saliency mask, the feature map is divided into salient features and residual features, and the salient features and residual features are fused by depthwise separable convolution to obtain short-term memory features.

[0073] Specifically, the feature map is divided into salient features (central concavity). With residual features (surroundings) :

[0074]

[0075] Short-term memory perceptual features are obtained by fusing channel splicing and depthwise separable convolution. This preserves the high-resolution features of the key target areas without losing background context information.

[0076]

[0077] in, This indicates an element-wise multiplication operation. This represents a 3×3 depth separable convolution, where BN is the batch normalization operation.

[0078] Figure 3 The diagram shows the architecture of the short-term memory perception module, which includes a dynamic foveal perception unit that simulates the foveal-peripheral visual division of labor in the human eye and a lightweight weight generator that simulates the biological attention allocation mechanism, in order to quickly capture the instantaneous salient region features in the current search frame and suppress background interference.

[0079] Furthermore, the long-term memory perception module updates the long-term memory bank, specifically including three processes: a memory-guided perception modulation process, a hierarchical perception fine-grained modeling process, and a long-term memory bank construction and update process.

[0080] The goal of the memory-guided perceptual modulation process and the hierarchical perceptual refinement modeling process is to obtain candidate memory keys and memory values ​​from the feature map, and to update the long-term memory bank using the candidate memory keys and memory values.

[0081] Memory-guided perceptual modulation process: confidence distribution of position prediction results in the current search frame Explicitly embedding the feature map enables consistency guidance for the current search frame during the perception phase, enhancing the feature response of the target region and suppressing background interference.

[0082]

[0083] in, The modulation intensity coefficient is used to modulate the degree of influence of the target prediction result on the current feature input. The feature map of the current search frame; The response map obtained from the target location prediction results of the current search frame is represented as the confidence distribution of the target in spatial location.

[0084] Hierarchical perception refinement modeling process: modulated features After processing by the ResNet network, the stable structural features of the target are enhanced through a hierarchical perception refinement process. The HPRB process consists of depthwise separable convolution, channel attention mechanism, spatial attention mechanism, batch normalization and nonlinear activation function, which simulates the hierarchical feature processing process of the visual cortex from low level to high level.

[0085]

[0086]

[0087] The features processed by the HPRB procedure are mapped using a linear mapping. Generate candidate memory keys and memory values ;

[0088] Furthermore, the long-term memory construction and update process specifically includes:

[0089] Long-term memory bank Constructed using a key-value pair structure:

[0090]

[0091] in, For memory keys, Here, C is the memory value, H and W are the height and width of the feature, respectively.

[0092] Furthermore, in the confidence-driven long-term memory update strategy, a pre-set confidence threshold is included. It is used to filter candidate memory keys and memory values, balancing memory reliability and temporal diversity.

[0093] If the confidence level of the current search frame position prediction result is higher than the preset threshold (0.6), the candidate memory key and memory value are... Write to the long-term memory. When the long-term memory reaches its capacity limit, a time decay strategy is used to remove expired or low-contribution memory keys and values.

[0094]

[0095] in, This represents the state of memory bank M after the memory update is completed at time t. This represents the state of memory bank M after the memory update is completed at time t-1.

[0096] Furthermore, at the start of the tracking task (i.e., the first frame processing stage), the long-term memory is empty because there are no prediction results for historical tracking frames. At this time, the target features of the template image are used to replace the prediction results to initialize the long-term memory. During the formal tracking stage, the long-term memory perception process uses the query key of the current search frame features. With all memory bonds in the long-term memory bank The similarity between them is used to calculate the retrieval weight.

[0097] Specifically, the long-term memory perception module and memory retrieval unit include a calculation process for obtaining long-term memory features, specifically:

[0098] The feature map of the current search frame is obtained through the CRB module. Mapped to query key and query value ,in The global affinity matrix is ​​used to calculate the semantic similarity between the current query and all historical memories.

[0099]

[0100] in, Represents the global affinity matrix. The number of channels for the memory keys. This refers to the query key. This refers to the memory key in the long-term memory bank.

[0101] After Softmax normalization, the memory values ​​in the long-term memory bank We perform a weighted summation to obtain the long-term memory semantic compensation features of the current search frame. :

[0102]

[0103] Concatenate it with the current query features to output long-term memory features :

[0104]

[0105] Figure 4 The diagram illustrates the architecture of the long-term memory perception module, which includes a hierarchical perception refinement module (HPRB), a long-term memory bank that simulates the biological memory storage structure, and a memory retrieval unit. This module is used to maintain stable semantic features across time scales and provide continuous support for target representation.

[0106] Furthermore, the dynamic memory gating fusion module constructs initial fusion features based on the short-term memory features and the long-term memory features, specifically including:

[0107] Short-term memory characteristics Characteristics of long-term memory Semantic alignment via linear mapping and reshaping into a two-dimensional spatial mapping form Construct a joint feature space across time domains, i.e., the initial fused features. :

[0108]

[0109] in, It is a lightweight convolutional fusion block that enables non-linear interaction and spatial alignment.

[0110] Then through the access gate With spatial gating Fusion features Modulation, channel gating Simulating biological judgments on the importance of semantic features to enhance discriminative semantic channel responses; spatial gating. Simulate biological filtering of spatial location relevance, suppress background and uncertain region responses, and obtain the final fused features. :

[0111]

[0112] Among them, the residual pathway preserves the temporal continuity of short-term response, ensuring the model's sensitivity to instantaneous changes and reproducing the core characteristics of agile response and stable cognition in biological visual dynamic equilibrium.

[0113] Furthermore, the fusion features The system takes a prediction head as input and outputs the target position regression result and classification confidence score. A Hanning window is used for position penalty. The position prediction result of the current search frame is used as reference historical information for the next frame, and the target state is updated. Then, the process repeats in the next frame to achieve continuous target tracking.

[0114] Furthermore, the total loss function used in the training process of the dual-memory collaborative sensing tracking model is:

[0115]

[0116] in, For weighted focus loss, For GIoU loss, For L1 loss, =2.0、 =5.0 is the regularization parameter.

[0117] Figure 5 The diagram shows the architecture of the dynamic memory gating fusion module, which includes a channel gating unit and a spatial gating unit. It simulates the dynamic regulation mechanism of the biological brain for different memory types and is used to adaptively adjust the contribution ratio of short-term memory features and long-term memory features.

[0118] S103. Input the feature map of the current search frame into the trained dual-memory collaborative sensing tracking model to obtain the position prediction result and confidence of the target in the current search frame.

[0119] Figure 2 The diagram illustrates the overall architecture of this invention. This invention constructs a dual-pathway structure of short-term memory perception and long-term memory perception. Template frames and the current search frame are input into the backbone network to extract hierarchical features. The short-term memory perception module uses Dynamic Foveal Perception (DNP) units to adaptively focus on key locations within the search area, quickly capturing significant visual cues related to the target in the current search frame. The long-term memory perception module stores high-quality feature representations of historical keyframes in a Memory Bank, responsible for stable semantic modeling across time scales. Information from both modules is ultimately fed into a Dynamic Memory Gated Fusion (DMGF) module. DMGF generates dynamic fusion weights based on the perception state of the current search frame, adaptively adjusting the contribution ratio of the two types of memory information in different scenarios. The fused features are used by a prediction head to perform target location regression and classification prediction, and the memory bank is updated based on memory confidence. This invention is suitable for target tracking tasks in dynamic scenarios such as low-altitude flight, rapid target maneuvering, complex backgrounds, and occlusion by UAVs.

[0120] This invention, starting from the biological visual cognitive mechanism, addresses the common problems of insufficient cross-temporal stability and limited rapid adaptability in UAV target tracking in complex dynamic scenarios. It proposes a UAV target tracking method, STL-Track, driven by the collaborative efforts of short-term and long-term memory. Unlike existing methods that use homogenized modeling of historical information, this invention explicitly decouples the functional division between short-term rapid perception and long-term stable semantics at the structural level, introducing a more cognitively reasonable multi-timescale perception paradigm. It has outstanding practical value and engineering application prospects in UAV dynamic tracking and complex environment perception fields such as military reconnaissance, traffic monitoring, and disaster relief.

[0121] On the other hand, the present invention also provides a UAV target tracking system based on biomimetic visual dual-memory collaborative perception, comprising: a feature extraction unit: used to acquire the current search frame and extract the feature map of the current search frame;

[0122] The drone continuously acquires video frame sequences through its onboard monocular imaging device to obtain the current search frame image. At the initial moment of tracking, the drone initializes the target based on the initial position given manually or automatically, and extracts the corresponding initial features as a tracking reference.

[0123] Model building and training unit: used to build and train the dual-memory collaborative perception tracking model; the dual-memory collaborative perception tracking model includes a short-term memory perception module, a long-term memory perception module, a dynamic memory gating fusion module, and a prediction head;

[0124] The short-term memory perception module generates a saliency mask for a given video frame based on its feature map, and uses the saliency mask to obtain short-term memory features. The long-term memory perception module performs a linear mapping on the feature map of a given video frame to generate a query key and a query value; calculates the similarity between the query key and all memory keys stored in a preset long-term memory bank, and calculates long-term memory features based on the similarity, all memory values ​​stored in the long-term memory bank, and the query value. The dynamic memory gating fusion module constructs an initial fusion feature based on the short-term memory features and the long-term memory features, and adjusts the initial fusion feature through a channel gating mechanism and a spatial gating mechanism to obtain the final fusion feature; the final fusion feature is input to the prediction head, and finally the target's position prediction result and confidence level are obtained from the given video frame.

[0125] In the model building and training unit, the Short-Term Memory (SMP) perceptual module can quickly focus on the salient regions of the target in the current search frame, enabling instantaneous responses to instantaneous changes in the target's appearance and rapid movements. At each tracking moment, the feature representation of the current search frame is generated, and this feature representation is then input into the SMP perceptual module. This module, by simulating the foveal mechanism in biological visual systems, focuses on modeling the salient regions in the current search frame. Specifically, the SMP perceptual module generates a salient response map through a saliency mapping function. Based on this saliency response map, the search features of the current search frame are weighted element-wise to obtain the short-term memory features. .

[0126] The Long Memory Perceptual Module (LMP) is used to maintain a stable semantic representation of the target over long time sequences. A learnable long-term memory bank is constructed to store target features from historical keyframes. The long-term memory features are obtained by weighted summation of all memory values ​​in the long-term memory bank. Furthermore, a confidence-driven memory update strategy is adopted, which only writes the corresponding features into the long-term memory when the confidence of the target tracking position prediction result in the current search frame meets the preset conditions (greater than or equal to the preset threshold). If the capacity of the long-term memory reaches the upper limit N, the memory key and memory value with the earliest time or the lowest contribution are removed, thereby avoiding error accumulation.

[0127] The Dynamic Memory Gated Fusion Module (DMGF) adaptively fuses two types of memory features, fully leveraging the complementary advantages of short-term and long-term memory. This involves integrating short-term memory features... Characteristics of long-term memory The data is concatenated and input into a gating weight generation function to obtain fusion weights. Based on these fusion weights, the contribution ratios of short-term memory and long-term memory are adaptively adjusted according to changes in the current scene to obtain the final fusion feature. .

[0128] The result prediction unit is used to input the feature map of the current search frame into the trained dual-memory collaborative sensing tracking model to obtain the position prediction result and confidence of the target in the current search frame.

[0129] This invention includes a feature extraction unit, a model building and training unit, and a result prediction unit. Through a collaborative working mode, it ensures that short-term memory can adapt quickly, long-term memory provides stable semantic compensation, and a dynamic gating fusion mechanism ensures the effective collaboration between the two. This enables the invention to maintain accurate, stable and continuous target representation capabilities in various scenarios.

[0130] It should be noted that, in order to implement the above method embodiments, the specific functions of the UAV target tracking system based on biomimetic visual dual memory collaborative perception provided in this embodiment can be referred to the above method embodiments.

[0131] To make the embodiments of the present invention clearer and to verify the effectiveness of the method and system proposed in the present invention, the present invention also includes the following experiments:

[0132] 1. Experimental Environment

[0133] Hardware specifications: CPU is Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz, memory size is 64GB, GPU is NVIDIA Quadro GV100;

[0134] Software platform: Python version 3.9, CUDA version 12.8, and models are built and trained based on the PyTorch 2.0.1 deep learning framework.

[0135] 2. Experimental Dataset

[0136] To comprehensively evaluate the proposed method, we conducted extensive comparisons with state-of-the-art tracking methods on several standard UAV benchmark datasets, including DTB70, UAV123, UAV123@10fps, UAVTrack112, UAVDT, and VisDrone2018. These datasets cover a wide range of UAV applications, from low-altitude aerial photography and traffic monitoring to fast-moving targets, occlusion, and dense crowd scenarios, as summarized in Table 1. Table 1 represents the UAV tracking benchmark dataset information.

[0137] Table 1. Information on the UAV tracking benchmark dataset

[0138]

[0139] To further demonstrate the effectiveness and robustness of the proposed STLTrack, we conducted a comprehensive evaluation on three challenging benchmark datasets (DTB70, UAV123, and UAV123@10fps). We also compared our method with other datasets including OSTrack. SuperSBT and TCTrack++ Several state-of-the-art (SOTA) trackers were compared.

[0140] like Figure 6 , Figure 7 as well as Figure 8 As shown, STLTrack's overall performance is compared with that of state-of-the-art (SOTA) trackers on the DTB70, UAV123, and UAV123@10fps benchmark datasets. Figure 6 , Figure 7 , Figure 8The diagrams show the precision plot, success plot, and normalized precision plot. As can be seen, STLTrack (red curve) outperforms the other comparative methods across all metrics. Specifically, STLTrack (red curve) consistently outperforms the other comparative methods in all three benchmark tests. On the large-scale UAV123 dataset, our method achieves a 92.9% accuracy score and a 70.7% success rate score, significantly surpassing the second-ranked method.

[0141] 3. Experimental Setup

[0142] During training, the batch size is 32, 60,000 frames are extracted per epoch, and training lasts for 100 epochs. The optimizer is AdamW with weight decay. Initial learning rate The 80th epoch dropped to To comprehensively measure depth estimation performance, the standard OPE index system is used to quantitatively measure tracking performance, including accuracy and success rate as core performance indicators.

[0143] 4. Comparison of Tracking Indicators of Different Models: Several UAV target tracking models in recent years and the model of this invention were selected for detailed comparison. The results are shown in Table 2.

[0144] Table 2 shows the comparison results of the proposed method and advanced tracking methods in terms of Success (%) and Precision (%) in the DTB70, UAVDT, UAV123, VisDrone2018, and UAV123@10fps test sets.

[0145] Table 2. Comparative experimental results of the method of this invention and various advanced tracking methods on multiple test sets.

[0146]

[0147] From the experimental results in Table 2 and Figure 9The visualization results show that the STLTrack method proposed in this invention significantly outperforms existing advanced methods such as AVTrack, ROMTrack, and Aba-ViTrack on multiple UAV target tracking datasets. It exhibits stronger robustness in complex scenarios such as rapid movement, occlusion, and background interference. Furthermore, experiments on multiple UAV target tracking benchmark datasets (DTB70, UAV123, VisDrone2018, etc.) demonstrate that the method significantly outperforms existing advanced methods in core indicators such as accuracy and success rate. It can effectively cope with complex challenges such as rapid movement, occlusion, sudden changes in appearance, and background interference, and has good prospects for engineering applications.

[0148] In summary, this invention, starting from the biological visual cognitive mechanism, addresses the common problems of insufficient cross-temporal stability and limited rapid adaptability in UAV target tracking in complex dynamic scenarios. It proposes a UAV target tracking method, STLTrack, driven by the collaborative efforts of short-term and long-term memory. This invention explicitly decouples the functional division between short-term rapid perception and long-term stable semantics at the structural level, and introduces a more cognitively reasonable multi-temporal scale perception paradigm, enabling the model to maintain accurate, stable, and continuous target representation capabilities under various challenging scenarios.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A UAV target tracking method based on biomimetic visual dual-memory collaborative perception, characterized in that, Include: Get the current search frame and extract its feature map; A dual-memory collaborative perception and tracking model is constructed and trained; the dual-memory collaborative perception and tracking model includes a short-term memory perception module, a long-term memory perception module, a dynamic memory gating fusion module, and a prediction head; The short-term memory perception module generates a saliency mask for a given video frame based on its feature map, and uses the saliency mask to obtain short-term memory features. The long-term memory perception module performs a linear mapping on the feature map of a given video frame to generate a query key and a query value; calculates the similarity between the query key and all memory keys stored in a preset long-term memory bank, and calculates long-term memory features based on the similarity, all memory values ​​stored in the long-term memory bank, and the query value. The dynamic memory gating fusion module constructs an initial fusion feature based on the short-term memory features and the long-term memory features, and adjusts the initial fusion feature through a channel gating mechanism and a spatial gating mechanism to obtain the final fusion feature; the final fusion feature is input to the prediction head, and finally the target's position prediction result and confidence level are obtained from the given video frame. The feature map of the current search frame is input into the trained dual-memory collaborative sensing and tracking model to obtain the position prediction result and confidence of the target in the current search frame.

2. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 1, characterized in that, Generate a saliency mask for a given video frame based on its feature map, specifically including: Global average pooling is used to extract the overall information of the feature map. The overall information is then used to predict multiple sets of scale weights through multi-head mapping. The feature map is then weighted and fused based on the scale weights. The weighted and fused features are then subjected to function activation processing to generate a saliency mask for the current search frame.

3. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 1, characterized in that, The acquisition of short-term memory features using the saliency mask specifically includes: Using the saliency mask, the feature map is divided into salient features and residual features, and the salient features and residual features are fused by depthwise separable convolution to obtain short-term memory features.

4. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 1, characterized in that, Also includes: Updating the long-term memory specifically includes: The feature map is linearly mapped to generate candidate memory keys and memory values. If the confidence result of the location prediction result in the current search frame is higher than a preset threshold, the candidate memory keys and memory values ​​are written into the long-term memory. When the capacity of the long-term memory reaches the upper limit, a time decay strategy is used to remove expired or low-contribution memory keys and memory values.

5. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 4, characterized in that, The feature map is linearly mapped to generate candidate memory keys and memory values, specifically including: The feature map is modulated based on the confidence level of the target location prediction result in the current search frame. The modulated feature map is then processed by a ResNet network and further processed by a module consisting of depthwise separable convolution, channel attention mechanism, spatial attention mechanism, batch normalization and nonlinear activation function to obtain processed features. Candidate memory keys and memory values ​​are generated from the processed features.

6. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 1, characterized in that, The computational process for obtaining long-term memory characteristics specifically includes: A global similarity matrix is ​​obtained by performing a global similarity calculation between the query key and all memory keys stored in the long-term memory: in, Represents the global affinity matrix. The number of channels for the memory keys. This refers to the query key. This represents a memory key in the long-term memory bank; Next, the global affinity matrix is ​​used to perform a weighted summation of all memory values ​​stored in the long-term memory bank to obtain the long-term memory semantic compensation features of the current search frame. The long-term memory semantic compensation features are then concatenated with the query value to obtain the long-term memory features.

7. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 1, characterized in that, The initial fusion features are constructed based on the short-term memory features and the long-term memory features, specifically including: The short-term memory features and the long-term memory features are reshaped into a two-dimensional spatial mapping form through linear mapping, and the initial fusion features are constructed using the two-dimensional spatial mapping form. : in, For lightweight convolutional fusion blocks, This is a characteristic of short-term memory. It is a two-dimensional spatial mapping form.

8. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 1, characterized in that, The method of jointly adjusting the initial fusion features through channel gating and spatial gating mechanisms to obtain the final fusion features specifically includes: The initial fusion features are modulated using channel gating and spatial gating to obtain the final fusion features. : in, For initial fusion features, For channel gating, For spatial gating.

9. The UAV target tracking method based on biomimetic visual dual-memory collaborative perception according to claim 1, characterized in that, The total loss function used in the training process of the dual-memory collaborative sensing tracking model is: in, For weighted focus loss, For GIoU loss, For L1 loss, and This is the regularization parameter.

10. A method system for tracking unmanned aerial vehicles (UAVs) based on biomimetic visual dual-memory collaborative perception, characterized in that, Include: Feature extraction unit: used to acquire the current search frame and extract the feature map of the current search frame; Model building and training unit: used to build and train the dual-memory collaborative perception tracking model; the dual-memory collaborative perception tracking model includes a short-term memory perception module, a long-term memory perception module, a dynamic memory gating fusion module, and a prediction head; The short-term memory perception module generates a saliency mask for a given video frame based on its feature map, and uses the saliency mask to obtain short-term memory features. The long-term memory perception module performs a linear mapping on the feature map of a given video frame to generate a query key and a query value; calculates the similarity between the query key and all memory keys stored in a preset long-term memory bank, and calculates long-term memory features based on the similarity, all memory values ​​stored in the long-term memory bank, and the query value. The dynamic memory gating fusion module constructs an initial fusion feature based on the short-term memory features and the long-term memory features, and adjusts the initial fusion feature through a channel gating mechanism and a spatial gating mechanism to obtain the final fusion feature; the final fusion feature is input to the prediction head, and finally the target's position prediction result and confidence level are obtained from the given video frame. The result prediction unit is used to input the feature map of the current search frame into the trained dual-memory collaborative sensing tracking model to obtain the position prediction result and confidence of the target in the current search frame.

Images