Lightweight single target tracking method and system for unmanned aerial vehicle strapdown visual angle

Abstract

This invention relates to the field of single-target tracking technology, and particularly to a lightweight single-target tracking method and system for UAV strapdown perspective. The invention proposes the following scheme: acquiring the target to be tracked in the initial frame; establishing a global reference template and a dynamic scale template based on the target; for the current frame, determining the search region based on the target position in the previous frame; matching the search region with the global reference template and the dynamic scale template using a lightweight Siamese network to obtain the target position in the current frame; determining the target state corresponding to the target to be tracked based on the target position in the current frame, and performing single-target tracking according to the state. The target state includes at least a normal tracking state, an illumination change state, and an occlusion state. This invention effectively solves the problem of sudden changes in target scale at the end of the attack, greatly reduces model inference latency, and improves tracking robustness in complex scenarios.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to a lightweight single-target tracking method and system for strapdown view of unmanned aerial vehicles (UAVs). Background Technology

[0002] Currently, single-target tracking algorithms based on deep learning Siamese networks are widely used in UAV target detection due to their good balance between accuracy and speed. Existing typical technical solutions usually adopt a strategy of matching the search region with a fixed first-frame template, using ResNet, VGG, etc. as backbone networks, and combining average peak correlation energy ratio (APCE) and color features (CN) for occlusion judgment. When the target is occluded, Kalman filtering is used for kinematic position prediction.

[0003] However, in complex application scenarios where strapdown drones (i.e., the payload camera is rigidly connected to the drone body, without gimbal physical stabilization and attitude compensation) perform close-range reconnaissance or dive-strike missions, the aforementioned existing technologies have significant technical shortcomings. On the one hand, when the drone dives or approaches the target rapidly, the pixel proportion of the target in the image increases exponentially. The receptive field of the fixed template in the first frame, which the existing technology relies on, cannot completely cover the currently magnified target, causing severe shifts and overflows in the target's local high-frequency texture features, leading to irreversible tracking box drift or even complete loss of lock. On the other hand, existing tracking networks have a large number of parameters, computational redundancy, and high inference latency. When deployed on airborne edge computing platforms where the computing power, power consumption, and memory bandwidth of the drone system are strictly limited, the model inference time surges, resulting in a significant drop in the actual running frame rate. Due to the severe inference lag caused by the bloated network, the system cannot provide accurate measurement values ​​for the current frame in real time, causing severe time misalignment in the state update of the Kalman filter. This drastically amplifies the cumulative error of the state transition matrix during the high-maneuverability flight of the strapdown drone, causing the tracker to completely lose the target after the occlusion ends.

[0004] To address the above issues, this application presents a lightweight single-target tracking method and system for strapdown view of unmanned aerial vehicles (UAVs). Summary of the Invention

[0005] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing a lightweight single-target tracking method and system for UAV strapdown perspectives. The method involves acquiring the target to be tracked in the initial frame, establishing a global reference template and a dynamic scale template based on the target, determining the search area based on the target position in the previous frame for the current frame, and matching the search area with the global reference template and the dynamic scale template using a lightweight Siamese network to obtain the target position in the current frame. The target state corresponding to the target to be tracked is determined based on the target position in the current frame, and single-target tracking is performed according to the state. The target state includes at least a normal tracking state, an illumination change state, and an occlusion state. This invention effectively solves the problem of sudden changes in target scale at the end of the attack, greatly reduces model inference latency, and improves tracking robustness in complex scenarios.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A lightweight single-target tracking method for UAVs with strapdown inertial navigation is provided. The UAV includes a processor with a pre-defined lightweight twin network. The method includes: Obtain the target to be tracked in the initial frame, and establish a global reference template and a dynamic scale template based on the target to be tracked; For the current frame, the search region is determined based on the target position of the previous frame. The lightweight Siamese network is used to match the search region with the global reference template and the dynamic scale template to obtain the target position of the current frame. The target state corresponding to the target to be tracked is determined based on the target position in the current frame, and single target tracking is performed based on the state. The target state includes at least normal tracking state, illumination change state, and occlusion state.

[0007] The lightweight twin network is used to encode the global baseline template and dynamic scale template input to the template branch and the search region input to the search branch, respectively, and generate a target location response map based on the encoded template features and search region features. The main structure of the lightweight twin network is a twin backbone network constructed by depthwise separable convolutions, and the twin backbone network includes a template branch and a search branch with shared parameters.

[0008] The global baseline template is constructed in the following ways: In the initial frame, the initial target bounding box of the target to be tracked is determined, the target region corresponding to the initial target bounding box is extracted, and the extracted target features are saved as the global reference template. The initial target bounding box is determined by a target detection algorithm, and the target features are extracted by a feature extraction algorithm based on histogram of oriented gradients.

[0009] The construction methods of the dynamic scale template include: A scale-adaptive region is constructed based on the central region of the initial target box, and features are extracted from the scale-adaptive region to obtain scale features; The scale feature is used as the dynamic scale template; The central region is determined by shrinking the initial target box inward according to a preset ratio.

[0010] The methods for constructing the scale-adaptive region include: The corner features and gray-level centroid of the central region are extracted. The frame descriptor is calculated based on the angle between the corner features and the gray-level centroid. The frame descriptor is transformed into high-dimensional data using an MLP network and added to the gray-level centroid to generate matching features. The region corresponding to the matching features is determined as the scale-adaptive region.

[0011] The methods for determining the target position in the current frame include: The search region is input into the search branch of the lightweight Siamese network for feature encoding to obtain a search region feature map; The global baseline template is input into the template branch of the lightweight Siamese network for feature encoding to obtain a global template feature map. The search region feature map is then matched with the global template feature map to generate a first response map that characterizes the overall position of the target. The dynamic scale template is input into the template branch of the lightweight Siamese network for feature encoding to obtain a dynamic template feature map. The search region feature map is then matched with the dynamic template feature map to generate a second response map that characterizes the local scale changes of the target. The first response map and the second response map are weighted and fused to obtain a joint response map, wherein the weights are determined based on the peak response intensity and response distribution concentration of the first response map and the second response map; The target position of the current frame is determined based on the position corresponding to the maximum response value in the joint response graph, wherein the maximum response value is obtained by peak search calculation on the joint response graph.

[0012] The method for determining the state includes: The response quality index is determined based on the joint response map, and the current target color feature is extracted at the initial position of the target in the current frame. The similarity between the response quality index and the current target color feature relative to the preset color reference feature is jointly determined to obtain the target state corresponding to the current frame. When the response quality index is less than the preset occlusion determination condition and the similarity between the current target color feature and the color reference feature is lower than the preset similarity threshold, it is determined to be an occlusion state.

[0013] The method for determining the response quality index includes: Obtain the maximum response value, minimum response value, and average energy distribution of each response value in the joint response graph; Based on the difference between the maximum and minimum response values ​​and the average energy distribution of the joint response map, a response quality index is calculated to characterize the significance of the response peak and the concentration of the response distribution.

[0014] Performing single-target tracking based on the stated state includes: When the target is in a normal tracking state or an illumination change state, the scale change rate of the target box in the current frame relative to the target box in the previous frame is calculated. When the scale change rate is greater than a preset scale threshold, the target features of the current frame are fused into the dynamic scale template using an exponential moving average algorithm to update the dynamic scale template. The updated dynamic scale template is then used as an observation value and input into a preset Kalman filter for state correction to obtain the final position of the target in the current frame. Tracking is then repeated for subsequent video frames based on the final position of the target in the current frame.

[0015] Performing single-target tracking based on the aforementioned state also includes: When the target is in an occluded state, the dynamic scale template is locked and updates are stopped. Prior state prediction is performed through a preset Kalman filter to output the predicted position of the target in the current frame as the final position of the target in the current frame. Tracking is then repeated for subsequent video frames based on the final position of the target in the current frame.

[0016] Prior state prediction is performed using a pre-defined Kalman filter, including: After determining that the target to be tracked is in an occluded state, the target position of multiple consecutive frames before the occlusion occurred is obtained; The position change trend of the target to be tracked is determined based on the target position in the consecutive frames, and the target motion state is constructed based on the position change trend, wherein the target motion state includes at least one of the target position, motion speed and motion direction; Based on the target motion state and combined with the global reference template, the Kalman filter is used to perform prior state prediction and output the predicted position.

[0017] Based on the final position of the target in the current frame, the tracking is repeated in subsequent video frames, including: The final position of the target in the current frame is used as the center position of the search area in subsequent frames to construct an extended search area; The extended search region is feature-encoded using the lightweight Siamese network, and the encoded extended search region features are matched with the global baseline template to obtain multiple candidate response positions. Based on the location of each candidate response, a corresponding candidate region is determined, and candidate features of each candidate region are extracted. Each candidate feature is compared with the baseline feature corresponding to the dynamic scale template to obtain the consistency evaluation result corresponding to each candidate region. Based on the consistency evaluation results corresponding to each candidate region, the search region is determined from the plurality of candidate response locations.

[0018] A lightweight single-target tracking system for strapdown drones, the system comprising: The template construction module is used to obtain the target to be tracked in the initial frame and to build a global reference template and a dynamic scale template based on the target to be tracked. The search matching module is used to determine the search area based on the target position of the previous frame for the current frame, and to match the search area with the global reference template and the dynamic scale template through a preset lightweight twin network to obtain the target position of the current frame. The state determination module is used to determine the state of the target to be tracked based on the target position in the current frame. The tracking execution module is used to perform single-target tracking based on the stated state.

[0019] Compared with the prior art, the beneficial effects of the present invention are: This invention employs depthwise separable convolutions to construct a lightweight backbone network and optimizes it for edge hardware, compressing single-frame inference latency to an extremely low level. This provides dense, lag-free real-time observation data for the Kalman filter, fundamentally eliminating the prediction divergence problem caused by observation lag and improving the recovery success rate of the classic Kalman filter under strapdown high-maneuverability perspectives. Attached Figure Description

[0020] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 An exemplary application scenario diagram provided for an embodiment of the present invention; Figure 2 A flowchart illustrating a lightweight single-target tracking method for UAV strapdown perspective provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the processing procedure of the lightweight twin network provided in an embodiment of this application.

[0021] Explanation of reference numerals in the attached figures: 100, UAV; 101, airborne imaging device; 102, object to be tracked. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0023] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0024] The lightweight single-target tracking method proposed in this application for strapdown view of UAVs is mainly applicable to continuous target tracking scenarios under conditions of high-speed maneuvering, continuous attitude changes, and tight coupling between the payload's line of sight and the aircraft's attitude. Unlike tracking tasks where the target's appearance changes relatively smoothly under gimbal-stabilized view, in strapdown view imaging mode, the imaging field of view changes directly with the UAV's attitude. The target's displacement, scale, orientation, and local saliency in the image are rapidly coupled and changed with flight trajectory, pitch angle changes, and platform vibration. Especially when UAVs perform close-in reconnaissance, escort flight, dive approach, low-altitude sweeping, and continuous turning flight missions in narrow airspace, the target often exhibits a combination of phenomena such as sudden magnification, local distortion, edge occlusion, and instantaneous weak texture within a short time series. This makes it difficult for traditional single-target tracking methods based on the assumption of a stable view to maintain continuous, stable, and recoverable tracking results.

[0025] This application is particularly suitable for the following types of application environments: scenarios where the target is located on the ground or in low-altitude open environments, and the UAV needs to directly utilize airborne imaging equipment to complete target locking and continuous tracking tasks without independent stable gimbal compensation or with weak compensation; scenarios where the target exhibits significant scale abrupt changes during imaging, manifested as a rapid transition from a small target to a medium-to-large target in the image as the aircraft rapidly approaches; scenarios where there are complex background textures, similar interfering objects, or local occlusion sources around the target's trajectory, causing the target to retain only incomplete appearance information within several consecutive frames; and scenarios where there are strong constraints on airborne processing latency, power consumption budget, and edge deployment resources, requiring real-time tracking and decision-making on a limited computing platform.

[0026] The selection of the aforementioned application scenarios does not depend on the fixed form of the target category itself, nor is it limited by the premise that a specific target must belong to a certain category such as vehicle, ship, personnel, or facility. Instead, it is determined based on the imaging characteristics common to the mission scenario. In other words, as long as the target tracking process is simultaneously affected by factors such as rapid changes in the field of view caused by strapdown perspective, nonlinear magnification of the target scale, local occlusion disturbances, and limited airborne computing power, the technical solutions proposed in this application are applicable.

[0027] In such scenarios, those skilled in the art typically employ two approaches to address the problem: First, they use conventional neural networks, correlation filtering, or integrated detection and tracking methods to directly output the target position using the matching results between the current frame and the template frame. Second, they overlay a motion prediction model into the tracking link, such as smoothing the trajectory using a Kalman filter or other state estimation methods, to mitigate the instability caused by short-term target loss or image jitter. The former approach can achieve certain results under general viewpoint changes or slow scale changes, but under strapdown viewpoints, since the target's projection scale relative to the camera no longer follows a gradual evolution pattern, fixed templates or single-updated templates are prone to apparent mismatch when the target rapidly approaches. Especially when the target rapidly transitions from a distant, small-scale state to a close-range, large-scale state, the original template's ability to represent the target structure will significantly decrease, leading to a shift in the main peak of the response map, local texture mismatch, or even tracking drift. Although the latter type of method introduces motion continuity constraints, if the front-end visual observation itself has a large delay or is susceptible to occlusion interference, the observations on which the state estimation depends will have temporal misalignment or reduced reliability. In this case, even if conventional prediction mechanisms are used, only formal position extrapolation results can be obtained, and it is difficult to truly play a role in target recovery after occlusion and subsequent frame stable recapture.

[0028] Furthermore, in the actual deployment of UAV edge platforms, those skilled in the art face another often underestimated constraint: the computational cost of front-end visual feature extraction and matching cannot be discussed in isolation from airborne real-time requirements. Many existing high-precision tracking networks are designed in environments with ample computing power or servers, characterized by deep networks, large parameter sets, and long inference links. While they provide strong feature representation capabilities under laboratory conditions, migrating them to airborne processors, embedded AI chips, or low-power heterogeneous computing platforms often results in significant inference lag. For strapdown-based perspective tracking tasks, this lag not only means a decrease in frame rate but also further amplifies the discrepancy between the visual observation output and the actual target state, making subsequent motion predictions based on lagging observations and thus weakening the effectiveness of trajectory extrapolation. In other words, in such scenarios, lightweight design is not simply a choice for ease of engineering implementation but is directly related to the accuracy of target state estimation, trajectory continuity during occlusion, and subsequent recovery tracking capabilities.

[0029] Based on the above understanding, this application does not simply reduce the problem to how to improve tracking accuracy or how to enhance occlusion robustness. Instead, it further recognizes that, from a strapdown perspective, the key constraint on tracking stability is not the performance limit of a single module, but whether a coordinated closed loop is formed between the template representation method, the timeliness of front-end observation, and the position maintenance and reacquisition logic under occlusion conditions. On the one hand, if only the template of the first frame is retained, although the stability of the initial appearance reference can be maintained, when the target undergoes significant scale expansion during flight approach, the initial template is difficult to continuously reflect the true changes in the target's local structure. On the other hand, if the template is updated indiscriminately online, non-target information is easily introduced into the template under local occlusion, viewpoint perturbation, or background intrusion, resulting in template contamination and thus worsening the subsequent matching results. Furthermore, even if the predicted position is output through a conventional Kalman filter when the target is occluded, if the predicted result is only used for position compensation in the current frame and cannot be further transformed into effective search constraints in subsequent frames, the reacquisition of the target after occlusion recovery may still fail.

[0030] Specifically, the basic premise of this application is as follows: Addressing the characteristic that the target appearance evolves rapidly with distance under strapdown conditions, but the initial structural reference still has long-term value, the target template is divided into a global reference template for providing stable appearance anchoring and a dynamic scale template for adapting to changes in target scale. Considering that observation delay under airborne real-time deployment conditions can negatively impact state prediction performance, a lightweight twin network is introduced as the front-end feature encoding and matching carrier, enabling target position observations to be output with lower latency and higher temporal consistency. To address the issue that relying solely on the matching results of the current frame in occlusion states can easily lead to misjudgments, the target state is jointly discriminated by combining response quality and target color consistency to distinguish between normal tracking, illumination disturbances, and genuine occlusion. Once an occlusion state is confirmed, the dynamic scale template is no longer updated. Instead, the motion state is constructed based on the target position change trend over several consecutive frames before the occlusion occurs. Prior prediction maintains the continuity of the target trajectory, and the prediction results are used to guide candidate region searches in subsequent frames. The global reference template is then used to perform consistency verification on the candidate regions, thereby improving the reliability of re-acquisition after occlusion recovery.

[0031] It should be noted that the lightweight single-target tracking method for UAV strapdown perspective described in this application does not presuppose that the UAV must be equipped with a long-stroke mechanically stabilized gimbal, nor does it rely on the target having complete boundaries, strong texture contours, or a fixed motion model as an applicable condition. It is more suitable to be understood as a method framework for airborne visual tracking scenarios where the viewpoint is linked to the platform's attitude, the target's appearance changes rapidly, and computational resources are limited. For scenarios where the external viewpoint changes gradually, the target scale is stable over a long period, and there is no obvious risk of occlusion, the multi-template collaboration and occlusion recovery mechanism of this application may not constitute a decisive advantage; however, for the tracking instability problem caused by the combined effects of attitude changes, distance changes, local occlusion, and edge computing constraints under strapdown perspective, this application can, while maintaining airborne deployability, more reasonably explain and handle the common and interconnected technical dilemmas in the entire engineering process: why the target drifts, why the template fails, and why it is difficult to recover after occlusion.

[0032] refer to Figure 1 , Figure 1 This is an exemplary application scenario diagram provided for an embodiment of this application.

[0033] like Figure 1 As shown, the application scenario includes a drone 100, an airborne imaging device 101 mounted on the drone 100, and a target 102 to be tracked located within the field of view of the airborne imaging device 101. The airborne imaging device 101 is used to acquire image information of the target 102 to be tracked and send the acquired image data to the processing unit of the drone 100 to perform single-target tracking processing in this embodiment of the application.

[0034] In one optional embodiment, the UAV of this application is configured with a processor for executing the method of this application. It is understood that embodiments of this application also provide a lightweight single-target tracking system for a strapdown view of a UAV, the system being configured as program instructions within the processor, the system comprising: The template construction module is used to obtain the target to be tracked in the initial frame and to build a global reference template and a dynamic scale template based on the target to be tracked. The search matching module is used to determine the search area based on the target position of the previous frame for the current frame, and to match the search area with the global reference template and the dynamic scale template through a preset lightweight twin network to obtain the target position of the current frame. The state determination module is used to determine the state of the target to be tracked based on the target position in the current frame. The tracking execution module is used to perform single-target tracking based on the stated state.

[0035] Next, with reference to the accompanying drawings, the lightweight single-target tracking method for UAV strapdown view provided in the embodiments of this application will be further described. Figure 2 The method shown is applied to a drone equipped with strapdown inertial navigation, the drone including a processor with a preset lightweight twin network configured within the processor, the method comprising: S1: Obtain the target to be tracked in the initial frame, and establish a global reference template and a dynamic scale template based on the target to be tracked; In this embodiment, the target to be tracked in the initial frame can be determined through target detection results, manual initialization results, or target indication information transmitted from upstream of the task. As long as a target bounding box representing the initial position of the target can be obtained, this application does not impose any strict limitations. After the target to be tracked is determined, features are extracted from the initial target region to form a global baseline template and a dynamic scale template. The global baseline template is used to store the overall appearance reference of the target at the initial moment, while the dynamic scale template uses the features of the target's central region or a scale-constrained local region as the initial template to represent the local scale evolution of the target as distance changes during subsequent tracking.

[0036] It is understandable that by establishing a global baseline template and a dynamic scale template simultaneously in the initial stage, a stable and complete target appearance reference can be preserved on the one hand, and a representation basis can be reserved for subsequent scale adaptive updates on the other hand. This enables the subsequent matching process to have both anti-drift capability and adaptability to local scale changes of the target.

[0037] Those skilled in the art will understand that the feature form used in the initial target region can be selected according to the imaging conditions, processor resources and subsequent matching strategies. Specifically, it can be gradient features, texture features, local structural features or other image features that can meet the target representation requirements. This application does not impose any further limitations.

[0038] S2: For the current frame, determine the search region based on the target position of the previous frame, and match the search region with the global reference template and the dynamic scale template through the lightweight Siamese network to obtain the target position of the current frame; In this embodiment, the search region for the current frame is not blindly generated across the entire image. Instead, it is cropped locally, centered on the target position of the previous frame and using a preset search scale. This reduces interference from irrelevant backgrounds in the subsequent matching process while also meeting the real-time requirements of airborne processing. After the search region is determined, it is input into the search branch of the lightweight Siamese network for feature encoding. Simultaneously, the global baseline template and dynamic scale template are input into the template branch for feature encoding, yielding search region features, global template features, and dynamic template features, respectively. Subsequently, relevant matching is used to obtain response results corresponding to the overall target position and the local scale changes of the target, respectively. Based on these response results, the target position in the current frame is determined. The lightweight Siamese network employs a lightweight backbone structure rather than a highly complex feature extraction network. The purpose is not simply to reduce the number of parameters, but to reduce the latency of the front-end observation output, making the target position in the current frame closer to the target state at the actual imaging moment. This provides input with better temporal consistency for subsequent state determination and occlusion processing. Especially in UAV strapdown scenarios, the flight platform's attitude and field of view change synchronously. The displacement and scale changes of the target between adjacent frames are usually greater than the changes under conventional image-stabilized perspective conditions. If there is a significant delay in the front-end matching process, it can easily amplify the time deviation between the observation and the actual state, thereby affecting the effectiveness of subsequent position correction and state prediction. Using a global reference template and a dynamic scale template to participate in the matching process is beneficial to simultaneously consider the overall structural stability of the target and the adaptability to local scale changes. This ensures that the target position in the current frame is no longer dependent on a single apparent reference, thereby reducing the drift probability caused by a single template.

[0039] S3: Determine the target state corresponding to the target to be tracked based on the target position in the current frame, and perform single target tracking based on the state. The target state includes at least normal tracking state, illumination change state, and occlusion state. In this embodiment, the determination of the target state is not based solely on a rough judgment of a single matching peak at the current position, but rather on a comprehensive identification that combines the quality changes of the joint response results with the consistency between the current target area and the baseline appearance, in order to distinguish between three different situations: normal tracking, appearance disturbance, and real occlusion.

[0040] Understandably, target response attenuation under strapdown perspective does not necessarily mean target loss. Changes in flight viewpoint, downward view disturbances, and local shadows can all cause changes in response peak. If different sources of instability are not distinguished, occlusion processing can be mistakenly triggered under normal disturbance conditions, or the dynamic scale template can continue to be updated when actual occlusion occurs, leading to template contamination. By performing hierarchical identification of target states, under normal tracking conditions or illumination changes, the current observation results can continue to be used to maintain template adaptation and position correction. Under occlusion conditions, dynamic scale template updates can be stopped in time, and the process can switch to prediction and subsequent reacquisition processing based on historical motion states. This avoids writing occlusion features into the template and improves tracking continuity and reacquisition accuracy after occlusion recovery.

[0041] In some optional implementations, when the target is determined to be in a normal tracking state or an illumination change state, the dynamic scale template can be updated based on the scale change rate of the target box in the current frame relative to the target box in the previous frame, and the final position of the target in the current frame can be output in combination with the state correction result. When the target is determined to be in an occluded state, the dynamic scale template is stopped from being updated, and the target motion state is constructed based on the target position change trend of multiple consecutive frames before the occlusion occurs. The continuity of the current position is maintained by prior prediction, and candidate regions are constructed around the predicted position in subsequent frames. The consistency verification of the candidate regions is performed by the global reference template to achieve target re-capture.

[0042] Before delving into the specific technical details of the steps, the embodiments of this application need to be emphasized again.

[0043] For single-target tracking from an airborne strapdown perspective, changes in the target in the image are not simply scale magnification or positional shifts from a single source, but rather a complex evolutionary process influenced by multiple apparent and temporal factors. Differences in the appearance of the same target across consecutive frames can arise from projection magnification caused by a rapid decrease in the distance between the UAV and the target, observation angle shifts caused by changes in flight attitude, and local response instability due to partial occlusion, instantaneous illumination changes, and background intrusion. Treating these changes merely as template mismatch or short-term detection fluctuations often fails to explain why the target position can be roughly estimated in certain scenarios but then rapidly drifts away, or why some tracking results, seemingly reasonable in a single frame, cannot maintain stability over time. Based on this understanding, this embodiment does not interpret the tracking process as a simple search for the optimal matching position in a single frame, but rather as a continuous judgment process that gradually unfolds around the stability of the target representation, its evolutionary capability, and state transition boundaries.

[0044] Furthermore, this embodiment focuses not on whether the target changes in the image itself, but rather on the distinguishability of different changes in a processing sense. In actual airborne tracking, target scale expansion does not necessarily mean tracking failure, local response weakening does not necessarily correspond to actual occlusion, and short-term appearance shifts are not necessarily sufficient to trigger template replacement. For those skilled in the art, what truly affects tracking continuity is usually not whether a deviation occurs at a certain moment, but whether the deviation is a normal change that can be absorbed by appearance evolution, or whether it has entered an unstable state where it can no longer be updated based on current observations. This embodiment introduces a hierarchical understanding of target change types in the overall processing path, treating apparent fluctuations, scale expansion, and actual occlusion of the target during continuous imaging as different types of processing objects, and assigning different template usage methods and position update methods accordingly.

[0045] It should be noted that the reason for simultaneously setting a global reference template and a dynamic scale template in this embodiment is not simply to add up the number of templates, but rather based on the understanding that the stable and evolving parts of the target appearance information have different roles in temporal processing. For continuous tracking tasks, the initial appearance of the target usually contains relatively complete overall structural information. This information still has reference value over a long period of time, especially when there is local occlusion, local brightness fluctuations, or short-term spurious peak interference. The complete appearance reference can provide a stable basis for whether the target has deviated substantially. At the same time, as the flight platform gradually approaches the target, the local details and resolvable structures of the target in the image will continue to enhance. If only the initial overall template is used as the sole reference, the matching response may be weakened due to scale mismatch. Based on this, this embodiment divides the template into a global reference template for maintaining the overall structural anchoring relationship and a dynamic scale template for absorbing local scale evolution information. The former mainly plays the role of long-term stable reference, while the latter mainly plays the role of short-term adaptive update. The degree of participation of the two is not fixed in different states, but plays different roles as the continuous imaging conditions of the target change.

[0046] Next, we will further elaborate on the technical content of the lightweight twin network method in this application.

[0047] refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the processing procedure of the lightweight twin network provided in an embodiment of this application.

[0048] In one example, the lightweight Siamese network is used to encode features of the global baseline template and dynamic scale template input to the template branch, and the search region input to the search branch, respectively, and generates a target location response map based on the encoded template features and search region features. The main structure of the lightweight Siamese network is a Siamese backbone network constructed from depthwise separable convolutions, which includes a template branch and a search branch with shared parameters. Specifically, the template branch receives template image blocks corresponding to the global baseline template and the dynamic scale template, and the search branch receives search region image blocks cropped from the neighborhood of the target location in the previous frame in the current frame. Both the template image blocks and the search region image blocks can undergo uniform scale normalization, pixel value normalization, and channel rearrangement processing before being input into the network to reduce the impact of brightness fluctuations and size differences between different frames on subsequent encoding results. For template image blocks, a global template input tensor and a dynamic template input tensor can be formed according to the template type; for search region image blocks, a search input tensor is formed. The input tensors mentioned above can all be in the form of three-channel image tensors, or they can be in the form of grayscale image tensors, multispectral tensors, or preprocessed feature tensors, depending on the actual sensor conditions, as long as they can meet the requirements of subsequent feature encoding and related matching processing. This application does not impose any further restrictions on this.

[0049] In the feature encoding stage, the template branch and the search branch adopt the same backbone extraction structure and share network parameters to ensure that the template features and search features are in the same feature space. The backbone extraction structure includes at least an input convolutional layer, several levels of lightweight feature extraction units, and an output feature integration layer. The input convolutional layer is used to perform initial downsampling and shallow texture encoding on the input image blocks, using convolution operations with a stride greater than or equal to 1, combined with a normalization layer and a non-linear activation layer. The lightweight feature extraction unit is composed of multiple depthwise separable convolutional modules cascaded together. Each depthwise separable convolutional module includes at least a channel-wise convolutional layer and a pointwise convolutional layer. The channel-wise convolutional layer performs spatial convolution operations in each input channel to extract spatial information such as local edges, corners, and texture distribution. The pointwise convolutional layer performs cross-channel linear combination of the channel-wise convolution outputs to establish semantic relationships between channels and compress redundant features. To improve the deployment efficiency of the network on the airborne processor, in some embodiments, the lightweight feature extraction units at each level may further include a batch normalization layer and an activation layer. The activation layer may be ReLU, ReLU6, Leaky ReLU, Hard-Swish or other nonlinear mapping functions suitable for lightweight inference, which are not limited in this application.

[0050] Furthermore, considering that the template image patch and the search region image patch are usually different in spatial size, the template feature map output by the template branch and the search feature map output by the search branch can maintain a preset ratio in resolution. The template branch performs independent forward propagation on the global baseline template and the dynamic scale template, thereby outputting a global template feature map and a dynamic template feature map respectively; the search branch performs forward propagation on the search region image patch once and outputs a search region feature map. To reduce redundant computation, in one embodiment, the search branch encodes the search region image patch only once, and then matches it with the global template feature map and the dynamic template feature map respectively, thereby generating two response results. This allows for the simultaneous use of the global template and the dynamic template in position estimation without repeatedly extracting search region features, which is beneficial for reducing computational load and maintaining the temporal consistency of the two response results under the same search region.

[0051] In one example, the lightweight Siamese network is pre-trained offline before being deployed to the processor, and the network parameters for online tracking are obtained after training. The goal is to ensure a stable correspondence between the template features output by the template branch and the search region features output by the search branch within the same feature space. This allows the network to maintain a high response to the true target location and a low response to non-target regions even under conditions of target scale changes, local appearance changes, and background interference. To meet the requirements of rapid target appearance changes under strapdown perspective, the samples used in the training phase are not limited to target image pairs from a static perspective, but can also include temporal sample pairs composed of consecutive video frames. This allows the network to be exposed to image changes such as target magnification from far to near, viewpoint shifts, local occlusion, illumination changes, and background disturbances during the training phase, thereby improving its adaptability during subsequent online tracking.

[0052] Specifically, training samples can consist of multiple target tracking datasets, airborne imaging sequences, or time-series image sequences generated through simulation. Each set of training samples includes at least a template image patch, a search region image patch, and location annotation information corresponding to the search region image patch. The template image patch can be cropped from the frame containing the target according to the labeled bounding box, and is used to represent the appearance of the target. The search region image patch can be cropped from the same frame or adjacent frames as the template image patch, centered on the real location of the target, according to a preset expansion ratio, and is used to construct the region to be matched. For training samples of the global baseline template, the complete target region or a target region containing a small amount of contextual background can be used as the template input. For training samples of the dynamic scale template, the target center region, scale-normalized region, or a local region cropped by a preset shrinkage ratio can be used as the template input, so that the network learns the feature representation mode corresponding to the local scale change of the target during the training phase. The search region image patch is usually larger than the template image patch to ensure that it contains both the real target location and a certain range of background and negative sample regions, so that the network can establish the response difference between the target region and non-target regions during the training process.

[0053] During sample construction, data augmentation processing can be further performed on the original samples to simulate common imaging perturbations under the strapdown view of UAVs. The data augmentation processing can include at least one or more of the following: target scale scaling, translational perturbation, rotational perturbation, brightness variation, contrast variation, blurring perturbation, local occlusion perturbation, and background replacement perturbation. For target scale scaling, different scale ratios can be artificially constructed between the template image patch and the search region image patch to enhance the network's ability to model scale changes caused by rapid approach or departure of the target. For local occlusion perturbation, occlusion blocks, random texture blocks, or background fragments can be superimposed on the target's local area in the search region image patch to enhance the network's robustness to the real location response under conditions of partial target information loss. For brightness and contrast variations, the apparent differences caused by changes in illumination, shadows, and sensor exposure fluctuations during flight can be simulated. For translational and rotational perturbations, the changes in target position and orientation caused by changes in flight attitude under the strapdown view can be simulated. Those skilled in the art will understand that the specific combination, magnitude, and trigger probability of the enhancement methods can be set according to the dataset size and task scenario, as long as they can minimally cover the main appearance changes that the target may undergo during continuous airborne imaging. This application does not impose any further limitations.

[0054] Regarding the construction of the supervision signal, the training objective of the lightweight Siamese network can be established through response map supervision. Specifically, for each set of training samples, a supervision label map is generated in the image patch of the search region based on the true location of the target. The supervision label map can be a two-dimensional label distribution with the true target center as the peak center. The two-dimensional label distribution can adopt Gaussian distribution labels, center decay labels, or other continuous supervision label forms that can represent high response near the true location and low response away from the true location. Among them, the region corresponding to the true center location of the target in the label map has a higher label value, and the location far from the true center has a lower label value, thereby enabling the network to learn the mapping relationship of producing a peak response to the true target location and producing a low response to the background region and the error location. For the case of training with a dual-template structure, supervision labels corresponding to the global template matching result and the dynamic template matching result can be generated separately, or supervision can be uniformly applied to the fusion result of the two response maps, as long as the global template branch learns the overall structure localization ability and the dynamic template branch learns the local scale change adaptation ability.

[0055] Regarding the loss function setting, the training loss of the lightweight Siamese network includes at least a response map matching loss. The difference between the network's output response map and the supervised label map can be used as the basic loss term, for example, using pixel-wise mean squared error loss, binary cross-entropy loss, focus loss, or other methods that can measure the relationship between the response map and the label. Figure 1 Consistency loss forms. For a dual-template matching structure, in one embodiment, the first matching loss between the first response map and the corresponding label map, and the second matching loss between the second response map and the corresponding label map can be calculated separately, and the two can be weighted and summed according to preset weights to obtain the total matching loss; in another optional specific implementation, the first response map and the second response map can be fused first, and then the loss of the fused joint response map and the label map can be calculated. To improve the network's discrimination ability under complex background and target scale variation conditions, positive and negative sample discrimination loss, feature consistency loss, or template stability constraint loss can be further introduced. For example, the position close to the real target center in the search area can be defined as a positive sample position, and the region far from the target center can be defined as a negative sample position. By increasing the response difference between positive and negative sample positions, the network's ability to suppress background interference can be improved; for the same target image patch from adjacent frames, feature consistency constraints can be used to make its encoding results maintain a high similarity in the feature space, so as to enhance the network's representation stability of time-continuous targets.

[0056] In a further example, considering that the dynamic scale template needs to represent local scale changes of the target in the subsequent online phase, scale adaptation constraints can be imposed on the dynamic template branch during the training phase. Specifically, different scale local regions of the target can be artificially constructed in the training samples as inputs to the dynamic template, and matched with a search region containing the real target location. The supervised network learns the response correspondence between local templates of different scales and search features. In this way, during subsequent online tracking, when the target scales up in the image due to the rapid approach of the drone, the dynamic template branch can better maintain its sensitivity to changes in local details. Correspondingly, for the global baseline template branch, more emphasis is placed on the stable learning of the overall outline and initial appearance anchor of the target, so that it can still provide a relatively robust overall matching reference when similar targets appear in the background, when local regions are temporarily distorted, or when occlusion is restored. By assigning different sample organization methods and supervision focuses to the two types of template branches during the training phase, the dual-template mechanism can form a clearer division of labor in the online phase.

[0057] In terms of parameter optimization, the lightweight Siamese network can update network parameters using an iterative training method based on gradient backpropagation. Stochastic gradient descent, dynamic stochastic gradient descent, adaptive moment estimation, or other parameter optimization methods suitable for deep network training can be employed. During training, a batch of training samples is sequentially input into the template branch and the search branch, the network output response map and corresponding loss value are calculated, and then backpropagation and gradient updates are performed on the network parameters based on the loss value until the preset training epochs, loss convergence conditions, or validation set performance requirements are met. To prevent overfitting, gradient oscillations, or convergence instability during training, learning rate decay strategies, weight decay terms, gradient pruning strategies, or early stopping strategies can be further implemented. For example, a larger learning rate can be used in the early stages of training to accelerate convergence, and the learning rate can be gradually reduced in the later stages of training to stabilize parameter updates; another example is applying weight decay to the convolutional layer parameters to limit model complexity and improve generalization ability. Those skilled in the art will understand that the optimizer type, initial learning rate, batch size, training epochs, and learning rate decay method can be adjusted according to data scale, hardware resources, and accuracy requirements. This application does not impose further limitations in this regard. For example, the training optimizer for the lightweight Siamese network uses the AdamW optimizer, and the initial learning rate is set to... Batch size is set to 32, training epochs are set to 120, and weight decay factor is set to... During training, the learning rate can be decayed using a segmented decay strategy, that is, the learning rate is decayed to a certain value at the 60th epoch. It further decayed to [a certain value] in the 100th round. This approach aims to balance convergence speed in the early stages of training with parameter stability in the later stages. Alternatively, the optimizer can also employ a momentum-driven stochastic gradient descent optimizer, where the momentum factor can be set to 0.9 and the initial learning rate can be set to... The batch size can be set from 16 to 64, the number of training rounds can be set from 80 to 200, and the learning rate decay method can be cosine annealing decay, exponential decay, or step decay according to a preset number of rounds.

[0058] It should be noted that the template image patches, search region image patches, supervision label maps, and loss term settings used in the training phase are not required to be completely consistent across all implementations. Those skilled in the art can adaptively adjust the training data source, template cropping ratio, search region scale, enhancement strategy, supervision method, and loss function combination based on the target type, camera resolution, flight altitude, target motion pattern, and processor resource conditions of the actual airborne mission scenario. The fundamental requirement is that the trained lightweight Siamese network can encode the template image patches and search region image patches into a unified feature space, and generate a significant response to the real target location within the search region, while exhibiting a suppressive response to non-target regions. This provides a reliable front-end observation foundation for subsequent online dual-template matching, state determination, and occlusion handling.

[0059] In one example, the global benchmark template is constructed in the following ways: In the initial frame, the initial target bounding box of the target to be tracked is determined, the target region corresponding to the initial target bounding box is extracted, and the extracted target features are saved as the global reference template. The initial target bounding box is determined by a target detection algorithm, and the target features are extracted by a feature extraction algorithm based on histogram of oriented gradients.

[0060] The construction methods of the dynamic scale template include: A scale-adaptive region is constructed based on the central region of the initial target box, and features are extracted from the scale-adaptive region to obtain scale features; The scale feature is used as the dynamic scale template; The central region is determined by shrinking the initial target box inward according to a preset ratio.

[0061] It is understood that the construction processes of the aforementioned global baseline template and dynamic scale template can both be implemented using existing image target initialization and feature extraction methods in this field. For the global baseline template, in the initial frame, the initial target bounding box of the target to be tracked can first be determined using a target detection algorithm. This target detection algorithm can be a candidate region-based detection algorithm, a regression-based detection algorithm, or other detection methods capable of outputting target bounding boxes; this application does not limit this. After obtaining the initial target bounding box, the target region corresponding to the initial target bounding box is cropped, and feature extraction processing based on histogram of oriented gradients is performed on the cropped target region. Specifically, the target area can first be grayscaled or normalized in brightness, then the gradient magnitude and gradient direction in the neighborhood of each pixel can be calculated, and the gradient direction can be quantized and statistically analyzed according to a preset angle range to form a gradient direction histogram corresponding to each local unit. The preset angle range can be 0° to 180° or 0° to 360°. Subsequently, the gradient direction histograms of multiple local units are spliced ​​in a preset order to obtain target features that characterize the overall outline and texture distribution of the target. The target features are then saved as a global reference template. The preset order can be sorted in descending order according to the gradient magnitude of the gradient direction histogram.

[0062] For a dynamic scaling template, after the initial target box is determined, its geometric center can be used as a reference to shrink the initial target box inward by a preset shrinkage ratio to obtain a central region located at the center of the target. The preset shrinkage ratio can be set according to the target size, image resolution, or empirical parameters, as long as it allows the central region to retain the main target area relative to the initial target box and reduce edge background interference; this application does not impose further limitations on this. After obtaining the central region, a scale-adaptive region can be constructed based on the central region, and features can be extracted from the scale-adaptive region to obtain scale features. The scale-adaptive region can directly use the central region itself, or it can undergo maximum normalization processing based on the central region to adapt to subsequent scale change scenarios. The extraction method for the scale features can be the same as the feature extraction method corresponding to the global baseline template, or other feature extraction methods suitable for characterizing local structure and scale changes can be used, as long as they can reflect the local appearance information of the target's central region; in one example, the scale features can also be obtained using a feature extraction algorithm based on histogram of oriented gradients, and these scale features can be used as a dynamic scaling template.

[0063] It should be noted that if the target size is small and edge information has a significant impact on target discrimination, the preset shrinkage ratio can be appropriately reduced, for example, set to 10% to 15%, to avoid excessive shrinkage leading to loss of effective target information; if the target size is large and edge background interference is strong, the preset shrinkage ratio can be appropriately increased, for example, set to 20% to 30%, to enhance the central region's ability to focus on the target subject.

[0064] In one example, the scale-adaptive region is constructed in the following ways: The corner features and gray-level centroid of the central region are extracted. The frame descriptor is calculated based on the angle between the corner features and the gray-level centroid. The frame descriptor is transformed into high-dimensional data using an MLP network and added to the gray-level centroid to generate matching features. The region corresponding to the matching features is determined as the scale-adaptive region.

[0065] Specifically, after obtaining the central region of the initial target bounding box, the central region can first be converted to grayscale to obtain a grayscale image of the central region. Subsequently, multiple corner features are extracted from the grayscale image. These corner features can be obtained through Harris corner detection, Shi-Tomasi corner detection, FAST corner detection, or other corner extraction methods that can characterize the location of local structural abrupt changes; this application does not limit this method. The corner features at least include corner position coordinates, and may further include corner response intensity, local orientation information, or local neighborhood texture statistics when necessary.

[0066] While extracting corner features, the grayscale centroid of the central region can be calculated. Specifically, the grayscale values ​​of each pixel within the central region can be used as weights, and a weighted average can be performed on the horizontal and vertical coordinates of each pixel to obtain the two-dimensional coordinate position of the grayscale centroid. The grayscale centroid is used to characterize the main aggregation position of grayscale distribution within the central region. When the target has a clear main structure, the grayscale centroid can usually reflect the main energy distribution position of the target's central region relatively stably, thus providing a reference benchmark for establishing the subsequent spatial distribution relationship of corner points.

[0067] After obtaining the corner features and gray-level centroids, the relative positional relationship between each corner and the gray-level centroid can be calculated. Specifically, for any corner, the direction of the line connecting the corner to the gray-level centroid can be calculated based on the coordinate difference between the corner coordinates and the gray-level centroid coordinates; further, the angle between this line direction and a preset reference direction can be used as the angular representation of the corner. The preset reference direction can be the horizontal direction, vertical direction, or other preset reference coordinate axis direction of the image, and this application does not impose further limitations on this. Based on the angle information corresponding to multiple corners, a frame descriptor for representing the spatial distribution relationship of the local structure in the central region can be constructed. The frame descriptor can include not only the angle values ​​between each corner and the gray-level centroid, but also the distance values ​​from the corner to the gray-level centroid, the corner response intensity ranking information, or the gradient statistics within the local neighborhood, so that the frame descriptor simultaneously possesses directional distribution features and spatial hierarchy features.

[0068] After the frame descriptor is formed, it is input into an MLP network for feature mapping. The MLP network may include an input layer, at least one hidden layer, and an output layer. These layers are connected via fully connected layers, and a non-linear activation function is applied after the hidden layer to map the low-dimensional structural description to a high-dimensional semantic representation. In one example, the input layer receives the frame descriptor vector, the hidden layer performs a non-linear combination of angular, distance, and local structural relationships within the frame descriptor, and the output layer outputs a high-dimensional feature vector corresponding to the current structural distribution of the central region. This high-dimensional feature vector enhances the frame descriptor's ability to express local scale changes and structural perturbations, enabling the low-dimensional description, originally based on geometric relationships, to be further mapped into a feature representation suitable for subsequent matching and region localization. Those skilled in the art will understand that the number of layers, the dimension of the hidden layer, the form of the activation function, and the output dimension of the MLP network can all be set according to the feature complexity of the central region and the subsequent matching accuracy requirements, as long as it can complete the high-dimensional mapping of the frame descriptor. This application does not impose further limitations.

[0069] Furthermore, the addition with the gray-level centroid can be understood as mapping the high-dimensional feature vector to a spatial offset representation associated with the gray-level centroid based on the gray-level centroid coordinates, and then forming a matching feature indicating the local structural range through coordinate superposition; alternatively, it can be understood as encoding the position information of the gray-level centroid and fusing it with the high-dimensional feature vector in a unified feature space to generate a matching feature that simultaneously contains central position constraints and local structural distribution information. Those skilled in the art will understand that, in specific implementations, the addition operation can be either direct numerical superposition or a weighted fusion process after dimensional alignment or linear transformation, as long as it enables the central position information represented by the gray-level centroid and the local structural information represented by the high-dimensional feature vector to form a unified representation.

[0070] Furthermore, the matching features can be decoded into scale range parameters corresponding to the central region. These scale range parameters may include one or more of the following: region center coordinates, region width, region height, and region orientation parameters. Subsequently, a region boundary is generated in the original image coordinate system or the target bounding box local coordinate system based on the scale range parameters, thereby obtaining a scale-adaptive region. Compared to a region obtained solely through fixed-ratio cropping, this scale-adaptive region can more accurately maintain coverage of the target subject region even when there are subtle changes in the local structural distribution at the target center, thus improving the dynamic scale template's ability to represent local scale changes in the target.

[0071] Those skilled in the art will understand that the aforementioned corner feature extraction method, gray-scale centroid calculation method, angle calculation method, frame descriptor composition, MLP network structure configuration, and feature matching to region range decoding method can all be adaptively adjusted according to the target image resolution, the texture complexity of the target center region, and subsequent template update requirements. It is sufficient to establish a minimum mapping relationship between the local structure distribution of the center region and the scale-adaptive region range. This application does not impose any further limitations.

[0072] In yet another example, the method for determining the target position in the current frame includes: S2.1: Input the search region into the search branch of the lightweight Siamese network for feature encoding to obtain a search region feature map; Specifically, the search region in the current frame is not directly matched across the entire image. Instead, it is cropped around the target position in the previous frame according to a preset search scale to obtain local image patches, which are then input into the search branch of the lightweight Siamese network for encoding.

[0073] Understandably, while the target may move rapidly in the image due to changes in platform attitude under strapdown conditions, its actual possible location between adjacent frames remains concentrated within a limited area near the target's position in the previous frame. Directly encoding and matching features across the entire frame would introduce a large number of irrelevant background areas, increasing the probability of mismatches, and significantly increase the computational burden on the onboard processor, causing a time lag in the current position observation and affecting the temporal consistency of subsequent state determination and tracking updates.

[0074] In this embodiment, the search area can be constructed based on the center position, width, and height of the target box in the previous frame. Specifically, the rectangular search area to be encoded in the current frame can be obtained by expanding the target box from the center outwards by a preset expansion ratio. The preset expansion ratio can be set according to the stability of the target motion, the drone's flight speed, the image frame rate, and the rate of change of the field of view. In one example, when the video frame rate is 30 to 60 frames per second and the drone is performing a routine follow-flight or accompanying observation task, the width and height of the search area can be set to 2 to 3 times the width and height of the target box in the previous frame, respectively. In a more specific example, the width of the search area can be set to 2.5 times the width of the target box in the previous frame, and the height can be set to 2.5 times the height of the target box in the previous frame. If the target state in the previous frame is detected to be close to the edge of instability or the flight attitude changes rapidly, the expansion ratio can be appropriately increased, for example, to 3.2 or 3.5 times. Those skilled in the art will understand that the expansion ratio does not need to be fixed in all scenarios, as long as it can control the amount of background introduction while ensuring search coverage. In this embodiment, after constructing the search region, the search region is scaled to adapt its size to the input specifications of the lightweight Siamese network search branch. As an example, the original search region can be scaled to an input image block of 255 x 255 pixels or 287 x 287 pixels, and pixel value normalization, brightness normalization, and channel order unification processing are performed on the input image block.

[0075] Furthermore, after the search region enters the lightweight Siamese network search branch, it sequentially passes through an input convolutional layer, multiple depthwise separable convolutional units, and a feature integration layer, outputting a feature map of the search region. The input convolutional layer is used to perform initial texture extraction and spatial downsampling on the normalized search region, enabling local edges, corners, and gray-level variation areas in the image patch to form initial feature responses. The multiple depthwise separable convolutional units are used to complete spatial feature extraction and channel feature fusion under low computational requirements. Among them, channel-wise convolution is responsible for extracting the local spatial structure within each channel, and pointwise convolution is responsible for establishing the correlation between different channels and compressing redundant features.

[0076] It should be noted that the tracking process under strapdown is quite sensitive to inference latency. If a conventional convolutional structure with high parameter and computational cost is used, the output of the search region feature map is prone to lag behind the actual imaging time. This results in subsequent position matching results that are spatially acceptable but temporally deviate from the true target state of the current frame. By extracting search region features through a lightweight backbone structure, a search region feature map that combines local texture information and mid-level structural semantics can be formed in a shorter time. The search region feature map preserves the spatial distribution relationship near the potential target location and suppresses large-scale background redundancy, which is beneficial for subsequent correlation matching with the two types of templates. The output resolution and number of channels of the feature map can be set according to the processor performance and matching accuracy requirements. In one example, when the input size is 255 x 255 pixels, a search region feature map with a spatial resolution of 17 x 17 to 31 x 31 and a number of channels of 128 to 128 can be output; in a more specific example, a search region feature map with a spatial resolution of 25 x 25 and a number of channels of 192 can be output to balance position sensitivity and onboard inference efficiency.

[0077] S2.2: Input the global baseline template into the template branch of the lightweight Siamese network for feature encoding to obtain a global template feature map, and perform correlation matching between the search region feature map and the global template feature map to generate a first response map for characterizing the overall position of the target; Specifically, the global baseline template serves to anchor the overall appearance of the target. Its core purpose is not to describe all the detailed changes of the target at a certain moment, but to continuously provide an overall reference that remains stable over a long period of time.

[0078] Understandably, while the appearance of a target under strapdown perspective changes with flight attitude, distance, and illumination disturbances, the overall structural relationships, contour distribution, and main configuration of the target remain continuous over a period of time. In actual tracking, relying solely on a dynamically updated template over time, while improving sensitivity to local scale changes, is more susceptible to contamination and gradual deviation from the true target appearance under conditions of local occlusion, strong local reflections, intrusion of similar background objects, or short-term matching biases. The global baseline template, maintaining a relatively complete initial representation, provides an overall matching reference for the current frame that remains unchanged despite short-term disturbances. Therefore, encoding it separately in the template branch and participating in relevant matching is intended to form a relatively stable chain of evidence for the overall target position in subsequent response maps.

[0079] In this embodiment, the global baseline template can be an image patch of the overall target region extracted from the initial target bounding box in the initial frame as input. Before being input into the template branch, the template image patch undergoes scale normalization and pixel value normalization to ensure its input specifications are consistent with the feature encoding system of the search branch. The template branch and the search branch share parameters; therefore, after encoding by the template branch, the output global template feature map and the search region feature map are in the same feature space, and they have directly comparable semantic expression scales. The reason for sharing parameters is that if the template branch and the search branch use different parameter sets for encoding, it is easy to cause a shift in the distribution of template features and search features, and subsequent related matching results are easily affected by the inconsistency of feature domains, reducing the reliability of position estimation. After the global baseline template is input into the template branch, the template branch first performs shallow texture encoding on it, and then extracts the overall structural features through several depthwise separable convolutional units, finally outputting a smaller but semantically more concentrated global template feature map. The feature map focuses on preserving relatively stable appearance information such as the overall outline of the target, the edges of the main structure, and the distribution of the main texture, while weakening the instantaneous changes in local details.

[0080] Furthermore, relevance matching can be understood as using the global template feature map as a matching reference, sliding it position by position on the search region feature map, and calculating the similarity between each position and the template features, thus obtaining a two-dimensional response distribution corresponding to the spatial position of the search region. The higher the response value, the more consistent the search region features at that spatial position are with the global template features; the lower the response value, the more likely that the position corresponds to the background or non-target region. The first response map mainly reflects which positions in the current search region are consistent with the initial overall appearance of the target. When the target undergoes slight deformation, local brightness changes, or some local areas are temporarily invisible, the first response map can still maintain a relatively stable main peak position due to the overall structural similarity, thus providing a basic positional constraint for the subsequent joint response map. To ensure that the first response map has both positional resolution and a certain degree of smoothness, in one example, the size of the global template feature map output by the template branch can be set to 7x7 to 13x13, the size of the search region feature map can be set to 25x25, and the corresponding generated first response map size can be 19x19 or other response planes consistent with the sliding matching step size. Those skilled in the art will understand that the dimensional relationships can be adjusted according to the input specifications and the network downsampling rate, as long as a decodeable overall positional response distribution can be formed.

[0081] S2.3: Input the dynamic scale template into the template branch of the lightweight Siamese network for feature encoding to obtain a dynamic template feature map, and perform correlation matching between the search region feature map and the dynamic template feature map to generate a second response map for characterizing the local scale change of the target; Specifically, the dynamic scale template differs from the global reference template. Its focus is not on maintaining the initial complete appearance in the long term, but on following the local scale changes and local detail enhancements that occur during the imaging process of the target, so as to form a representation of the current target subject area that is closer to the current state.

[0082] Understandably, targets in strapdown mode tend to enlarge rapidly as the drone approaches, and local structures that were initially indistinguishable or only had rough outlines in the initial frame gradually become apparent. If only a global reference template is used for overall matching, although it can maintain strong anti-interference capabilities, after the target scale changes significantly, the inconsistency between the template and the actual target in terms of local geometry and texture will continue to accumulate, thus gradually weakening the overall matching response. Especially when there are other background targets with roughly similar outlines in the search area, relying solely on the global template can easily lead to inaccurate localization of the main peak.

[0083] In this embodiment, the dynamic scale template can be extracted from the scale-adaptive region constructed from the central region of the initial target bounding box, and updated conditionally according to the scale change rate in subsequent non-occluded states. The dynamic scale template currently used for S2.3 is the template representation saved before entering the current frame. After inputting this template into the template branch, it is encoded using the same lightweight backbone structure as the search branch, outputting a dynamic template feature map. Since the dynamic scale template typically focuses more on the local region of the target center, local structural boundaries, and scale-related textures, its encoded output emphasizes fine-grained local matching capabilities in the feature space. When performing correlation matching with the search region feature map subsequently, the matching results are better able to reflect which locations in the search region are consistent with the dynamic scale template in terms of local details, local scale, and central region structure. Compared to the first response map, which mainly serves as an overall position anchor, the second response map is more suitable as evidence of local adaptability, used to further distinguish between candidate locations with similar overall structures but different local scales and central details.

[0084] Furthermore, high response positions in the second response map indicate that the search region at that location is not only suspected of being the target in terms of its presence, but also closer to the dynamic scale template in terms of local scale, central region texture density, edge distribution, and detailed structure. The reason for employing this dual-path matching is that overall appearance consistency and local scale consistency do not always change synchronously in airborne continuous tracking scenarios. In some frames, the overall outline of the target is still roughly discernible, but local details have been significantly magnified; in other frames, the local structure is similar to the dynamic template, but without overall template constraints, it may be attracted by similar local textures in the background. By constructing a separate second response map, supplementary constraints at the local scale level can be provided for the target's current position during the joint decoding stage, reducing positional deviations caused by a single path template. In this embodiment, the calculation of the second response map can reuse the search region feature map, eliminating the need for repeated encoding of the search region. This allows for dual-template matching without increasing the number of forward propagations of the search region, reducing the airborne computational burden and maintaining the temporal synchronization of the two response results. To ensure the local localization capability of the second response map, in one example, the spatial size of the dynamic template feature map can be controlled within a range that is similar to or slightly smaller than that of the global template feature map, while the number of channels remains consistent, so that position-by-position correspondence can be performed during subsequent weighted fusion.

[0085] S2.4: The first response map and the second response map are weighted and fused to obtain a joint response map, wherein the weights are determined based on the peak response intensity and response distribution concentration of the first response map and the second response map; Specifically, after obtaining the first response map and the second response map, neither response map is directly selected as the basis for determining the target location. Instead, the two response results are weighted and fused.

[0086] Understandably, the reliability of the global and dynamic templates is not constant across different frames. The global baseline template is generally more reliable when the overall target structure is discernible, local information is disturbed but the overall outline is still intact; the dynamic scale template is generally closer to the current target state when the target is rapidly magnified, local details are enhanced, and there is no occlusion. If one of the response maps is used consistently, the compensation information provided by the other evidence chain will be lost in some frames, thus reducing the stability of the current position estimation. Therefore, it is necessary to dynamically adjust the contribution ratio of the two response maps in the joint response map according to their respective reliability in the current frame.

[0087] In this embodiment, the weights are not determined by subjective fixed values, but are calculated based on the peak response intensity and response distribution concentration of the first and second response maps, respectively. Peak response intensity reflects the consistency between the optimal matching position and the template in the response map; a higher peak indicates that the template path is more likely to provide a reliable position in the current frame. Response distribution concentration reflects whether the main peak of the response map is prominent and whether there are many similar response regions around the peak; a higher concentration indicates a clearer discrimination boundary of the current response map. Determining weights solely based on peak size is insufficient, as accidental similarity in local textures can also form higher peaks in complex backgrounds. Determining weights solely based on distribution concentration is also insufficient, as some low-quality response maps, although narrowly distributed, may have main peaks corresponding to incorrect positions. Therefore, combining both methods better aligns with the actual judgment logic for matching reliability in continuous tracking. Specifically, the maximum response value, the second-highest response value, the average response of the main peak's neighborhood, and the response fluctuation degree of non-main peak regions can be extracted from the first and second response maps respectively. These are then combined to form their respective reliability evaluation values, and the corresponding fusion weights are calculated accordingly. In one example, the weights corresponding to the first response map can be limited to between 0.3 and 0.7, and similarly, the weights corresponding to the second response map can be limited to between 0.3 and 0.7, with the sum of the two being one. When the first response map is more stable overall, the weight of the first response map can be 0.65, and the weight of the second response map can be 0.35. When the target is in a rapidly magnifying but unoccluded phase, the weight of the second response map can be increased to 0.55 or 0.6. Those skilled in the art will understand that the weight ranges can be adjusted according to the task scenario and network output characteristics, but it is important to avoid a situation where the weight of a certain path is close to one for a long time, so as to retain the practical significance of dual-template collaboration.

[0088] Furthermore, after determining the weights, the first and second response maps are fused position-by-position to obtain a joint response map. The joint response map retains the structural stability constraints of the overall template and the scale adaptation constraints of the dynamic template, ensuring that the final response distribution is neither entirely affected by the initial template lag nor by the short-term offset of the dynamic template. When the target is rapidly approaching and local textures are enhanced but the overall target remains the same, the joint response map can provide a position closer to the true center than the first response map. Conversely, when the dynamic template is locally affected by noise or edge background interference, the joint response map can maintain the main peak without abrupt changes thanks to the first response map. To avoid fusion bias caused by different dimensions of the two response maps, amplitude normalization can be performed on the first and second response maps before fusion. For example, their respective response values ​​can be mapped to a unified numerical range, or standardized based on their respective peak values ​​and background mean values. This preprocessing helps ensure that the weight adjustment reflects the relative reliability of the response maps, rather than the original numerical scale differences. The joint response map obtained after weighted fusion is more suitable as the final solution basis for the target position in the current frame than either individual response map.

[0089] S2.5: Determine the target position of the current frame based on the position corresponding to the maximum response value in the joint response graph, wherein the maximum response value is obtained by peak search calculation on the joint response graph; Specifically, after forming the joint response map, the optimal matching position in the two-dimensional response plane needs to be converted into the specific target position in the current frame. The joint response map essentially represents the relative consistency between different spatial positions within the search area and the combined matching of the two templates. The response peak corresponding to the target's true position changes dynamically in the current frame. Especially in strapdown perspective scenarios, the target's center relative to the previous frame is not fixed. Without solving for the optimal position through response peak search, it is impossible to achieve adaptive position updates based on the specific matching situation in the current frame.

[0090] In this embodiment, peak search can be performed by scanning the joint response map position by position to obtain the candidate peak position with the highest response value. To prevent local noise peaks or single-pixel spikes from affecting position decoding, a light smoothing process or a local neighborhood consistency check can be performed on the joint response map before performing peak search. Specifically, neighborhood mean filtering, median filtering, or local response clustering can be used to weaken isolated noise response points and make the continuous peak region corresponding to the true target position clearer. Subsequently, the position of the maximum response value is searched in the joint response map, and this position is used as the initial peak position.

[0091] Furthermore, after obtaining the peak position in the joint response map, it needs to be mapped back from the response map coordinate system to the search region coordinate system, and then further mapped to the original image coordinate system of the current frame to obtain the target position of the current frame. Specifically, the position calculation can be completed based on the center position of the search region during cropping, the scaling ratio of the search region, the network downsampling step size, and the coordinate correspondence between the response map and the search region. As an example, when the search region is cropped from the original image and scaled to a 255x255 pixel input, and the network feature map and response map correspond to a fixed downsampling step size, the actual pixel offset in the search region can be calculated based on the offset of the peak position in the response map relative to the center of the response map, and then superimposed on the target position center of the previous frame to obtain the target center coordinates of the current frame.

[0092] It should be noted that the specific calculation methods for peak response intensity, response distribution concentration, and maximum response value described in this application can be determined with reference to conventional processing methods for evaluating response map quality in the field of target tracking. Specifically, peak response intensity can be calculated based on the response value at the peak position in the response map and its difference from the response values ​​of the surrounding neighborhood, used to characterize the matching strength of the current response map to the candidate target position; response distribution concentration can be calculated based on the difference in response distribution between the peak region and non-peak region in the response map, used to characterize whether the main peak is concentrated and whether the background response is dispersed; the maximum response value can be obtained directly by performing a peak search on the joint response map, or it can be refined based on the peak search combined with the local response distribution within the peak neighborhood. For those skilled in the art, it can be understood that the above evaluation quantities can be evaluated using the average peak correlation energy method, or using the peak-sidelobe ratio, the difference between the peak and the mean, the contrast between the peak and the local background, or other evaluation methods that can reflect the prominence of the main peak and the degree of response distribution aggregation in the response map, as long as they can at least support the determination of the weights of the two response maps and the decoding of the current position. This application does not impose further limitations in this regard.

[0093] Next, we will further elaborate on the technical content of the state determination method in this application.

[0094] In one example, the method for determining the target state includes: The response quality index is determined based on the joint response map, and the current target color feature is extracted at the initial position of the target in the current frame. The similarity between the response quality index and the current target color feature relative to the preset color reference feature is jointly determined to obtain the target state corresponding to the current frame. When the response quality index is less than the preset occlusion determination condition and the similarity between the current target color feature and the color reference feature is lower than the preset similarity threshold, it is determined to be an occlusion state.

[0095] Understandably, from the perspective of UAV strapdown, the sources of change in the response peak are not singular: partial occlusion of the target can cause the main peak to attenuate, and rapid changes in illumination, image blurring, localized high-brightness reflections, or similar textures in the background entering the search area can also cause the main peak to decrease or the side lobes to rise. If only the maximum response value itself is used to determine whether occlusion exists, recoverable states such as changes in illumination and attitude disturbances are easily misjudged as occlusion states, leading to premature cessation of template updates and affecting subsequent normal tracking. Conversely, if only color or appearance similarity is used for judgment, it is easy to miss true occlusion when the color of the occluder is close to the target color. Therefore, this embodiment adopts a dual-criteria method of response quality index and color similarity. The response quality index is used to reflect the reliability of the spatial distribution of the matching result at the current position, and the color similarity is used to reflect whether the area corresponding to the current position still maintains the continuity of the initial target in terms of appearance attributes. The combination of the two can distinguish between unstable matching but the target is still visible and unstable matching and the target has been occluded.

[0096] In this embodiment, the response quality index can be determined by the distribution relationship between the main peak and non-main peak regions in the joint response map. Specifically, the global peak position can be searched in the joint response map first, and the peak response at that position can be obtained; then, the mean response, response fluctuation range, and background response level outside the neighborhood of the main peak in the remaining regions of the joint response map can be statistically analyzed to construct the response quality index. As an exemplary implementation, the difference between the peak response and the average response of the non-peak regions can be used as the prominence of the main peak, and then combined with the overall dispersion of the response map to obtain the final response quality index; it can also be understood that the higher the main peak and the flatter the non-peak regions, the larger the response quality index, indicating that the matching result at the current position is more reliable. To facilitate engineering implementation, in the offline testing phase, the distribution of the response quality index can be statistically analyzed using labeled normal tracking samples, illumination change samples, and occlusion samples, and the threshold range can be determined accordingly. In a specific example, after the joint response map is normalized, the response quality index can be mapped to a range of 0 to 1. Under normal tracking conditions, the response quality index is typically above 0.45; under conditions of varying or slight illumination, it is typically between 0.25 and 0.45; and under occlusion conditions, it is typically below 0.25. Therefore, the preset occlusion determination condition can be set to 0.25, or in some embodiments, it can be set to an empirical threshold between 0.22 and 0.3. Those skilled in the art will understand that the empirical threshold is not fixed and can be calibrated based on sensor noise levels, search area size, and network response distribution characteristics, as long as it can statistically distinguish between occluded and non-occluded states. For the extraction of the current target's color features, a target candidate region can be extracted from the initial position of the target in the current frame, and the color descriptor can be extracted after color space conversion of this region. Candidate regions can be converted from the RGB color space to the HSV color space or a color name feature space, and then binning and statistically analyzing the hue, saturation, and brightness components to form a color histogram. Alternatively, a predefined dimension color name feature vector can be directly extracted to enhance tolerance to brightness variations. In one example, if an HSV color histogram is used, the hue can be divided into 16 intervals, the saturation into 4 intervals, and the brightness into 4 intervals, resulting in a color feature vector with a total dimension of 256. If color name features are used, an 11-dimensional or higher-dimensional color semantic vector can be output. The color baseline feature can be extracted and saved from the same target region after the initial frame target bounding box is determined, and used for similarity comparison in subsequent frames. The color similarity between the current target color feature and the color baseline feature can be calculated using the Bach coefficient, cosine similarity, or histogram intersection-union ratio. In a specific example, the Bach coefficient can be used and normalized to between 0 and 1; a larger value indicates that the appearance color is closer to the initial target.

[0097] Furthermore, during joint determination, the reliability of the current position matching can be determined first based on the response quality index, and then the corresponding area can be determined based on color similarity to see if it can still be considered as a continuous appearance of the original target. When the response quality index is not lower than the preset occlusion determination condition, it can be considered that the main peak of the joint response map still has sufficient spatial discrimination ability. At this time, even if there is some fluctuation in the color distribution, it usually belongs to the normal tracking state or the short-term appearance change state. When the response quality index is lower than the preset occlusion determination condition, it indicates that the current position matching has become significantly unstable. At this time, color similarity is further introduced as a secondary constraint. If the similarity between the current target color feature and the color reference feature is also lower than the preset similarity threshold, it indicates that the current position is not only unstable in spatial response, but also deviates from the initial target in appearance attributes, and can be determined as an occlusion state. As an example, if the normalized color similarity value ranges from 0 to 1, the preset similarity threshold can be set to 0.6; in tasks with large color fluctuations, it can be set to 0.55; in tasks with complex background colors and stable target colors, it can be set to 0.65. Correspondingly, when the response quality index is less than 0.25 and the color similarity is less than 0.6, the target can be determined to be in an occluded state; when the response quality index is less than 0.25 and the color similarity is not less than 0.6, it is more likely to be a non-occluded unstable state caused by changes in lighting, local shadows, or short-term attitude changes.

[0098] Next, we will further elaborate on the technical content of the single-target tracking method in this application.

[0099] It is understandable that this application divides single-target tracking into two specific cases. For continuous target tracking under airborne strapdown perspective, obtaining the target position in the current frame does not mean that subsequent processing can uniformly adopt a single update strategy. Although the target exhibits position and appearance changes between adjacent frames, the underlying mechanisms of these changes are not the same: one type of change originates from the continuous evolution of the relative distance, viewing angle, and local illumination conditions between the target and the imaging device. In this case, the observation results of the current frame usually still have high reliability, and subsequent processing is more suitable to proceed along the path of observation correction-template absorption-continuous position update. The other type of change corresponds to a state where local target information is significantly lacking, the main peak response is degraded, and the appearance continuity is disrupted. In this case, although the current frame can still obtain a certain form of position output, this output is closer to an unstable observation. If the aforementioned update path is continued, it is easy to write abnormal information into the dynamic representation, thereby shifting the matching basis of subsequent frames. Based on this difference, this application divides single-target tracking into two paths during implementation: one where the observation results can directly participate in the current frame update, and another where the observation results are no longer suitable as the basis for updating. This ensures that subsequent steps no longer rely on mechanically extrapolating the target position, but instead select different tracking maintenance methods based on the state attributes corresponding to the current position. During normal continuous evolution, the target can retain its adaptability to scale changes and local appearance enhancements, while avoiding representational deviations caused by unreliable observations during information-deficient phases. This allows the same tracking framework to have clearer boundary handling capabilities for different imaging states.

[0100] In one example, performing single-target tracking based on the state includes: When the target is in a normal tracking state or an illumination change state, the scale change rate of the target box in the current frame relative to the target box in the previous frame is calculated. When the scale change rate is greater than a preset scale threshold, the target features of the current frame are fused into the dynamic scale template using an exponential moving average algorithm to update the dynamic scale template. The updated dynamic scale template is then used as an observation value and input into a preset Kalman filter for state correction to obtain the final position of the target in the current frame. Tracking is then repeated for subsequent video frames based on the final position of the target in the current frame.

[0101] It is understandable that the aforementioned single-target tracking corresponds to the processing scenario where the target remains observable, rectifiable, and continuously updated in the current frame. In this scenario, the current position given by the joint response map not only still has good spatial peak constraints, but also maintains continuity between the region corresponding to the current position and the baseline appearance. This indicates that the target changes reflected in the current frame image mainly originate from changes in imaging distance, viewing angle, or local morphological unfolding, rather than information breaks caused by real occlusion or severe background intrusion.

[0102] In specific processing, the scale change rate can be calculated first based on the relative change relationship between the width and height of the target box in the current frame and the width and height of the target box in the previous frame. The scale change rate can be the target box area change rate, or the average or weighted value can be obtained after statistically analyzing the width change rate and the height change rate respectively.

[0103] In one example, the ratio of the difference between the current frame's target box area and the previous frame's target box area to the previous frame's target box area can be used as the scale rate of change; in another example, half the sum of the width rate of change and the height rate of change can also be used as the scale rate of change.

[0104] Furthermore, to ensure that the dynamic scale template only participates in updates when the target undergoes a perceptible scale evolution, a preset scale threshold can be determined through offline calibration or sample statistics. In a commonly used example, the preset scale threshold can be set to 8% to 15%; when the tracked target magnifies rapidly in the airborne view, the threshold can be set to 10%; when the target magnification speed is slow in the application scenario but there is a lot of background interference, the threshold can be appropriately increased to 12% or 15% to avoid frequent template updates due to small boundary fluctuations.

[0105] In this embodiment, when the scale change rate is greater than the preset scale threshold, it indicates that the target in the current frame has undergone a scale change sufficient to affect the subsequent template matching compared to the previous frame. At this time, the target features are extracted from the target box in the current frame and fused into the dynamic scale template using an exponential moving average method.

[0106] It should be noted that although the target features in the current frame are closer to the current real target state, they may still contain a certain degree of local noise, illumination deviation, or boundary clipping error. If the original dynamic scale template is directly replaced, the template is prone to abrupt changes under single-frame perturbation. If a fixed equal-weighted average is used, there is a lack of distinction between the retention of historical and current templates, making it difficult to balance template continuity and update sensitivity. Exponential moving average processing enables the updated dynamic scale template to simultaneously retain the stable structural information in the existing template and the newly emerging scale details in the current frame, thus forming a smooth evolution between adjacent frames.

[0107] In practice, the target features of the current frame can be combined with the dynamic scale template before the update according to a preset fusion coefficient. The fusion weight of the target features of the current frame can be set to 0.2 to 0.4, and the retention weight of the original features of the dynamic scale template can be set to 0.6 to 0.8. If the scale change rate is detected to be continuously higher than the preset scale threshold for multiple consecutive frames, the fusion ratio of the current frame features can be appropriately increased to enhance the ability of the dynamic scale template to follow the continuous magnification process. The updated dynamic scale template no longer only represents the initial central region, but gradually absorbs the new local structures that appear during the target's approach, so that the second response map in the subsequent matching stage can more accurately reflect the current local scale state of the target, thereby reducing the main peak shift caused by template lag.

[0108] Furthermore, when the target in the current frame is in a normal tracking state or a state of changing illumination, the current position observation result can still be regarded as a reliable input. Therefore, the updated dynamic scale template can further participate in the state correction process to obtain the final position of the target in the current frame.

[0109] Specifically, the target position obtained from the joint response map of the current frame and the current appearance state obtained from the updated dynamic scale template can be input into the Kalman filter to complete the current frame state correction based on the existing state estimation.

[0110] It is important to emphasize that using the updated dynamic scale template as the input observation does not mean directly replacing the position coordinates with the template itself. Instead, it uses the current position observation result corresponding to the matching with the updated dynamic scale template as the external observation of the filter. This allows the filter to retain the historical motion trend while introducing the current frame's scale-compensated position information to correct the target center position.

[0111] It is understood that using a Kalman filter to perform state correction or prior state prediction of the target position in the current frame is a common state estimation processing method in this field, and those skilled in the art can implement it based on the existing technology. This application will not elaborate on this further.

[0112] In yet another example, performing single-target tracking based on the stated state further includes: When the target is in an occluded state, the dynamic scale template is locked and updates are stopped. Prior state prediction is performed through a preset Kalman filter to output the predicted position of the target in the current frame as the final position of the target in the current frame. Tracking is then repeated for subsequent video frames based on the final position of the target in the current frame.

[0113] It is understandable that the aforementioned single-target tracking corresponds to the processing situation where the target in the current frame is no longer suitable as the basis for continuous template updates. In this situation, although the joint response map may still provide a candidate position, this candidate position reflects more the local optimum of the residual matching information in the current search area, rather than the stable observation result supported by the complete target appearance. At this time, if the area corresponding to the candidate position in the current frame is still used for dynamic scale template updates, it is very easy to absorb the appearance information of the occlusion edge, background texture, or local pseudo-peak area into the template, causing the dynamic scale template to gradually deviate from the real target. Once this kind of deviation accumulates in several consecutive frames, even if the occlusion ends later, the effective correspondence between the template and the real target may have been lost, leading to difficulties in re-locking after recovery. Therefore, in this embodiment, when the target state is determined to be occluded, the processing path of the non-occlusion stage is no longer used. Instead, the dynamic scale template is immediately locked and the update is stopped, keeping it in the state after the last reliable update before entering the occlusion stage, thereby preventing abnormal visual information from continuing to enter the template representation link.

[0114] It should be noted that the locking described in this application refers to the dynamic scale template no longer receiving any feature fusion input from the target region of the current frame within several consecutive occlusion frames. Its template content remains unchanged, retaining only the local scale representation established before entering the occlusion state. Simultaneously, the global reference template maintains its initial stable reference function, used for candidate region verification in the subsequent occlusion recovery stage. This approach prioritizes protecting existing template representations from contamination during occlusion, ensuring that a stable appearance basis for matching recovery is retained once valid target information is regained in subsequent frames.

[0115] In one example, prior state prediction is performed using a pre-defined Kalman filter, including: After determining that the target to be tracked is in an occluded state, the target position of multiple consecutive frames before the occlusion occurred is obtained; Specifically, after entering the occlusion state, it is no longer suitable to directly rely on the matching results in the current frame image to correct the target position. This is because although the joint response map may still have local response peaks, the regions corresponding to these local response peaks are likely to be mixed with occlusion boundaries, high-texture background regions, or locally similar interference regions. If these are still used as the direct observation input for the current frame, the original trajectory continuity will be destroyed. In order to ensure that subsequent position prediction is still based on reliable temporal information, it is necessary to backtrack to a reliable continuous target position sequence that was confirmed before the occlusion occurred, and use this position sequence as the data source for prior state prediction.

[0116] The position change trend of the target to be tracked is determined based on the target position in the consecutive frames, and the target motion state is constructed based on the position change trend, wherein the target motion state includes at least one of the target position, motion speed and motion direction; In this embodiment, the position change trend can be obtained through temporal difference, local smoothing, and direction statistics of a series of consecutive target center coordinates. Specifically, the displacement between the center coordinates of two adjacent frames can be calculated first to obtain a set of continuous displacement vectors; then, the direction and magnitude of change of this set of displacement vectors over time can be statistically analyzed to form the main displacement direction and average velocity level of the target before occlusion occurs. For example, the target center positions of the six consecutive frames before entering the occlusion state can be recorded as the first position to the sixth position, and the displacement vectors of the second position relative to the first position, the third position relative to the second position, and so on up to the sixth position relative to the fifth position can be calculated respectively; if these displacement vectors are generally in the same direction and the magnitude changes are within a small range, their average result can be used as the dominant motion trend of the current target; if individual displacement vectors have large deviations, their influence can be reduced by moving average or median filtering. In a specific example, the five most recent adjacent displacement vectors can be smoothed to obtain a smoothed horizontal displacement component and a vertical displacement component, and this smoothing result can be used as the basis for estimating the velocity of the current target. For the direction of motion, the quadrant and azimuth angle interval can be determined based on the smoothed horizontal and vertical displacement components. Specifically, the combination of horizontal and vertical displacements can be used to determine whether the target is mainly moving in the upper left, upper right, lower left, lower right, or other directions within the image plane. If explicit output of the azimuth angle is not required in actual deployment, the direction of motion can also be expressed directly in the form of horizontal and vertical velocity components.

[0117] Furthermore, the target center coordinates of the last reliable frame can be used as the position item in the state, the smoothed average displacement can be converted into the velocity per frame as the velocity item, and the sign relationship or direction category of the velocity direction can be used as the direction item. If the target bounding box width and height are recorded at the same time, an additional scale change trend item can be formed to help determine whether the search area needs to be expanded when occlusion continues. In order to ensure that the state construction process is fully disclosed, exemplary values ​​can be given in the implementation: For example, when the horizontal coordinates of the target center of six consecutive frames are 300, 306, 313, 320, 327, 335, and the vertical coordinates are 200, 202, 204, 207, 210, 214, after smoothing, it can be determined that the target is continuously moving in the lower right direction in the image, with an average horizontal movement of about seven pixels per frame and an average vertical movement of about three pixels per frame. At this time, the target motion state containing the current position (335, 214), a horizontal velocity of about 7 pixels per frame, and a vertical velocity of about 3 pixels per frame can be constructed.

[0118] Based on the target motion state and combined with the global reference template, the Kalman filter is used to perform prior state prediction and output the predicted position. Specifically, after constructing the target motion state, the prior state prediction stage begins. This application relies not only on the motion state but also on a global reference template because processing in occlusion scenarios involves not only the continuity of the current position but also whether the subsequent recovery stage can return to the correct target. The Kalman filter performs the task of continuous position extrapolation, while the global reference template retains the overall target appearance anchoring information. In the occlusion stage, they respectively serve two types of problems: unbroken trajectory and unwavering target identity. If only the motion state is used for position extrapolation, although a continuous predicted position can be obtained, after several frames of occlusion, multiple locally similar regions may appear around the predicted position. Without a stable overall appearance reference, subsequent recovery matching may easily result in mis-attaching to adjacent interfering targets. Therefore, in this embodiment, although the prior prediction stage is mainly performed by the Kalman filter for position recursion, the global reference template does not exit the processing chain but is retained as a stable reference for constraining the confidence range of the subsequent predicted position and verifying the recovery matching.

[0119] In this embodiment, the input received by the Kalman filter is not the visual observation value of the current frame, but the target motion state established in the previous moment. Specifically, the target center position, smoothed velocity components, and optional scale trend information of the last reliable frame can be written into the filter state. In the current frame, no new visual correction value is input, but only a time update is performed, and the predicted position of the current frame is output.

[0120] It should be noted that the visual observation of the current frame under occlusion is considered unreliable. If it is still used as the input observation value, it will disrupt the continuity of the original motion trend. Performing only time updates can ensure that the prediction results are completely based on the reliable state before occlusion.

[0121] In one example, when the target's average velocity before occlusion is approximately 7 pixels per frame and its average vertical velocity is approximately 3 pixels per frame, and the last reliable location is (335, 214), the predicted location for the current frame can be recursively extrapolated along this trend to obtain a prediction center approximately (342, 217). If two consecutive frames are in an occluded state, the prediction for the next frame can continue to be extrapolated based on the prediction result of the previous frame. To control the accumulation of prediction errors during prolonged occlusion, in some embodiments, an attenuation coefficient or confidence constraint can be set for the velocity component. For example, when the number of consecutive occluded frames exceeds 3, the velocity component is attenuated frame by frame at a ratio of 0.95 to 0.9 to avoid the extrapolation distance from continuously increasing under long periods of no observation. This attenuation coefficient can be obtained through historical sample statistics. For example, by comparing the occlusion recovery localization error under different attenuation coefficients on the validation set, the minimum average recovery error corresponding to 0.92 can be used as an exemplary setting value.

[0122] In another example, tracking is repeatedly performed on subsequent video frames based on the final position of the target in the current frame, including: Using the final position of the target in the current frame as the center position of the search region in subsequent frames, an extended search region is constructed. The construction method of the extended search region can refer to the construction method of the aforementioned search region, and will not be elaborated here.

[0123] The extended search region is feature-encoded using the lightweight Siamese network, and the encoded extended search region features are matched with the global baseline template to obtain multiple candidate response positions. Based on the location of each candidate response, a corresponding candidate region is determined, and candidate features of each candidate region are extracted. Each candidate feature is compared with the baseline feature corresponding to the dynamic scale template to obtain the consistency evaluation result corresponding to each candidate region. Based on the consistency evaluation results corresponding to each candidate region, the search region is determined from the plurality of candidate response locations.

[0124] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A lightweight single target tracking method for unmanned aerial vehicle strapdown visual angle, characterized in that, The method, applied to a drone equipped with strapdown inertial navigation, the drone including a processor having a pre-defined lightweight twin network configured within the processor, comprises: Obtain the target to be tracked in the initial frame, and establish a global reference template and a dynamic scale template based on the target to be tracked; For the current frame, the search region is determined based on the target position of the previous frame. The lightweight Siamese network is used to match the search region with the global reference template and the dynamic scale template to obtain the target position of the current frame. The target state corresponding to the target to be tracked is determined based on the target position in the current frame, and single target tracking is performed based on the target state. The target state includes at least one of the following: normal tracking state, illumination change state, and occlusion state.

2. The lightweight single-target tracking method for unmanned aerial vehicle strapdown visual orientation according to claim 1, wherein, The lightweight twin network is used to encode the global baseline template and dynamic scale template input to the template branch and the search region input to the search branch, respectively, and generate a target location response map based on the encoded template features and search region features. The main structure of the lightweight twin network is a twin backbone network constructed by depthwise separable convolutions, and the twin backbone network includes a template branch and a search branch with shared parameters.

3. The lightweight single-target tracking method for unmanned aerial vehicle strapdown visual perspective of claim 1, wherein, The global baseline template is constructed in the following ways: In the initial frame, the initial target bounding box of the target to be tracked is determined, the target region corresponding to the initial target bounding box is extracted, and the extracted target features are saved as the global reference template. The initial target bounding box is determined by a target detection algorithm, and the target features are extracted by a feature extraction algorithm based on histogram of oriented gradients.

4. The lightweight single-target tracking method for UAV strapdown perspective according to claim 3, characterized in that, The construction methods of the dynamic scale template include: A scale-adaptive region is constructed based on the central region of the initial target box, and features are extracted from the scale-adaptive region to obtain scale features; The scale feature is used as the dynamic scale template; The central region is determined by shrinking the initial target box inward according to a preset ratio.

5. The lightweight single-target tracking method for UAV strapdown perspective according to claim 4, characterized in that, The methods for constructing the scale-adaptive region include: The corner features and gray-level centroid of the central region are extracted. The frame descriptor is calculated based on the angle between the corner features and the gray-level centroid. The frame descriptor is transformed into high-dimensional data using an MLP network and added to the gray-level centroid to generate matching features. The region corresponding to the matching features is determined as the scale-adaptive region.

6. The lightweight single-target tracking method for UAV strapdown perspective according to claim 2, characterized in that, The methods for determining the target position in the current frame include: The search region is input into the search branch of the lightweight Siamese network for feature encoding to obtain a search region feature map; The global baseline template is input into the template branch of the lightweight Siamese network for feature encoding to obtain a global template feature map. The search region feature map is then matched with the global template feature map to generate a first response map that characterizes the overall position of the target. The dynamic scale template is input into the template branch of the lightweight Siamese network for feature encoding to obtain a dynamic template feature map. The search region feature map is then matched with the dynamic template feature map to generate a second response map that characterizes the local scale changes of the target. The first response map and the second response map are weighted and fused to obtain a joint response map, wherein the weights are determined based on the peak response intensity and response distribution concentration of the first response map and the second response map; The target position of the current frame is determined based on the position corresponding to the maximum response value in the joint response graph, wherein the maximum response value is obtained by peak search calculation on the joint response graph.

7. The lightweight single-target tracking method for UAV strapdown perspective according to claim 6, characterized in that, The method for determining the state includes: The response quality index is determined based on the joint response map, and the current target color feature is extracted at the initial position of the target in the current frame. The similarity between the response quality index and the current target color feature relative to the preset color reference feature is jointly determined to obtain the target state corresponding to the current frame. When the response quality index is less than the preset occlusion determination condition and the similarity between the current target color feature and the color reference feature is lower than the preset similarity threshold, it is determined to be an occlusion state.

8. The lightweight single-target tracking method for UAV strapdown perspective according to claim 7, characterized in that, The method for determining the response quality index includes: Obtain the maximum response value, minimum response value, and average energy distribution of each response value in the joint response graph; Based on the difference between the maximum and minimum response values ​​and the average energy distribution of the joint response map, a response quality index is calculated to characterize the significance of the response peak and the concentration of the response distribution.

9. The lightweight single-target tracking method for UAV strapdown perspective according to claim 1, characterized in that, Performing single-target tracking based on the stated state includes: When the target is in a normal tracking state or an illumination change state, the scale change rate of the target box in the current frame relative to the target box in the previous frame is calculated. When the scale change rate is greater than a preset scale threshold, the target features of the current frame are fused into the dynamic scale template using an exponential moving average algorithm to update the dynamic scale template. The updated dynamic scale template is then used as an observation value and input into a preset Kalman filter for state correction to obtain the final position of the target in the current frame. Tracking is then repeated for subsequent video frames based on the final position of the target in the current frame.

10. The lightweight single-target tracking method for UAV strapdown perspective according to claim 1, characterized in that, Performing single-target tracking based on the aforementioned state also includes: When the target is in an occluded state, the dynamic scale template is locked and updates are stopped. Prior state prediction is performed through a preset Kalman filter to output the predicted position of the target in the current frame as the final position of the target in the current frame. Tracking is then repeated for subsequent video frames based on the final position of the target in the current frame.

11. The lightweight single-target tracking method for UAV strapdown view according to claim 10, characterized in that, Prior state prediction is performed using a pre-defined Kalman filter, including: After determining that the target to be tracked is in an occluded state, the target position of multiple consecutive frames before the occlusion occurred is obtained; The position change trend of the target to be tracked is determined based on the target position in the consecutive frames, and the target motion state is constructed based on the position change trend, wherein the target motion state includes at least one of the target position, motion speed and motion direction; Based on the target motion state and combined with the global reference template, the Kalman filter is used to perform prior state prediction and output the predicted position.

12. The lightweight single-target tracking method for UAV strapdown view according to claim 10, characterized in that, Based on the final position of the target in the current frame, the tracking is repeated in subsequent video frames, including: The final position of the target in the current frame is used as the center position of the search area in subsequent frames to construct an extended search area; The extended search region is feature-encoded using the lightweight Siamese network, and the encoded extended search region features are matched with the global baseline template to obtain multiple candidate response positions. Based on the location of each candidate response, a corresponding candidate region is determined, and candidate features of each candidate region are extracted. Each candidate feature is compared with the benchmark feature corresponding to the dynamic scale template to obtain the consistency evaluation result of each candidate region. Based on the consistency evaluation results corresponding to each candidate region, the search region is determined from the plurality of candidate response locations.

13. A lightweight single-target tracking system for UAV strapdown perspective, used to implement the lightweight single-target tracking method for UAV strapdown perspective as described in any one of claims 1-12, characterized in that, The system includes: The template construction module is used to obtain the target to be tracked in the initial frame and to build a global reference template and a dynamic scale template based on the target to be tracked. The search matching module is used to determine the search area based on the target position of the previous frame for the current frame, and to match the search area with the global reference template and the dynamic scale template through a preset lightweight twin network to obtain the target position of the current frame. The state determination module is used to determine the state of the target to be tracked based on the target position in the current frame. The tracking execution module is used to perform single-target tracking based on the stated state.

Images