Target tracking method, electronic equipment and readable storage medium

Abstract

The invention provides a target tracking method, electronic equipment and a readable storage medium. According to the target tracking method provided by the invention, by combining generation and updating of the static template and the dynamic template, the problems of time-varying characteristics and background noise in infrared small target tracking can be effectively solved. Firstly, a generated static target image template is cut by multiple scales, so that the adaptability to different scales of a target is improved; and the generation of the dynamic target image template is combined with the dynamic template of the previous frame and the temporary template of the current frame, and updating is carried out through a fusion strategy, so that the morphological change and the thermal radiation fluctuation of the target can be reflected in time, and the problem of response lag caused by the time-varying characteristic of the target is solved. And finally, inputting the static and dynamic templates into a pre-trained target tracking model for processing, thereby effectively suppressing interference of complex background noise, and accurately positioning the target.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a target tracking method, an electronic device, and a readable storage medium. Background Art

[0002] Object tracking is a technology in computer vision that aims to monitor and locate objects in dynamic scenes in real time using continuous video frames. This technology extracts target features, such as color, shape, and texture, from video sequences to identify and track the target's position in each frame. The core task of object tracking is to accurately identify and continuously track one or more targets within a dynamically changing background in a video, even if the target is deformed, occluded, or subjected to external interference during motion.

[0003] In the tracking of small infrared targets, the traditional single-template mechanism is unable to cope with the time-varying characteristics of the target (deformation / thermal radiation changes) and complex background interference at the same time, and there is a technical problem of low tracking accuracy. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide a target tracking method, an electronic device and a readable storage medium to solve the technical problem in the related art that in the infrared small target tracking scenario, the traditional single template mechanism is difficult to simultaneously cope with the time-varying characteristics of the target and the complex background interference, resulting in low tracking accuracy.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a target tracking method, comprising: Receiving video data carrying at least one target and corresponding target frame data; generating static target image templates of at least two scales for a first image frame of the video data based on the target frame data; Generate dynamic target image templates of at least two scales for the current T-th image frame; wherein the dynamic target image template of the T-th image frame is generated by fusing the temporary dynamic target image template of the T-th image frame and the dynamic target image template of the T-1-th image frame; wherein the temporary dynamic target image template of the T-th image frame is obtained from the corresponding target frame data in the T-1-th image frame; wherein T is a positive integer greater than or equal to 3; The static target image templates of at least two scales and the dynamic target image templates of at least two scales of the current T-th image frame are input into a pre-trained target tracking model to obtain target frame data.

[0006] Furthermore, the step of generating a dynamic target image template of at least two scales for the T-th image frame includes: Based on the target frame data and a preset multi-scale factor, multi-scale cropping is performed on at least one target in an initial video frame of the video data to obtain a dynamic target image template of at least two scales.

[0007] Furthermore, the step of generating a dynamic target image template of at least two scales for the current T-th image frame includes: Extracting a temporary dynamic target image template of the T-th image frame at at least two scales from the T-th image frame based on previously determined target frame data of the T-1-th image frame and a preset multi-scale factor; wherein the target frame data of the T-1-th image frame is output by the target tracking model; A preset exponentially weighted moving average method is used to weightedly fuse the dynamic target image template of at least two scales of the previously generated T-1th image frame with the temporary dynamic target image template of at least two scales of the Tth image frame to obtain the dynamic target image template of at least two scales of the current Tth image frame; wherein, the dynamic target image template corresponds to a first weight, and the temporary dynamic target image template corresponds to a second weight; wherein, the first weight is a forgetting factor, and the second weight is one minus the forgetting factor.

[0008] Furthermore, the target tracking model includes a feature extraction module, a spatiotemporal cross-attention module and a tracking head module connected in sequence; wherein the feature extraction module is used to perform feature extraction on at least two scales of dynamic target image templates of the current T-th image frame, at least two scales of static target image templates and the current T-th image frame, respectively, to obtain dynamic target image template features, static target image template features and search area image features; the spatiotemporal cross-attention module is used to generate temporal attention features and spatial attention features based on the dynamic target image template, the static target image template and the T-th image frame; the tracking head module is used to generate target frame data based on the temporal attention features and spatial attention features.

[0009] Furthermore, the feature extraction module includes a Vision-Transformer encoder, which is used to perform the following operations on each input image: Split the input image into multiple non-overlapping image blocks; Linearly project each image patch into an embedding vector; Adding a learnable position embedding based on the embedding vector to obtain an embedding vector sequence; An encoding operation is performed on the embedding vector sequence to obtain dynamic target image template features, static target image template features and search area image features.

[0010] Furthermore, the feature extraction module includes a context enhancement module, which is connected after the Vision-Transformer encoder and includes: A feature input layer, which is used to receive static target image template features; The mask prediction subnetwork is used to generate soft mask features; A feature weighting unit is used to perform element-wise multiplication of the soft mask feature and the static target image template feature to perform context enhancement on the static target image template feature.

[0011] Furthermore, the spatiotemporal cross attention module includes: A temporal attention submodule, which is used to receive static target image template features and dynamic target image template features, and then generate temporal attention features through temporal attention calculation; wherein the temporal attention features are used to model the temporal evolution relationship between the dynamic target image template and the static target image template; The spatial attention submodule is used to receive static target image template features, dynamic target image template features and search area image features, then splice the static target image template features with the dynamic target image template features to obtain spliced ​​features, and then perform spatial attention calculation on the spliced ​​features and the search area image features to obtain spatial attention features; wherein, the spatial attention features are used to enhance the spatial matching ability between the template and the search area.

[0012] Furthermore, the tracking head module includes: A feature reconstruction unit, which is used to reconstruct the features input by the spatiotemporal cross attention module into a two-dimensional feature map; The scoring head branch is used to generate a confidence map representing the probability of each location being the target center based on the two-dimensional feature map; An offset prediction branch is used to generate an offset of the target center point relative to the feature map coordinates based on the two-dimensional feature map; A size estimation branch, configured to generate target width and height based on the two-dimensional feature map; The fusion unit is used to determine the center position of the candidate target based on the confidence map, correct the center position of the candidate target based on the offset, and generate the final target box data based on the target width and height.

[0013] In a second aspect, the present invention provides an electronic device comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors to enable the one or more processors to implement the above-mentioned method.

[0014] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer program is stored in the computer-readable storage medium, and the computer program implements the above method when executed by a processor.

[0015] Beneficial effects: The target tracking method provided by the present invention can effectively deal with the time-varying characteristics and background noise problems in infrared small target tracking by combining the generation and updating of static templates and dynamic templates. First, the generated static target image template is cropped at multiple scales to improve the adaptability to different scales of the target. The generation of the dynamic target image template combines the dynamic template of the previous frame with the temporary template of the current frame, and is updated through a fusion strategy, so that the morphological changes and thermal radiation fluctuations of the target can be reflected in a timely manner, solving the response lag problem caused by the time-varying characteristics of the target. Finally, the static and dynamic templates are input into a pre-trained target tracking model for processing, which effectively suppresses the interference of complex background noise and accurately locates the target. This method significantly improves the accuracy and robustness of target tracking, especially in low signal-to-noise ratio and dynamically changing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 1 is a flow chart of a target tracking method provided by an embodiment of the present invention; Figure 2 is a block diagram of a target tracking model used in an embodiment of the present invention; Figure 3 is a block diagram of a feature extraction module used in an embodiment of the present invention; Figure 4 is a schematic diagram of a multi-scale static template generation process used in an embodiment of the present invention; Figure 5 : is a schematic diagram of tracking results in a target scale-changing motion scenario adopted in an embodiment of the present invention; wherein, Figure 5 The left picture in is a schematic diagram of the tracking result at the previous moment. Figure 5 The right picture in is a schematic diagram of the tracking result at the next moment; Figure 6 Schematic diagram of the tracking effect on a sequence of tiny and small targets adopted in an embodiment of the present invention; wherein, Figure 6 The left picture in is a schematic diagram of the tracking effect on a small target sequence. Figure 6 The right figure in is a schematic diagram of the tracking effect on a small target sequence; Figure 7 This is a block diagram of an electronic device used in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0018] With the rapid development of drone technology, the demand for infrared small target tracking in critical scenarios such as no-fly zone security monitoring and drone inspections is increasing. Infrared imaging systems, with their all-weather detection capabilities, have become a core sensing method in these areas. Infrared small target tracking, in particular, faces numerous challenges in complex low-altitude surveillance environments. The weak signal from target thermal radiation, interference from complex backgrounds, and dynamic target changes significantly limit the accuracy and stability of traditional infrared small target tracking methods. Therefore, addressing these technical issues is crucial to improving the performance of surveillance systems.

[0019] In related technologies, in the practical application of infrared small target tracking, the hardware architecture of the system can include three main parts: Ground-based infrared cameras: These can be deployed at fixed locations to continuously capture low-altitude video streams. Using images captured by infrared sensors, the system can monitor targets in diverse weather and environmental conditions. Infrared imagery is unaffected by changes in lighting and weather, ensuring 24 / 7 operation.

[0020] Processing Unit: This unit can execute target detection and tracking algorithms based on hardware acceleration units such as GPUs and TPUs. The processing unit is responsible for analyzing real-time captured video data and outputting information such as target location, size, and status.

[0021] Control Center: The control center can be responsible for receiving target tracking results and triggering alarms or interception mechanisms according to preset strategies.

[0022] However, despite the hardware architecture's high computing power and real-time processing capabilities, existing technologies still face algorithmic pain points, particularly in the detection and tracking of small infrared targets, where the target's thermal radiation intensity is low and the pixel area occupied by the target is very small. Further complicating matters, the target's thermal radiation intensity and size vary significantly as the distance from the infrared sensor changes.

[0023] A related technique involves using filtered templates for target tracking, such as MOSSE and DSST. These methods use a fixed template to match the target, but this approach is not adaptable to changes in target scale. When the target moves away from the camera, the fixed template may cover a large area of ​​background, causing tracking drift. Specifically, when the target moves farther away, the template no longer matches the target, resulting in erroneous tracking results.

[0024] Another technical solution uses a deep learning detector, such as an infrared-adapted version of YOLO. This method searches for targets within an image using a sliding window mechanism, but it often ignores the temporal correlation between frames. A target's thermal radiation can fluctuate between consecutive frames. In infrared imaging, this can cause flickering due to changes in equipment such as the engine. This can lead to missed targets, especially in complex scenes. These subtle fluctuations in heat sources often go undetected, affecting tracking accuracy.

[0025] In addition, some technical solutions can use twin network trackers, such as SiamRPN, which can distinguish between targets and backgrounds through shared convolution kernels. However, in practical applications, this method also has problems. The use of shared convolution kernels makes the boundary between the target and the background blurred, especially in low-contrast conditions. The thermal noise of the background (for example, thermal radiation from clouds or the ground) is easily confused with the characteristics of the target, resulting in the inability to accurately extract the target features, thus affecting the precise tracking of the target.

[0026] Understandably, the aforementioned technical issues often suffer from delayed response to time-varying characteristics and low thermal radiation intensity of small infrared targets. Specifically, the deformation and thermal radiation fluctuations of small infrared targets require that the tracking template be dynamically updated to cope with target changes. However, traditional linear template update mechanisms fail to respond to these changes in a timely manner, especially under occlusion conditions such as fog and clouds, where the cumulative effect of background noise further exacerbates tracking drift. This leads to misjudgment of target position, especially when the target changes rapidly. Existing technologies are unable to adjust the template in a timely manner, resulting in tracking failure. In complex backgrounds, the thermal radiation intensity of small infrared targets is often low. Traditional feature extraction networks typically rely on local convolution operations, which cannot effectively distinguish subtle differences between the target and the background. Due to the interference of background thermal noise, target features are difficult to distinguish effectively from the background, making precise positioning of the target very difficult, which in turn affects the stability and accuracy of tracking.

[0027] like Figure 1 and Figure 2As shown, in order to solve the above problems, this embodiment provides a target tracking method, the execution subject of the target tracking method can be a processing device, and the processing device can be an embedded processing unit, a GPU / TPU acceleration unit or a server. The method may include: Step S12: receiving video data carrying at least one target and corresponding target frame data.

[0028] In this embodiment, the processing device may receive video data carrying at least one target, and the video data may be captured by a ground infrared camera and sent to the processing device via a wired or wireless manner.

[0029] In this embodiment, the processing device may receive corresponding target frame data, which may be generated by a user through desktop operations. For example, based on the video data, the user may use a corresponding frame selection tool to select a target frame in the first frame of the video data to obtain the target frame data.

[0030] In a specific embodiment, the processing device can receive an infrared video stream (video data) captured in real time by a ground-based infrared camera via a wired or wireless communication interface. The infrared video stream can be transmitted using the H.265 encoding format to ensure real-time and high-definition target tracking. Wired transmission can be achieved using the Gigabit Ethernet protocol or GMSL2 data cables, while wireless transmission can be achieved using the 5G millimeter wave band or a low-latency WiFi6 link, ensuring reliable data transmission in complex deployment environments. To obtain target frame data, the user can operate through the human-computer interaction interface provided by the processing device. Specifically, the processing device can first decode and render the first frame of the video data and display the image in the display area of ​​the interactive interface. The user can manually select the target to be tracked on the displayed image using an input device (e.g., a mouse or touchscreen), forming a rectangular bounding box covering the target's outer contour. This selection operation can be directly applied to the image pixel coordinate system, with the upper left corner of the image as the coordinate origin (0, 0), the horizontal direction as the positive X axis, and the vertical direction as the positive Y axis. The processing device can capture the user's selection action in real time and convert it into structured target frame data.

[0031] In this embodiment, the video data carrying at least one target may be infrared video data, which may carry multiple targets, such as multiple drones. It is understood that in some cases, multiple drones may be flying in the sky.

[0032] In this embodiment, the target may be, in addition to drones, birds, airships, balloons, flying insects, and the like.

[0033] Step S14: generating static target image templates of at least two scales for the first image frame of the video data based on the target frame data.

[0034] In this embodiment, the static target image template can be represented as a set of fixed reference images generated by multi-scale cropping based on the first image frame of the video data and the target area determined by the user's selection operation. In other words, the static target image template is a set of image regions captured from the first frame after the original target area is expanded by a scaling factor, using the user-annotated bounding box in the first frame as a reference.

[0035] In this embodiment, the step of generating dynamic target image templates of at least two scales for the T-th image frame may include: performing multi-scale cropping on at least one target in the initial video frame of the video data based on the target frame data and a preset multi-scale factor to obtain dynamic target image templates of at least two scales.

[0036] In a specific embodiment, the processing device may first read multiple scaling factors (multi-scale factors) from a preset multi-scale factor set. The set may include at least two positive real-number scaling factors (e.g., 2.0, 3.0, and 4.0). These positive real-number scaling factors may represent the magnitude of the change in size relative to the original target frame. The processing device may then crop the target frame based on these scaling factors to generate static target image templates at multiple scales.

[0037] Specifically, before generating a static target image template, the processing device first needs to calculate the reference coordinates of the original target frame. The target frame data may include the coordinates of the four vertices of the target area in the image, representing the positions of the upper left corner and the lower right corner, respectively. The processing device can calculate the center point coordinates, width, and height of the original target frame based on these coordinates. The processing device then generates a new target frame for each scaling factor. The width and height of the new target frame are the width and height of the original target frame multiplied by the scaling factor, respectively. The center point of the new target frame is the same as the center point of the original frame and remains unchanged.

[0038] Through the above calculations, the processing device can obtain a new target frame and ensure that the center of the new target frame is aligned with the center of the original frame. The processing device can crop a rectangular area defined by the vertices of the newly calculated target frame from the first frame of the video data. This rectangular area corresponds to the static target image template at the current zoom factor K.

[0039] Specifically, the cropping process can be achieved by extracting the rectangular area from the video frame to form a target image template. The processing device saves the template as an independent image file in the lossless PNG compression format to ensure that the image quality is not affected by compression loss.

[0040] Ultimately, the processing device can generate a set of static target image templates at multiple scales. For example, three static target image templates of different sizes can be generated using scaling factors of 2.0, 3.0, and 4.0, respectively. These static target image templates can serve as initial references for subsequent tracking. By generating static templates at multiple scales, the processing device can effectively cover the range of possible target size variations, particularly when the target's position or distance within the image changes. These static templates will be used in subsequent target tracking processes and provide a foundation for the generation of dynamic templates.

[0041] It is understood that the static target image template is the initial template in the target tracking process, and its role is to provide a reference template for targets in subsequent frames. Through preliminary matching with the target, the static template can help determine the target's approximate location and characteristics. Because the target has distinct thermal radiation characteristics in infrared images, the static template can help distinguish the target from the background. This reference role of the static template is particularly prominent when the background contains complex noise (such as clouds, ground thermal radiation, etc.). In multi-target tracking, the static template can serve as a reference template for each target, helping the processing device to accurately match targets in complex environments and avoid misidentification due to background interference.

[0042] Step S16: Generate dynamic target image templates of at least two scales for the current T-th image frame; wherein the dynamic target image template of the T-th image frame is generated by fusing the temporary dynamic target image template of the T-th image frame and the dynamic target image template of the T-1-th image frame; wherein the temporary dynamic target image template of the T-th image frame is obtained by the corresponding target box data in the T-1-th image frame; wherein T is a positive integer greater than or equal to 3.

[0043] In this embodiment, during the target tracking process, dynamic target image templates (of the current frame) of at least two scales can be generated in real time. In other words, the dynamic target image templates are updated in real time.

[0044] In this embodiment, the processing device first reads the target frame data of the T-1 frame image. The target frame data of the T-1 frame image is derived from the predicted output of the target tracking model in the T-1 frame (that is, previously acquired). The target frame data includes the position and size information of the target in the T-1 frame. Based on this data, the processing device can calculate the center coordinates, width, and height of the target frame. Using these parameters, the processing device can determine the position of the target in the image and perform multi-scale cropping based on this. The processing device can perform a multi-scale cropping operation based on a preset multi-scale factor set (for example, including scaling factors such as 2.0, 3.0, 4.0, etc.). For each selected scaling factor k (for example, 2.0), the processing device can calculate the width and height of the new target frame.

[0045] In this embodiment, the processing device can then determine the vertex coordinates of the new target frame based on the newly calculated width and height and the center coordinates of the original target frame. Through the above calculations, the processing device can obtain the new target frame coordinates and crop a rectangular area from the T-th frame image based on the frame coordinates to generate a temporary dynamic target image template. The temporary dynamic target image template includes the appearance features of the target in the T-th frame image, and each scaling factor k corresponds to a separate image template. This operation is performed for each scaling factor to generate temporary templates corresponding to multiple scales.

[0046] In this embodiment, after generating a temporary dynamic target image template, the processing device can perform a fusion update of the dynamic template. That is, the following data processing is performed independently for each scale factor k: The processing device loads a dynamic target image template of the same scale as frame T-1 from memory. This template is historical data and contains the target image information from the previous frame. The processing device then performs a weighted fusion of this historical template (the dynamic target image template from frame T-1) and the temporary template (the temporary dynamic target image template) using a preset forgetting factor α. It should be understood that this weighted fusion is performed independently on each pixel channel, ensuring that each pixel value undergoes a weighted calculation, resulting in a smoother and more accurate target appearance update.

[0047] In this embodiment, in the process of generating the dynamic target image template, the processing device ensures that the template update processes of all scales are independent of each other. Specifically, the dynamic target image template of the Kth scale of the current Tth image frame is generated by the fusion of the temporary dynamic target image template of the Kth scale of the Tth image frame and the dynamic target image template of the Kth scale of the T-1th image frame. For example, the dynamic target image template of the scale factor 2.0 of the current frame is obtained by the fusion of the historical template and the temporary template of the 2.0 scale, that is, there is no cross-data dependency or weighted operation between templates of different scales. In other words, this independence guarantee ensures that the template updates of each scale will not interfere with each other, and the target features of each scale remain independent, avoiding confusion of target information at different scales.

[0048] In this implementation, the temporary dynamic target image template reflects the real-time observations of the target's appearance in the current frame, but may carry transient noise. The historical template (the dynamic target image template from the previous frame) accumulates target features from previous frames, providing greater stability. Through a weighted fusion mechanism, the processing device balances the freshness of the target with historical reliability, ensuring the continuous updating and accuracy of the target template.

[0049] In this embodiment, the step of generating dynamic target image templates of at least two scales for the current T-th image frame may include: first, based on the target frame data of the T-1-th image frame determined previously and a preset multi-scale factor, extracting a temporary dynamic target image template of the T-th image frame of at least two scales from the T-th image frame; wherein the target frame data of the T-1-th image frame is output by the target tracking model; then, using a preset exponentially weighted moving average method, weightedly fusion is performed on the dynamic target image template of at least two scales of the previously generated T-1-th image frame and the temporary dynamic target image template of at least two scales of the T-th image frame to obtain a dynamic target image template of at least two scales for the current T-th image frame; wherein the dynamic target image template corresponds to a first weight, and the temporary dynamic target image template corresponds to a second weight; wherein the first weight is a forgetting factor, and the second weight is one minus the forgetting factor.

[0050] In this embodiment, when T=2, that is, when the current frame is the second frame, the dynamic target image template of the second frame is actually produced by fusing the static target image template of the first image frame with the temporary dynamic target image template of the second frame. In this case, the static target image template of the first image frame serves as the dynamic target image template of the T-1th image frame mentioned above. It can be understood that when T=2, T-1 is 1, but the first image frame does not have a dynamic target image template, so the static target image template of the first image frame serves as the dynamic target image template (that is, the historical template) of the T-1th image frame.

[0051] Step S18: Input the static target image templates of at least two scales and the dynamic target image templates of at least two scales of the current T-th image frame into a pre-trained target tracking model to obtain target frame data.

[0052] The target tracking method provided in this embodiment can effectively deal with the time-varying characteristics and background noise problems in infrared small target tracking by combining the generation and updating of static templates and dynamic templates. First, the generated static target image template is cropped at multiple scales to improve the adaptability to different scales of the target. The generation of the dynamic target image template combines the dynamic template of the previous frame with the temporary template of the current frame, and is updated through a fusion strategy, so that the morphological changes and thermal radiation fluctuations of the target can be reflected in a timely manner, solving the response lag problem caused by the time-varying characteristics of the target. Finally, the static and dynamic templates are input into a pre-trained target tracking model for processing, which effectively suppresses the interference of complex background noise and accurately locates the target. This method significantly improves the accuracy and robustness of target tracking, especially in low signal-to-noise ratio and dynamically changing scenarios.

[0053] In some embodiments, the step of generating a dynamic target image template of at least two scales for the T-th image frame includes: Step S142: Based on the target frame data and a preset multi-scale factor, multi-scale cropping is performed on at least one target in the initial video frame of the video data to obtain a dynamic target image template of at least two scales.

[0054] In this embodiment, step S142 performs multi-scale cropping of the target from the initial video frame based on the target frame data and a preset multi-scale factor to generate dynamic target image templates of multiple scales. The beneficial effect of this step is that it can effectively capture the appearance characteristics of the target at different scales, especially when the target distance from the camera changes. By generating target templates of multiple scales, changes in target scale can be flexibly responded to during the target tracking process, avoiding tracking failures due to scale mismatch. This method enhances the adaptability of the tracking algorithm, so that high tracking accuracy and robustness can be maintained in complex scenarios, especially in situations with low signal-to-noise ratio and rapid target changes.

[0055] In some embodiments, the step of generating a dynamic target image template of at least two scales for the current T-th image frame includes: Step S162: Based on the previously determined target frame data of the T-1th image frame and a preset multi-scale factor, extracting a temporary dynamic target image template of the T-th image frame at least two scales from the T-th image frame; wherein the target frame data of the T-1th image frame is output by the target tracking model.

[0056] In this embodiment, the goal of step S162 is to generate a temporary dynamic target image template of multiple scales for the current T-th image frame. This process generates a temporary target image template of multiple scales through multi-scale cropping based on the target frame data of the T-1-th image frame and a preset multi-scale factor set for subsequent use. Specifically, first, the processing device obtains the target frame data of the T-1-th image frame. This data is the result output by the target tracking model in the processing cycle of the T-1-th frame, including the center coordinates, width, and height of the target. These data determine the position and size of the target frame in the T-1-th frame and serve as the basis for generating a temporary target image template for the current T-th image frame. Next, based on these target frame data and a preset multi-scale factor set (including at least two positive real number scaling coefficients, for example, 2.0, 3.0, 4.0, etc.), the processing device performs multi-scale cropping on the target frame. For each scale factor k, the processing device generates a new target frame by selecting a scaling factor k, calculating the width and height of the new target frame, maintaining the center coordinates unchanged, and then calculating the coordinates of the four vertices of the new target frame (the upper left vertex and the lower right vertex) based on the new width and height. Finally, the processing device can crop the target region from the Tth image frame based on the calculated vertex coordinates of the target frame. This region corresponds to the target image at the current scale factor k, forming a temporary dynamic target image template. For each scale factor, the processing device generates a separate target template, resulting in a set of target image templates at multiple scales.

[0057] It can be understood that the temporary dynamic target image template of each scale is stored independently, representing the image features of the target at different scales in the current frame.

[0058] Step S164: Using a preset exponentially weighted moving average method, weighted fusion is performed on the dynamic target image template of at least two scales of the previously generated T-1th image frame and the temporary dynamic target image template of at least two scales of the Tth image frame to obtain the dynamic target image template of at least two scales of the current Tth image frame; wherein, the dynamic target image template corresponds to a first weight, and the temporary dynamic target image template corresponds to a second weight; wherein, the first weight is a forgetting factor, and the second weight is one minus the forgetting factor.

[0059] In this implementation, template generation and fusion at all scales are performed in independent data processing flows. The template generation and fusion process at each scale has no computational dependencies across scales. For example, template updates for scale factor k1 do not affect processing for scale factor k2. This independence ensures that target features at different scales are independently extracted and updated, avoiding cross-interference between information at different scales.

[0060] In this implementation, in frames 1 and 2, the dynamic template is directly generated from the static template and serves as the reference template. Starting from frame 3, the target template is generated and updated through a weighted fusion of the template from frame T-1 and the temporary template from frame T. This progressive update mechanism ensures smooth template transitions and stability.

[0061] In this embodiment, the dynamic target image template of at least two scales of the previously generated T-1th image frame and the temporary dynamic target image template of at least two scales of the Tth image frame can be weightedly fused using the following formula:

[0062] In the formula, It is represented as the dynamic target image template after the k-th scale update of the t-th frame, that is, the dynamic target image template of the current frame. It is represented as the dynamic target image template of the kth scale in the t-1th frame, which is also the historical target image template. is the temporary dynamic template of the kth scale in the tth frame, k is the scale index, and α is the forgetting factor to better capture the dynamic changes of the target.

[0063] In this implementation, the forgetting factor controls the influence of historical information on template updates. When α is close to 1, the influence of historical information is stronger, making template updates smoother and more resistant to noise. When α is close to 0, the influence of current information is stronger, making the template more sensitive to real-time changes in the target.

[0064] In this embodiment, steps S162 and S164 effectively generate a dynamic target image template for the current T-th image frame by combining the target frame data of the T-1th frame and the Tth frame using the exponentially weighted moving average (EWMA) method. The beneficial effect of this step is that it can balance historical information with real-time information by weighted fusion of target features of historical frames and the current frame, thereby more accurately reflecting the dynamic changes of the target. In particular, by adjusting the forgetting factor, this method can flexibly control the weight of historical information, ensuring the timeliness and accuracy of template updates when the target changes rapidly or there is significant background interference. At the same time, multi-scale cropping enables the dynamic template to adapt to the different scales of the target, enhancing the robustness and stability of the tracking system. In particular, it can effectively suppress noise and improve tracking accuracy in complex environments and low signal-to-noise ratios.

[0065] like Figure 2 As shown, in some embodiments, the target tracking model includes a feature extraction module, a spatiotemporal cross-attention module and a tracking head module connected in sequence; wherein the feature extraction module is used to perform feature extraction on at least two scales of dynamic target image templates of the current T-th image frame, at least two scales of static target image templates and the current T-th image frame, respectively, to obtain dynamic target image template features, static target image template features and search area image features; the spatiotemporal cross-attention module is used to generate temporal attention features and spatial attention features based on the dynamic target image template, the static target image template and the T-th image frame; the tracking head module is used to generate target frame data based on the temporal attention features and spatial attention features.

[0066] In this embodiment, the target tracking model effectively improves the accuracy and robustness of target tracking by integrating a feature extraction module, a spatiotemporal cross attention module and a tracking head module. The feature extraction module ensures that the multi-dimensional information of the target is fully captured by extracting features from the dynamic target image template, the static target image template and the image of the current image frame, and provides high-quality feature input for subsequent target matching. The spatiotemporal cross attention module further processes these features to generate temporal attention features and spatial attention features, which can capture the temporal evolution relationship and spatial distribution characteristics of the target, effectively suppress background interference and enhance the expression of target features. Finally, the tracking head module generates precise target frame data based on these attention features, achieving stable and accurate target tracking, especially in complex environments with target scale changes and large background noise, showing strong adaptability and robustness. This structure greatly improves the performance of the tracking system in dynamic and complex scenes.

[0067] like Figure 3As shown, in some embodiments, the feature extraction module includes a Vision-Transformer encoder, which is used to perform the following operations on each input image: Split the input image into multiple non-overlapping image patches.

[0068] Each image patch is linearly projected into an embedding vector.

[0069] Based on the embedding vector, a learnable position embedding is added to obtain an embedding vector sequence.

[0070] In this embodiment, it can be understood that the above data processing process is a preprocessing operation performed by the Vision-Transformer encoder on the input image.

[0071] In a specific implementation, the Vision-Transformer encoder first receives an input image of size H×W (e.g., 224×224 pixels) and divides it into rectangular blocks of a fixed size (16×16 pixels). The division can be done by sliding a window starting from the top left corner with a horizontal stride of 16 pixels and a vertical stride of 16 pixels, resulting in (H / 16)×(W / 16) blocks (e.g., 14×14=196 blocks).

[0072] Then, each image patch is flattened to dimensions The one-dimensional vector of is then linearly projected through a trainable fully connected layer to obtain an embedding vector.

[0073] Finally, a trainable parameter matrix corresponding to the image patch position is generated, with a dimension of N × D. This matrix is ​​added element-by-element to the embedding vector to obtain an embedding vector sequence.

[0074] An encoding operation is performed on the embedding vector sequence to obtain dynamic target image template features, static target image template features and search area image features.

[0075] In this embodiment, the Vision-Transformer encoder encodes the embedding vector sequence through a multi-layer self-attention mechanism and a feedforward neural network. Through the self-attention mechanism, the model is able to establish associations between different image blocks and capture global context information. The embedding vector of each image block interacts with the vectors of other image blocks, thereby comprehensively analyzing the global features of the target image. The embedded vector sequence processed by the self-attention mechanism is input into the feedforward neural network for further feature extraction. The feedforward neural network is used to perform nonlinear transformations on the features of each image block, further enhancing the expressiveness of the features.

[0076] In this embodiment, the feature extraction module uses the Vision-Transformer (ViT) encoder, which can efficiently extract the multi-dimensional features of the target image by segmenting the input image, converting embedding vectors, embedding positions, and encoding operations. The image is divided into multiple non-overlapping image blocks, and each image block is converted into an embedding vector through linear projection, ensuring efficient representation of the target image information. Adding learnable position embedding further enhances the expression of spatial position, enabling the model to better understand the relative position and contextual relationship of each part of the image. On this basis, the encoding operation is performed, and the processing device can extract dynamic target image template features, static target image template features, and search area image features, providing rich, context-related feature information. This feature extraction method can significantly improve the model's ability to capture target details and effectively enhance the robustness of target recognition and tracking, especially in environments with complex backgrounds and target changes, showing extremely high adaptability and accuracy.

[0077] In some embodiments, the feature extraction module includes a context enhancement module, which is connected after the Vision-Transformer encoder and includes: The feature input layer is used to receive the static target image template features.

[0078] In this implementation, the feature input layer receives the static target image template features output by the Vision-Transformer (ViT) encoder and prepares these feature data for subsequent processing. The feature dimensions are N×C, where N represents the number of image tokens and C represents the feature channel dimension of each image token (e.g., 196×768).

[0079] The mask prediction subnetwork is used to generate soft mask features.

[0080] In this embodiment, the mask prediction subnetwork is mainly used to generate soft mask features to guide the enhancement of static target image template features. The mask prediction subnetwork can be composed of two layers of 1×1 convolution and a Sigmoid activation function. The specific data processing performed by it can be: First, a 1×1 convolution kernel is used to process the input N×C dimensional features, outputting intermediate features. The convolution kernel parameter dimensions are C×D×1×1, where D is the number of intermediate channels. This layer uses standard random initialization, and the convolution kernel weights are updated via backpropagation during training. After convolution, a ReLU activation function is immediately applied to introduce nonlinearity.

[0081] Next, the intermediate features are further processed by the second layer of 1×1 convolution to output a single-channel feature map. The parameter dimension of the convolution kernel is D×1×1×1, and the initial value of the bias term is 0.

[0082] Finally, the Sigmoid activation function is applied to the single-channel features to obtain the soft mask matrix Mce, whose dimension is N × 1. Each element of this matrix corresponds to the saliency weight of an image block area, with a value range of [0, 1], indicating the difference in importance between the target area and the background area.

[0083] A feature weighting unit is used to perform element-wise multiplication of the soft mask feature and the static target image template feature to perform context enhancement on the static target image template feature.

[0084] In this embodiment, the feature weighting unit receives the static target image template feature (with dimensions of N×C) and the soft mask feature Mce (with dimensions of N×1), and the specific data processing performed by the feature weighting unit may be: First, to make the mask feature consistent with the dimension of the static target image template feature, the one-dimensional mask vector Mce can be expanded along the feature channel direction to become N×C dimensions, matching the dimension of the static feature. This operation ensures that the mask value of each image block can be weightedly calculated across all channels.

[0085] The expanded mask feature Mce is then element-wise multiplied with the static target image template feature. This calculation is performed on each token in the image, and the features of each image block are weighted according to the corresponding mask value. The result is the enhanced static target image template feature, with a dimension of N × C.

[0086] Finally, the generated enhanced features are stored in a new memory area and overwrite the original static target image template features. In this way, the context-guided features can be used in subsequent processing.

[0087] In this embodiment, during the training phase, in order to ensure that the network learns a valid mask, the mask value can be constrained as follows: During the training process, the token of the target area corresponds to the image block covered by the annotation box in the first frame, and the mask labels of these image blocks are set to 1.0, indicating that these areas contribute the most to the target features. The token of the background area corresponds to the image block outside the annotation box, and the mask label is set to a random value in the range of 0.1 to 0.3. The mask value in this interval ensures that the influence of the background area is effectively suppressed, thereby avoiding the interference of background noise on the target features. During the back propagation process, the Sigmoid function of the output layer forces the feature weights of the background area to be maintained in the range of 0.1 to 0.3, thereby further strengthening the expression of the target area features and suppressing the influence of background noise.

[0088] In this embodiment, the context enhancement module significantly improves the expressiveness of the target features by performing context enhancement on the static target image template features. Specifically, by element-by-element multiplication of the soft mask features generated by the mask prediction subnetwork and the static target image template features, the model can effectively suppress background noise and enhance the features of the target area. This context enhancement processing method can more accurately identify and highlight the key features of the target area, especially when the background around the target is complex or the signal-to-noise ratio is low, and can significantly improve the accuracy of target recognition and tracking. Through this enhancement method, the feature extraction module can provide a clearer and more stable target representation, further improving the robustness and accuracy of the entire tracking system.

[0089] In some implementations, the spatiotemporal crisscross attention module includes: The temporal attention submodule is used to receive static target image template features and dynamic target image template features, and then generate temporal attention features through temporal attention calculation; wherein, the temporal attention features are used to model the temporal evolution relationship between the dynamic target image template and the static target image template.

[0090] In this implementation, the static and dynamic template features are transformed using a linear projection layer to generate three matrices: a query matrix Q, a key matrix K, and a value matrix V. These matrices are all of N×C dimensions, and the transformations are performed using a trainable parameter matrix. The query matrix Q is obtained by linearly projecting the static template features, while the key matrix K and the value matrix V are obtained by linearly projecting the dynamic template features.

[0091] In this embodiment, the temporal attention feature can be calculated using the following formula:

[0092] In the formula, Represented as static target image template features, Represented as dynamic target image template features, It is expressed as normalized weight and C is the feature dimension.

[0093] The spatial attention submodule is used to receive static target image template features, dynamic target image template features and search area image features, then splice the static target image template features with the dynamic target image template features to obtain spliced ​​features, and then perform spatial attention calculation on the spliced ​​features and the search area image features to obtain spatial attention features; wherein, the spatial attention features are used to enhance the spatial matching ability between the template and the search area.

[0094] In this implementation, the static target template features and the dynamic target template features are first concatenated to form a fused template feature. The concatenated feature matrix has a dimension of 2N × C, where 2N represents the total number of concatenated tokens and C is the dimension of the feature channels. Simultaneously, the spatial attention submodule also receives search region features of dimension M × C, where M is the number of image patches in the search region.

[0095] Then, the spatial attention submodule performs linear projection on the concatenated template features and search area features to generate the query matrix Q, key matrix K, and value matrix V. The query matrix Q is obtained by linear projection on the concatenated template features, while the key matrix K and value matrix V are obtained by linear projection on the search area features.

[0096] In this embodiment, the spatial attention feature can be calculated and generated by the following formula:

[0097] In the formula, Represented as the search area image feature, It is represented as concatenating the static target image template features with the dynamic target image template features.

[0098] In this embodiment, the spatiotemporal cross attention module significantly improves the accuracy and robustness of target tracking through the collaborative work of the temporal attention submodule and the spatial attention submodule. The temporal attention submodule effectively captures the changes of the target in different time frames by modeling the temporal evolution relationship between the static target image template features and the dynamic target image template features, thereby ensuring a smooth transition and precise tracking of the target state. The spatial attention submodule enhances the spatial matching ability between the template and the search area by splicing the static and dynamic target image template features and calculating the spatial attention with the search area image features, reduces the interference of background noise, and improves the accuracy and stability of target positioning. Overall, the module strengthens the adaptability of the target to dynamic changes and complex environments through the two-dimensional attention mechanism of time and space, thereby significantly improving the tracking performance and robustness of the overall system.

[0099] In some embodiments, the tracking head module includes: The feature reconstruction unit is used to reconstruct the features input by the spatiotemporal cross attention module into a two-dimensional feature map.

[0100] In one specific implementation, the feature reconstruction unit rearranges the one-dimensional feature sequence into a two-dimensional grid structure, generating a feature map with a size of 14×14 pixels and C channels. Specifically, the feature reconstruction unit can map the original feature sequence to a 14×14 grid based on spatial position index, with each grid point corresponding to a C-dimensional feature vector. The grid arrangement order follows the spatial topology of the original image blocks (top left to bottom right).

[0101] The scoring head branch is used to generate a confidence map representing the probability of each location being the target center based on the two-dimensional feature map.

[0102] In this embodiment, the splitting head branch may include a three-layer convolutional network, specifically: The first convolutional layer uses a 3×3 convolution kernel, input channel C, output channel D (D=C / 4), and performs a ReLU activation function after convolution.

[0103] The second convolutional layer uses a 3×3 convolution kernel, input channel D, output channel D, and performs a ReLU activation function after convolution.

[0104] The third convolutional layer uses a 1×1 convolution kernel, input channel D, and output channel 1.

[0105] The final output is a 14×14 confidence map, where each pixel value represents the probability of that location being the center of the target. The mapping value range [0, 1] is constrained by the Sigmoid function.

[0106] The offset prediction branch is used to generate an offset of the target center point relative to the feature map coordinates based on the two-dimensional feature map.

[0107] In this embodiment, the offset prediction branch may include a four-layer convolutional network, specifically: Convolutional layer 1, using 3×3 convolution, input channels C, output channels C / 2, ReLU activation; Convolutional layer 2 uses 3×3 convolution, input channels C / 2, output channels C / 4, and ReLU activation; Convolutional layer 3 uses 3×3 convolution, input channels C / 4, output channels C / 8, and ReLU activation; The output layer uses 1×1 convolution, input channel C / 8, and output channel 2. Channel 0 is represented by the horizontal offset Δx, and channel 1 is represented by the vertical offset Δy. The offset represents the position deviation of the target center relative to the center of the feature map grid.

[0108] The size estimation branch is used to generate the object width and height based on the two-dimensional feature map.

[0109] In this embodiment, the size estimation branch structure is consistent with the offset prediction branch.

[0110] The fusion unit is used to determine the center position of the candidate target based on the confidence map, correct the center position of the candidate target based on the offset, and generate the final target box data based on the target width and height.

[0111] In this embodiment, the fusion unit combines the outputs of the scoring branch, the offset prediction branch, and the size estimation branch to generate the final target bounding box data. First, the confidence heatmap generated by the scoring branch is used to determine the center position of the candidate target. Next, the offset prediction branch corrects the position of the candidate target center. Finally, the target width and height information output by the size estimation branch are combined to generate the final target bounding box.

[0112] In this embodiment, the tracking head module significantly improves the accuracy and robustness of target tracking through the collaborative work of the feature reconstruction unit, the scoring head branch, the offset prediction branch, the size estimation branch and the fusion unit. First, the feature reconstruction unit reorganizes the features from the spatiotemporal cross attention module into a two-dimensional feature map, providing global and local information for subsequent target positioning. Through the confidence map generated by the scoring head branch, the model can accurately calculate the position probability of the target center, while the offset prediction branch corrects the precise position of the target center by analyzing the feature map. The size estimation branch further provides the width and height of the target, thereby accurately predicting the size of the target. Finally, the fusion unit synthesizes this information, determines the candidate target center position and corrects the position according to the offset, and generates the final target frame data in combination with the size of the target. By comprehensively considering the position information, size and probability of the target, this module not only improves the target positioning accuracy, but also enhances the adaptability to rapid changes of targets in complex backgrounds.

[0113] In a specific embodiment, a Transformer infrared small target tracking algorithm based on a multi-scale dynamic template is provided.

[0114] Tracking small infrared targets in complex low-altitude drone monitoring scenarios presents numerous challenges. Traditional methods rely on a fixed template in the initial frame. In this scenario, small infrared targets are susceptible to time-varying thermal radiation, resulting in grayscale drift. Furthermore, the target resolution is low (averaging approximately 8 pixels × 8 pixels), leading to multi-scale representation ambiguity. Furthermore, the single-channel grayscale nature of infrared images exacerbates the interference of similar objects on target tracking. Existing Transformer-based tracking algorithms, with their static templates, cannot effectively capture the dynamic deformation of infrared targets, struggle to address edge blurring caused by thermal diffusion, and lack effective integration of multi-scale contextual information. While Siamese networks offer strong real-time performance, their offline feature matching strategies struggle to adapt to the challenges of weakly textured scenes in infrared images. Dynamic template methods also fail to fully incorporate the unique spatiotemporal prior knowledge of infrared images.

[0115] This specific implementation plan addresses the shortcomings of existing technologies in infrared small target tracking in complex low-altitude UAV monitoring scenarios and provides a Transformer infrared small target tracking algorithm based on multi-scale dynamic templates.

[0116] This specific implementation plan captures the changes in target thermal radiation by constructing a multi-scale dynamic template group, combines the spatiotemporal cross-attention mechanism to suppress background noise, solves the problems of fixed template failure and context ambiguity in infrared small target tracking, realizes stable tracking of infrared small targets in complex scenes, and improves tracking accuracy and robustness.

[0117] The Transformer infrared small target tracking algorithm based on multi-scale dynamic template provided in this implementation includes the following steps: Step 1: Construct a multi-scale dynamic template generation module.

[0118] Step 1.1: In the first frame, based on the given target frame, crop the target area with different scale factors (such as 2.0, 3.0, 4.0) to generate multiple static templates to cover the different scale ranges where the target may appear, and to deal with the scale uncertainty of small infrared targets caused by factors such as distance changes. Figure 4 As shown, Figure 4 The multi-scale static template generation process is demonstrated, and the method of generating multiple static templates by cropping the target area with different scale factors in the first frame image is intuitively presented.

[0119] Step 1.2: During the target tracking process, the target box predicted in the previous frame is used as the center, and a dynamic template is generated by cropping with the same scale factor, and then updated using the exponentially weighted moving average (EWMA) method.

[0120]

[0121] In the formula, is the dynamic template updated at the kth scale in the tth frame, is the historical dynamic template of the kth scale in the t-1th frame, is the temporary dynamic template of the kth scale in the tth frame, k is the scale index, and α is the forgetting factor to better capture the dynamic changes of the target.

[0122] Step 2: Design feature extraction and encoding module.

[0123] Step 2.1: Perform standardization preprocessing on the static template, dynamic template, and search area images, including resizing the images to a uniform size of 224 × 224 pixels and normalizing them to eliminate differences in brightness, contrast, etc. between images.

[0124] Step 2.2: Use the VisionTransformer (ViT) architecture as the core feature extraction tool. The pre-processed image is finely segmented into multiple 16×16 pixel image blocks, converted into a one-dimensional vector and embedded with learnable position information. The vector is then input into the ViT network and processed by a multi-layer self-attention mechanism and a feedforward neural network to extract the static template feature Zs, the dynamic template feature Zd, and the search area feature Zx. Step 2.3: Integrate the contextual enhancement module into the static template feature extraction process. The contextual enhancement module mainly consists of three parts: a feature input layer, a mask prediction subnetwork, and a feature weighting layer, forming a complete context-guided feature enhancement structure. The feature input layer receives the initial static template features output by the Visual Transformer (ViT) with a dimension of N×C, where N represents the number of tokens and C represents the number of feature channels. The mask prediction subnetwork consists of two layers of 1×1 convolutions (with a ReLU activation function in the middle layer) and outputs a soft mask Mce with a dimension of N×1 that matches the number of tokens through a Sigmoid activation layer. The feature weighting layer performs element-wise multiplication of the initial static template features with the soft mask Mce to obtain the context-guided enhanced features Zs. Subsequently, the mask prediction subnetwork is guided by the annotation information of the target box in the first frame to learn, assigning a mask value close to 1 to the token corresponding to the target area, and assigning a mask value between 0.1 and 0.3 to the background area token (this value range is constrained by the Sigmoid layer), thereby effectively suppressing background interference features, enhancing the feature expression ability of the target area, and realizing context enhancement processing of static template features.

[0125] Step 3: Apply the spatiotemporal cross attention module.

[0126] The spatiotemporal cross attention module aims to jointly model the temporal dependency between template frames and the spatial correlation between the template and the search area. Its network structure consists of two sub-modules: the inter-template temporal attention sub-module and the template-search area spatial attention sub-module.

[0127] Step 3.1: The inter-temporal attention submodule takes the static template feature Zs and the dynamic template feature Zd as input. First, it passes through the linear projection layer and uses the parameter matrix to obtain the query matrix, key matrix, and value matrix, all of which are N×C dimensions. Subsequently, in the attention score calculation layer, the attention score matrix is ​​calculated and normalized by Softmax to obtain the weight matrix. Finally, the temporal attention feature output is obtained through the weighted fusion layer to obtain the inter-temporal attention feature Ats. The formula is:

[0128] In the formula, C is the feature dimension, which is used to capture the temporal evolution relationship between the static template and the dynamic template, and use the dynamic template information to supplement and update the static template.

[0129] Step 3.2: The template-search region spatial attention submodule takes the concatenated template features (the static and dynamic templates are concatenated in the token dimension) obtained by fusing the static template features Zs and the dynamic template features Zd, and the search region features Zx as input. The query matrix, key matrix, and value matrix are projected through linear transformation matrices, respectively. Subsequently, in the attention score calculation layer, the spatial attention score is calculated and normalized by Softmax to obtain the attention weight matrix. Finally, the spatial attention output is obtained through the weighted fusion layer to obtain the template-search region spatial attention feature Asx, which is formulated as follows:

[0130] It is used to enhance the spatial response matching capability between template information and the current search area, thereby improving target recognition and positioning accuracy.

[0131] Step 4: Build the tracking head module: The tracking head module is used to achieve final target positioning and selection. It mainly consists of three parts: the scoring head, the offset prediction branch, and the size estimation branch. Each sub-module works together to complete high-precision target position regression and scale prediction.

[0132] Step 4.1: The scoring head network structure consists of two layers of 3×3 convolutions (with the middle layer using the ReLU activation function) and one layer of 1×1 convolution, with an output channel of 1. The implementation process is as follows: First, the features related to the search area from the spatiotemporal crisscross attention module are rearranged into a 14×14×C feature map. A convolution operation then generates a 14×14 confidence map. This map is matched to the true label (constructed as a two-dimensional Gaussian distribution with the object center as the mean and a standard deviation of 1 / 5 the object size). The scoring head selects the features related to the search area from the feature set output by the spatiotemporal crisscross attention mechanism and reconstructs them into a 14×14 pixel two-dimensional feature map, with each pixel containing C feature channels. The confidence branch generates a 14×14 pixel confidence map, reflecting the probability of each pixel being the object center. A Gaussian distribution function is used to generate the true label of the object center, resulting in a coarse location and score map, reducing localization error.

[0133] Step 4.2: The offset prediction branch uses three layers of 3×3 convolutions (the number of channels is halved at each layer) and one layer of 1×1 convolution. The final output channel is 2, corresponding to the horizontal and vertical offsets (Δx, Δy) of the target center point. This branch regresses the offset values ​​pixel by pixel and optimizes them using the L1 loss function to improve the center position regression accuracy.

[0134] Step 4.3: The size estimation branch has the same network structure as the offset prediction branch, with two output channels, corresponding to the width w and height hh of the target. Considering that small infrared targets typically have stable aspect ratios, this branch is trained using the GIoU loss function to enhance robust estimation of target size.

[0135] Finally, the tracking head module fuses the outputs of the three branches: first, the scoring head generates a confidence heat map to obtain the candidate center position, then the offset branch corrects the position, and finally, combined with the width and height information output by the size estimation branch, the complete target bounding box is regressed to achieve accurate tracking of small infrared targets. The specific tracking results are as follows: Figure 5 and Figure 6 shown.

[0136] Step 5: Set the training and inference related parameters and perform experimental verification.

[0137] Step 5.1: Set the overall loss function The overall loss function is composed of classification loss, offset loss and size loss through their respective weight coefficients, and the formula is:

[0138] In the formula, 、 as well as are the weight coefficients of classification loss, offset loss and size loss respectively.

[0139] The classification loss uses FocalLoss to alleviate the imbalance problem of positive and negative samples. The formula is:

[0140] In the formula, p is the target probability predicted by the model, a is the weight coefficient for balancing positive and negative samples, which is usually set to 0.25, and γ is the adjustment factor, which is usually set to 2.

[0141] The offset loss adopts L1 loss, and the formula is:

[0142] In the formula, and is the target center offset predicted by the model, and is the actual target center offset.

[0143] The size loss uses the generalized intersection-over-union (GIoU) ​​loss to better reflect the change in target size. The formula is:

[0144] In the formula, is the target box predicted by the model, and b is the actual target box.

[0145] Step 5.2: During the training phase, the LaSOT large-scale single target tracking dataset and the AntiUAV600 low-altitude UAV perception dataset are jointly trained. LaSOT is used to improve the model's ability to generalize the apparent features of general targets, and AntiUAV600 is used to enhance its adaptability to specific infrared scenes of low-altitude UAVs. During the testing phase, only the AntiUAV600 test set is used to evaluate the algorithm performance.

[0146] Step 5.3: Evaluate the algorithm's performance using three metrics: precision, success rate, and processing speed. Precision evaluates the tracking algorithm's positioning accuracy by quantifying the absolute positional deviation between the predicted target center and the true target center, calculated using the center position error (CLE). Success rate evaluates the overall degree of match between the predicted and true bounding boxes by calculating the intersection over union (IoU) ratio. Processing speed refers to the number of frames per second (FPS) processed by the algorithm. Ablation and comparative experiments were conducted on the AntiUAV600 dataset, and the experimental results were analyzed to verify the effectiveness of the algorithm.

[0147] The algorithm in this implementation effectively addresses key issues such as low contrast, time-varying characteristics, and multi-scale representation ambiguity in infrared small targets. On the AntiUAV600 dataset, the algorithm achieved a tracking success rate and positioning accuracy of 0.569 and 0.664, respectively, with a processing frame rate of 20.7 FPS. This meets the real-time tracking requirements of low-altitude surveillance, providing an innovative solution for infrared small target tracking and promising application potential.

[0148] According to an embodiment of the present invention, an electronic device is provided. Figure 7 The electronic device in this embodiment may include one or more of the following components: a processor, a network interface, a memory, a non-volatile memory, and one or more applications, wherein the one or more applications may be stored in the non-volatile memory and configured to be executed by one or more processors, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0149] According to an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the computer executes the method described in any one of the above embodiments.

[0150] According to an embodiment of the present invention, a computer program product comprising instructions is further provided. When the instructions are executed by a computer, the computer is caused to perform a method in any one of the above embodiments.

[0151] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0152] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0153] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0154] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0155] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A target tracking method, characterized in that: include: Receiving video data carrying at least one target and corresponding target frame data; generating static target image templates of at least two scales for a first image frame of the video data based on the target frame data; Generate dynamic target image templates of at least two scales for the current T-th image frame; wherein the dynamic target image template of the T-th image frame is generated by fusing the temporary dynamic target image template of the T-th image frame and the dynamic target image template of the T-1-th image frame; wherein the temporary dynamic target image template of the T-th image frame is obtained from the corresponding target frame data in the T-1-th image frame; wherein T is a positive integer greater than or equal to 3; The static target image templates of at least two scales and the dynamic target image templates of at least two scales of the current T-th image frame are input into a pre-trained target tracking model to obtain target frame data.

2. The method according to claim 1, characterized in that The step of generating a dynamic target image template of at least two scales for the T-th image frame includes: Based on the target frame data and a preset multi-scale factor, multi-scale cropping is performed on at least one target in an initial video frame of the video data to obtain a dynamic target image template of at least two scales.

3. The method according to claim 1, characterized in that The step of generating a dynamic target image template of at least two scales for the current T-th image frame includes: Extracting a temporary dynamic target image template of the T-th image frame at at least two scales from the T-th image frame based on previously determined target frame data of the T-1-th image frame and a preset multi-scale factor; wherein the target frame data of the T-1-th image frame is output by the target tracking model; A preset exponentially weighted moving average method is used to weightedly fuse the dynamic target image template of at least two scales of the previously generated T-1th image frame with the temporary dynamic target image template of at least two scales of the Tth image frame to obtain the dynamic target image template of at least two scales of the current Tth image frame; wherein, the dynamic target image template corresponds to a first weight, and the temporary dynamic target image template corresponds to a second weight; wherein, the first weight is a forgetting factor, and the second weight is one minus the forgetting factor.

4. The method according to claim 1, wherein The target tracking model includes a feature extraction module, a spatiotemporal cross-attention module and a tracking head module connected in sequence; wherein the feature extraction module is used to perform feature extraction on at least two scales of dynamic target image templates of the current T-th image frame, at least two scales of static target image templates and the current T-th image frame, respectively, to obtain dynamic target image template features, static target image template features and search area image features; the spatiotemporal cross-attention module is used to generate temporal attention features and spatial attention features based on the dynamic target image template, the static target image template and the T-th image frame; the tracking head module is used to generate target frame data based on the temporal attention features and spatial attention features.

5. The method according to claim 4, characterized in that The feature extraction module includes a Vision-Transformer encoder, which is used to perform the following operations on each input image: Split the input image into multiple non-overlapping image blocks; Linearly project each image patch into an embedding vector; Adding a learnable position embedding based on the embedding vector to obtain an embedding vector sequence; An encoding operation is performed on the embedding vector sequence to obtain dynamic target image template features, static target image template features and search area image features.

6. The method according to claim 5, characterized in that The feature extraction module includes a context enhancement module, which is connected after the Vision-Transformer encoder and includes: A feature input layer, which is used to receive static target image template features; The mask prediction subnetwork is used to generate soft mask features; A feature weighting unit is used to perform element-wise multiplication of the soft mask feature and the static target image template feature to perform context enhancement on the static target image template feature.

7. The method according to claim 4, characterized in that The spatiotemporal crisscross attention module includes: A temporal attention submodule, which is used to receive static target image template features and dynamic target image template features, and then generate temporal attention features through temporal attention calculation; wherein the temporal attention features are used to model the temporal evolution relationship between the dynamic target image template and the static target image template; The spatial attention submodule is used to receive static target image template features, dynamic target image template features and search area image features, then splice the static target image template features with the dynamic target image template features to obtain spliced ​​features, and then perform spatial attention calculation on the spliced ​​features and the search area image features to obtain spatial attention features; wherein, the spatial attention features are used to enhance the spatial matching ability between the template and the search area.

8. The method according to claim 4, characterized in that The tracking head module includes: A feature reconstruction unit, which is used to reconstruct the features input by the spatiotemporal cross attention module into a two-dimensional feature map; The scoring head branch is used to generate a confidence map representing the probability of each location being the target center based on the two-dimensional feature map; An offset prediction branch is used to generate an offset of a target center point relative to a feature map coordinate based on the two-dimensional feature map; A size estimation branch, configured to generate target width and height based on the two-dimensional feature map; The fusion unit is used to determine the center position of the candidate target based on the confidence map, correct the center position of the candidate target based on the offset, and generate the final target frame data based on the target width and height.

9. An electronic device, characterized in that: include: a memory, and one or more processors communicatively coupled to the memory; Instructions executable by the one or more processors are stored in the memory. The instructions are executed by the one or more processors to enable the one or more processors to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images