Target tracking method, target tracking device, and computer-readable storage medium

Abstract

This application relates to the field of target tracking technology, and discloses a target tracking method, a target tracking device, and a computer-readable storage medium. Multiple tracking stages are pre-set, each with a verification strategy and tracking stage transition conditions. The method includes: tracking a target in a video stream using a tracker, analyzing the current image frame to determine current category information and current tracking state information, obtaining the tracking stage corresponding to the previous image frame to obtain a first tracking stage, determining a second tracking stage based on the current category information, current tracking state information, and the tracking stage transition conditions corresponding to the first tracking stage, verifying the tracked target using the verification strategy corresponding to the second tracking stage, and correcting the tracker based on the verification results. Through this method, state awareness during the tracking process is achieved, and processing modes are intelligently switched, thereby ensuring the accuracy of target tracking.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, specifically to a target tracking method, a target tracking device, and a computer-readable storage medium. Background Technology

[0002] Target tracking refers to the technique of continuously locating, associating, and outputting the motion trajectory of a specified target in consecutive frames of a video. Current target tracking methods always use the relevant information detected in the first frame of the image during the tracking process. They cannot update the target category as the target approaches, nor can they verify the target during the tracking process.

[0003] This results in poor target tracking accuracy. When similar interfering objects (such as birds, balloons, etc.) overlap or obstruct the target (such as drones) during the tracking process, the tracker may drift to non-target objects, and the system cannot detect the mistracking, resulting in continuous locking of the wrong target and loss of the target. Summary of the Invention

[0004] In view of the above problems, embodiments of this application provide a target tracking method, a target tracking device, and a computer-readable storage medium to solve the problem of poor accuracy in target tracking in the prior art.

[0005] According to one aspect of the embodiments of this application, a target tracking method is provided, which pre-sets multiple tracking stages, each tracking stage having a corresponding verification strategy and tracking stage transition conditions. The method includes: tracking a target in a video stream using a tracker, and analyzing the current image frame of the video stream to determine the current category information and current tracking state information of the target in the current image frame; obtaining the tracking stage corresponding to the previous image frame to obtain a first tracking stage; determining a second tracking stage corresponding to the current image frame based on the current category information, the current tracking state information, and the tracking stage transition conditions corresponding to the first tracking stage; verifying the target in the current image frame using the verification strategy corresponding to the second tracking stage, and correcting the tracker based on the verification result.

[0006] In one optional approach, the current image frame of the video stream is analyzed to determine the current category information and current tracking status information of the tracked target in the current image frame. Specifically, this includes: identifying the current image frame to determine a first category of the tracked target in the current image frame; obtaining a second category of the tracked target based on a first preset number of historical image frames, wherein the historical image frames are image frames in the video stream that precede the current image frame and are consecutive to the current image frame, and the second category is obtained by identifying the historical image frames; and determining the current category information of the tracked target in the current image frame based on the first category and the second category.

[0007] In an alternative approach, before tracking a target in a video stream using a tracker, the method further includes: acquiring the video stream and performing target detection on the video stream to determine the target in the video stream, as well as the initial category information and initial tracking state information of the target; setting the first tracking phase corresponding to the video stream as the start period; and initializing the tracker based on the initial category information and the initial tracking state information.

[0008] In one optional approach, the tracking phase includes an initiation phase and a stabilization phase. The second tracking phase corresponding to the current image frame is determined based on the current category information, the current tracking state information, and the tracking phase transition conditions corresponding to the first tracking phase. Specifically, if the first tracking phase is the initiation phase, the following steps are performed: determining the number of tracking frames for the tracker and judging whether the number of tracking frames is greater than a second preset number to obtain a first judgment result; acquiring historical category information and historical tracking state information of the tracked target based on a third preset number of historical image frames; judging whether the historical category information and the current category information meet the first preset category requirements to obtain a second judgment result; judging whether the historical tracking state information and the current tracking state information meet the first preset tracking state requirements to obtain a third judgment result; if the first judgment result, the second judgment result, and the third judgment result are all yes, then the second tracking phase is the stabilization phase.

[0009] In one optional approach, the tracking phase includes a stabilization period and a distance-away period. The second tracking phase corresponding to the current image frame is determined based on the current category information, the current tracking status information, and the tracking phase transition conditions corresponding to the first tracking phase. Specifically, if the first tracking phase is a stabilization period, the following steps are performed: obtaining historical tracking status information of the tracked target based on a fourth preset number of historical image frames; determining whether the tracked target in the historical image frames and the current image frame is continuously shrinking based on the historical tracking status information and the current tracking status information; if so, the second tracking phase is a distance-away period.

[0010] In one optional approach, the tracking phase includes a stabilization period and a distance-away period. The second tracking phase corresponding to the current image frame is determined based on the current category information, the current tracking status information, and the tracking phase transition conditions corresponding to the first tracking phase. Specifically, if the first tracking phase is a distance-away period, the following steps are performed: Historical category information and historical tracking status information of the tracked target are obtained from a fifth preset number of historical image frames; based on the historical tracking status information and the current tracking status information, it is determined whether the tracked target in the historical image frames and the current image frame continues to increase, resulting in a fourth determination; it is determined whether the historical category information and the current category information meet the second preset category requirements, resulting in a fifth determination; it is determined whether the historical tracking status information and the current tracking status information meet the second preset tracking status requirements, resulting in a sixth determination; if the fourth, fifth, and sixth determination results are all yes, then the second tracking phase is a stabilization period.

[0011] In one alternative approach, the tracking phase includes an initiation period; the tracking target in the current image frame is verified using the verification strategy corresponding to the second tracking phase, and the tracker is corrected based on the verification result, specifically including: if the second tracking phase is the initiation period, the tracker is corrected based on the current category information.

[0012] In one optional approach, the tracking phase includes an initiation phase and a stabilization phase. The tracking target in the current image frame is verified using the verification strategy corresponding to the second tracking phase, and the tracker is corrected based on the verification result. Specifically, if the second tracking phase is a stabilization phase, the following steps are performed: if the first tracking phase is an initiation phase, the current category information is determined as the target category information; the historical category information of the tracking target is obtained based on a sixth preset number of historical image frames; if both the current category information and the historical category information do not match the target category information, a re-detection is triggered to re-initialize the tracker.

[0013] In one optional approach, the tracking phase includes a distance period, and the tracking status information includes tracking confidence. The tracking target in the current image frame is verified using the verification strategy corresponding to the second tracking phase, and the tracker is corrected based on the verification result. Specifically, if the second tracking phase is a distance period, the historical tracking status information of the tracking target is obtained based on a seventh preset number of historical image frames. If the tracking confidence in the historical tracking status information is lower than a preset threshold, re-detection is triggered.

[0014] According to another aspect of the embodiments of this application, a target tracking device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the target tracking method described in any of the preceding claims.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the target tracking method described in any of the preceding claims.

[0016] This application embodiment achieves state awareness during the tracking process by pre-setting multiple tracking stages and their corresponding transition conditions and verification strategies. Furthermore, the tracking system can intelligently switch processing modes based on whether the target is approaching, stable, or moving away, thereby improving the accuracy of target tracking. During tracking, the tracking stage of the current image frame is determined by the tracking stage corresponding to the previous image frame, the category information of the current image frame, and the tracking state information, ensuring the continuity and accuracy of tracking stage switching, thus ensuring the robust operation of the entire tracking process. Then, the verification strategy corresponding to the tracking stage of the current image frame is used to verify the tracking process, effectively improving the tracking system's autonomous error correction capability and environmental adaptability.

[0017] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0018] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating the target tracking method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating a target tracking method provided in another embodiment of this application is shown; Figure 3 A schematic diagram of the target tracking device provided in an embodiment of this application is shown. Detailed Implementation

[0019] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein.

[0020] Target tracking algorithms are computer vision tasks that use target detection methods to detect targets in a video stream, identify the target to be tracked, and define its bounding box. The target is then continuously located in subsequent video frames using this bounding box. Taking a single-target tracking algorithm as an example, a unified encoder performs linked feature extraction and matching between a template image and the search region to output a heatmap of the target's location and bounding box regression results.

[0021] However, existing single-target tracking algorithms rely solely on template matching based on appearance similarity, failing to accurately identify the target. Furthermore, the use of the same tracking strategy throughout the entire process leads to low accuracy. For instance, as a target approaches from a distance, its appearance and detailed features change significantly; a target that appears as a small dot at a distance may become more identifiable up close. However, existing single-target tracking algorithms consistently use the target information detected in the first frame, failing to update this information to more refined details as the target approaches, resulting in low target recognition accuracy.

[0022] Furthermore, when similar interfering objects (such as birds, balloons, etc.) overlap or obstruct the tracking target (such as drones) during the tracking process, the tracker may drift to non-target objects. Moreover, due to the lack of ability to identify and verify the tracking target, the system cannot detect the mistracking on its own, resulting in the continuous locking of the wrong target.

[0023] Based on this, to improve the accuracy of the tracking process, this application provides a target tracking method by introducing multiple tracking stages and setting tracking stage transition conditions and differentiated verification strategies for each stage. During the tracking of the target, for the current image frame, relevant information about the target (such as category information, target scale, tracking confidence, etc.) and the tracking stage of the previous image frame are first obtained. Then, based on the relevant information of the target and the tracking stage transition conditions corresponding to the tracking stage of the previous image frame, the tracking stage corresponding to the current image frame is determined. Finally, the tracking target in the current image frame is verified using the verification strategy of the tracking stage corresponding to the current image frame, so as to correct the tracker in a timely manner when anomalies occur in the verification results.

[0024] In this approach, by dividing the tracking process into multiple stages and setting corresponding conversion conditions and verification strategies for each stage, the tracking system can accurately determine the tracking stage corresponding to the current image frame based on the tracking stage of the previous image frame and its corresponding conversion conditions. Then, by using the corresponding verification strategy for the tracking stage of the current image frame, the tracking target can be verified, and the tracker can be corrected in a timely manner based on the verification results, thereby ensuring the accuracy of the tracking process.

[0025] For example, when the tracked target flies closer from a distance, the relevant information of the tracked target can be dynamically switched based on the verification results, so that the relevant information of the tracked target can smoothly transition from coarse-grained to fine-grained, thereby making the relevant information of the tracked target more accurate; when the tracker drifts to a non-target object, the tracked target in the image frame can be verified in time to detect the deviation of the tracked target, thereby preventing continuous false tracking by triggering re-detection (i.e. re-detecting the target and re-initializing the tracker).

[0026] According to a first aspect of the embodiments of this application, a target tracking method is provided. Figure 1 A flowchart of a target tracking method provided in an embodiment of this application is shown. The method is executed by a target tracking device, which can automatically lock onto and continuously monitor a moving target (e.g., a person, a vehicle, an aircraft, etc.) and calculate the target's position and speed. The target tracking device can be a camera, an optoelectronic tracking turntable, a cloud camera, etc., and can be applied to radar tracking systems, optoelectronic tracking systems, laser tracking systems, multimodal fusion systems, etc.

[0027] The target tracking algorithm pre-sets multiple tracking stages. The tracking stage refers to the discrete state in the time dimension of the entire tracking process, which is divided according to information such as the scale change trend of the tracked target, the tracking confidence, and the stability of category prediction. The tracking stages include, but are not limited to, the initial stage, the stable stage, the distance stage, the occlusion stage, and the rapid maneuver stage.

[0028] The initial phase typically refers to the initial stage after the target is first detected and the tracker is initialized. At this time, the target may be moving from far to near, and its visual features gradually become clearer. Specifically, this target tracking method can be initialized through the following steps (steps S101 to S103): Step S101: Acquire the video stream and perform target detection on the video stream to determine the tracking target in the video stream, as well as the initial category information and initial tracking state information of the tracking target.

[0029] A video stream is a data stream consisting of a series of image frames ordered sequentially in time. Each image frame contains scene information at a specific moment. Specifically, a video stream is a sequence of video data captured by an image acquisition device (such as a camera or camcorder). Video streams can be obtained through a USB interface, network transmission, or by reading local video files.

[0030] Object detection is the process of searching for and identifying targets of interest in the first or initial frame of an acquired video stream using computer vision algorithms. Specifically, object detection can be performed by a trained object detection network (such as the YOLO series, Faster R-CNN, etc.). In essence, the object detection network performs a comprehensive scan of the input image and outputs the category labels and location bounding boxes for all potential targets in the image.

[0031] The tracking target is the object in the video stream that needs to be continuously tracked. The tracking target can be any moving entity such as a drone, bird, vehicle, or pedestrian. Specifically, it can be selected from multiple candidate targets output by the object detection network to identify the unique object that needs continuous tracking. For example, in a drone countermeasure scenario, the tracking system will detect all objects in the frame and then select the tracking target from among them. This selection process can be based on various rules, such as selecting the target closest to the center of the frame, selecting the target with the highest confidence, or tracking the target specified by the user interaction.

[0032] After identifying the target, the tracking system records the target's position coordinates (i.e., bounding box) and visual features in the initial frame. Initial category information is the category label assigned to the target by the tracking system when it identifies the target. This category information can be directly derived from the target detection network's detection results of the selected target. Furthermore, since the target may be relatively far away at the initial moment, the initial category information is often coarse-grained. For example, when a drone appears as only a small dot in the distance, the target detection network may only be able to identify it as a flying object or an unknown target, without being able to distinguish its specific category.

[0033] Initial tracking state information is a series of quantitative indicators describing the state of the tracking target and the tracking system itself at the start of tracking. This information constitutes the initial state variables of the state machine. Initial tracking state information includes, but is not limited to, the size of the initial bounding box (i.e., the initial target scale, used to calculate the subsequent scale change rate), the initial tracking confidence (which can be directly taken from the detection confidence of the target detection network), the initial velocity or velocity direction of the tracking target, and the initial timestamp or frame number, etc.

[0034] Step S102: Set the first tracking phase to which the target in the video stream belongs as the start phase.

[0035] In the initialization phase of the tracking system, the current tracking phase can be set as the start period. This allows the tracking system to analyze subsequent image frames of the video stream based on the start period, the corresponding verification strategy, and the tracking phase transition conditions. This enables the system to determine the tracking phase, target category information, and tracking status information for each subsequent image frame.

[0036] Step S103: Initialize the tracker based on the initial category information and the initial tracking status information.

[0037] Here, a tracker refers to an algorithmic model or computational module used to continuously locate a target in a video sequence, such as a tracking network. Specifically, the tracker can employ a single-target tracking algorithm, such as the OSTrack algorithm. The process of constructing and configuring the internal parameters of the tracking algorithm using the initial category information and initial tracking state information obtained in step S101 allows the tracker to perform joint feature extraction and matching between the template image and the search region in a unified encoder during subsequent tracking processes, and outputs the target's location heatmap, bounding box regression results, and tracking confidence scores.

[0038] Specifically, firstly, based on the determined initial bounding box of the target being tracked, a template image is cropped from the image frames of the video stream, and the initial category information and the template image are input into the feature encoder of the tracker to extract the template feature vector; secondly, the initial category information is placed into the initial position of the sliding window voting queue, so that the tracking system can determine the category information of the target being tracked in subsequent image frames based on the sliding window voting queue; finally, the initial tracking state information (such as initial scale, initial tracking confidence, etc.) is stored in a register, so that the tracking system can subsequently determine the tracking stage of the image frame based on this information.

[0039] As an example, suppose in a security monitoring system, a camera captures a black dot approaching from a distance. The tracking system first acquires image frames from the video stream and calls an object detection network for detection. If the object detection network detects an object in the upper left corner of the image, the tracking system selects this object as the tracking target according to rules and outputs initial category information as "flying object," with an initial tracking confidence of 0.9 and an initial target scale of 20×20 pixels. Subsequently, the tracking system uses this 20×20 target scale to crop a template image, inputs it into the tracking network (i.e., the tracker) to extract features, fills the "flying object" category information into a sliding window voting queue, and sets the current tracking phase as the start period, thus completing the initialization. When the next image frame arrives, the tracker can search based on the template image, and the tracking system can determine whether to maintain the start period or switch tracking phases based on the tracking phase transition conditions corresponding to the start period.

[0040] In the above embodiments, target detection is performed on the video stream to determine the tracking target and initial information (i.e., initial category information and initial tracking state information), and the tracking phase of the tracker and video stream is initialized based on the initial information, providing a data foundation for the subsequent tracking process and ensuring the accuracy of the tracking starting point.

[0041] The stabilization period refers to the stage where the target scale is relatively stable, the tracking confidence is high, and the category prediction is consistent; the distance-away period refers to the stage where the target scale continues to shrink and gradually moves away from the camera. Each tracking stage has a pre-set verification strategy, which is a specific verification rule formulated for the target characteristics of that stage. For example, in the initial stage, the category label is allowed to smoothly switch from coarse-grained to fine-grained as the tracked target approaches; in the stabilization period, the category information of the tracked target is locked and abnormal deviations are detected by comparing the category information; and in the distance-away period, the category information is frozen to prevent the introduction of noise.

[0042] In addition, each tracking phase defines the transition conditions for moving to other tracking phases. These conditions typically involve a logical combination of multiple parameters such as the number of tracking frames, the scale change rate, the confidence threshold, and the class voting consistency ratio. For example, the transition condition from the initial phase to the stable phase could be that the class information and tracking status information of the tracked target tend to stabilize; the transition condition from the stable phase to the distance phase could be that the target scale continues to shrink; and the transition condition from the distance phase to the stable phase could be that the target scale resumes to grow, the class information tends to stabilize, and the tracking confidence rises above a preset threshold.

[0043] like Figure 1 As shown, the target tracking method includes the following steps: Step S110: Track the target in the video stream using the tracker, and analyze the current image frame of the video stream to determine the current category information and current tracking status information of the target in the current image frame.

[0044] In this context, a tracker refers to an algorithmic model or computational module used to continuously locate a target in a video stream. As an example, a tracker can employ single-target tracking algorithms based on Vision Transformers, such as OSTrack, SeqTrack, and MixFormer, or it can be a Siamese network (such as SiamRPN++). This tracker continuously updates the target's position information by searching each image frame in the video stream. Furthermore, the tracked target typically has variable visual appearance and spatial location in the video stream. As the tracked target moves in three-dimensional space, its projection size, pose, and sharpness on the image plane dynamically change. The tracker can also adaptively handle these changes to ensure accurate target locking at different viewpoints and distances.

[0045] Each image frame in a video stream contains visual information about the target at a certain moment, while the current image frame refers to the image frame being processed in the video stream. The tracking system can obtain the current category information and current tracking status information of the target by performing a series of operations such as target detection, feature extraction, and state judgment on the current image frame.

[0046] The current category information is the category judgment result obtained after identifying the tracked target in the current image frame through the classification model built into the tracker or an external one. Specifically, the category information can be a coarse-grained category (such as "flying object") or a fine-grained category (such as "multi-rotor drone"). For example, a lightweight category classification head can be added to the encoder output features of the tracking network model corresponding to the tracker to determine the category information of the tracked target. Alternatively, an independent lightweight classification network (such as MobileNet or ShuffleNet) can be run on the cropped area of ​​each frame's tracking result to identify the tracked target.

[0047] As an example, in the output header structure of the tracking network model, a parallel category classification branch is added to the original localization branch (such as center heatmap, size, offset, etc.). The encoder of the tracking network model outputs a feature map of the search region, which has a shape of B×C×H_s×W_s, where B is the batch size, C is the number of channels in the feature map, and H and W are the dimensions of the feature map, i.e., the spatial dimensions of the feature map. After the feature map is input into the tracking network model, the model can first perform average pooling along the spatial dimensions to obtain a B×C dimensional feature vector. Then, it passes through two fully connected layers (with RelU activation in between) to map a B×N_cls dimensional category probability distribution, where N_cls is a preset number of categories. Finally, the category with the highest probability can be selected as the current category information.

[0048] Furthermore, during training, the tracking network model can be supervised using cross-entropy loss, which is jointly optimized with the tracking and localization loss. The total training loss function is:

[0049] in, For heatmap loss, For regression box loss, For coordinate offset loss, The new category cross-entropy loss is used.

[0050] Adding a category branch to the tracking network model allows for the sharing of encoder features when determining category information, without introducing additional feature extraction overhead. Moreover, the number of parameters in the category branch is extremely small (less than 0.1% of the total parameters of the tracking network model), resulting in minimal impact on inference frame rate. Furthermore, the classification gradient can provide additional semantic supervision to the encoder, which helps improve the accuracy of feature discrimination.

[0051] Current tracking state information refers to various quantitative indicators describing the current working quality of the tracker and the current motion state of the target, including but not limited to tracking confidence (reflecting the reliability of bounding box localization), target scale (i.e., the area or side length of the bounding box, reflecting the size of the tracked target in the image), scale change rate (reflecting whether the tracked target is approaching or moving away), and the smoothness of category prediction. This tracking state information is obtained through the tracker's output and auxiliary calculations. For example, tracking confidence can be directly obtained from the peak values ​​of the heatmap output by the tracking network model, and target scale can be calculated by multiplying the width and height of the bounding box.

[0052] Step S120: Obtain the first tracking stage to which the target in the previous image frame belongs.

[0053] The first tracking phase refers to the tracking phase of the previous image frame recorded by the system before processing the current image frame. This historical state information serves as the starting point for determining the transition between tracking phases. For example, if the previous image frame was in a "stable period," the tracking phase determination for the current image frame will be based on this "stable period" state. The system will determine whether to maintain the "stable period," transition to a "distant period" due to the target scale continuously decreasing, or revert to the "initial period" due to interference.

[0054] Step S130: Determine the second tracking stage to which the tracked target in the current image frame belongs based on the current category information, the current tracking status information, and the tracking stage transition conditions corresponding to the first tracking stage.

[0055] The tracking phase transition conditions define a set of logical rules that must be met to switch from one tracking phase to another. These conditions are typically based on various indicators in the current tracking status information. For example, the conditions for transitioning from the "start phase" to the "stable phase" can be set as follows: more than 20 frames have been tracked; the tracking confidence of the most recent 10 frames is higher than 0.9; and the category information of the tracked target has not changed in the most recent 10 frames. Only when all these conditions are met simultaneously will the tracking system update the tracking phase from the "start phase" to the "stable phase".

[0056] The tracking system can match the currently acquired category information and tracking status information with the transition conditions corresponding to the first tracking stage. If the current status information meets the conditions for transitioning from the first tracking stage to a new stage, then that new stage is designated as the second tracking stage; if no transition conditions are met, the first tracking stage remains unchanged, meaning the second tracking stage is equal to the first tracking stage. As an example, suppose the first tracking stage is a stable period. If it is determined that the target scale continues to decrease, satisfying the transition conditions from the stable period to the distance period, then the second tracking stage is designated as the distance period.

[0057] Step S140: The tracking target in the current image frame is verified using the verification strategy corresponding to the second tracking stage, and the tracker is corrected based on the verification result.

[0058] The verification strategy refers to the specific operational procedures formulated for a particular tracking stage to verify the validity of the current tracking results. Different tracking stages have different verification strategies. For example, in the initial stage, the verification strategy allows for updates and smooth transitions, that is, updating the category information corresponding to the tracked target using the current category information, allowing the category information to transition from coarse-grained and unstable information (such as flying objects) to fine-grained and stable information (such as drones). In the stable stage, the verification strategy is locking and anomaly detection, that is, locking the category at the start of this tracking stage as the baseline category, and subsequently comparing the category prediction of the current image frame with the baseline category; if multiple consecutive frames are inconsistent, an anomaly is determined. In the distance stage, the verification strategy is freezing and confidence monitoring, that is, ceasing to update the category information, maintaining a locked state, and only monitoring the tracking confidence.

[0059] The tracking system can also implement remedial measures to adjust the tracker's working state or output based on the verification results obtained from the verification strategy. These corrective measures mainly include updating the category label output by the tracker, triggering a re-detection process, or adjusting the search area. For example, during the initial phase, if the verification result shows that the category information detected in the current image frame is different from the category information identified in the previous image frame, the correction operation is to update the tracking system's output category to the category information detected in the current image frame. During the stable phase, if the verification result shows that the category of the tracked target continues to deviate from the locked category (e.g., the category detected in 5 consecutive images is "bird," while the locked category is "drone"), the correction operation is to determine that the bounding box has been intercepted by an interfering object. The tracking system immediately stops the current tracking, calls the target detection network to rescan the entire image or a local area (i.e., re-detection), obtains the location and category of the new tracked target, and reinitializes the tracker.

[0060] Assuming the tracking phase includes an initial phase, a stable phase, and a distance phase, the following is a detailed description of the operation process of the target tracking method: The target tracking method is used to track a drone flying in from a distance. At the start of tracking, the tracking system is in its initial phase, and the target detected by the tracker in the first image frame may be a small flying object. As the drone approaches, the tracker analyzes the current image frame, and the current category information gradually shifts towards the drone, while the target scale in the current tracking state information also gradually increases.

[0061] In the first K frames of images before the start of tracking (where K can be 10~30), the tracking system determines that the first tracking phase is the start period and detects the tracking phase transition conditions corresponding to the start period. It finds that not all stable conditions have been met (such as insufficient frame count, fluctuations in category information, or fluctuations in tracking confidence). Therefore, it determines that the second tracking phase is still the start period. At this time, the verification strategy of the start period is adopted, allowing category updates. The tracking system outputs a category that smoothly switches from flying object to drone.

[0062] Continuing tracking to frame K+1, the target is clearly visible in the video stream, the tracking confidence remains high, and the category information remains stable. The tracking system can determine that the conditions for transitioning from the initial tracking phase to the stable phase are met, thus identifying the second tracking phase as the stable phase. At this point, the tracker locks the baseline category to UAV, and during tracking in images after frame K+1, it matches the identified current category information with the baseline category in real time.

[0063] If, at frame K+n, a bird suddenly crosses the frame and overlaps with the drone, the tracker analyzes the current image frame and instantly changes the current category information to "bird." The first tracking phase of the tracking system is a stabilization period. If the transition conditions for this phase are not met, the second tracking phase remains a stabilization period. During this phase, a verification strategy is used: the current category information (bird) is compared with the baseline category (drone). If an inconsistency is found, but the number of consecutive deviations has not reached the frame threshold, the anomaly is recorded but no correction is made. If the number of frames with inconsistent category information reaches the frame threshold, the verification result is determined to be a tracking anomaly, and the tracking system performs a correction operation, triggering a re-detection to re-lock onto the actual drone target.

[0064] In the above embodiments, by pre-setting multiple tracking stages and their corresponding transition conditions and verification strategies, state awareness of the tracking process is achieved, and the tracking system can intelligently switch processing modes according to whether the tracking target is approaching, stable, or moving away. During the tracking process, the tracking stage of the current image frame is determined by the tracking stage corresponding to the previous image frame, the category information corresponding to the current image frame, and the tracking state information, ensuring the continuity and accuracy of the tracking stage switching, thereby ensuring the robust operation of the entire tracking process. Then, the verification strategy corresponding to the tracking stage of the current image frame is used to verify the tracking process, effectively improving the tracking system's autonomous error correction capability and environmental adaptability.

[0065] For example, during the initial phase of tracking a target that is flying closer from a distance, dynamic switching of category information is allowed, enabling the category information to smoothly transition from coarse-grained to fine-grained as the target details become richer, effectively improving recognition accuracy. During the stable phase, a locking and anomaly detection strategy is adopted. When the tracker drifts due to similar interference objects, the tracking system can detect this inconsistency in a timely manner through category verification and trigger a re-detection mechanism, thereby effectively preventing continuous false tracking and effectively improving the reliability of tracking.

[0066] Furthermore, to ensure the accuracy of category information, such as Figure 2 As shown, Figure 2 This illustration shows a flowchart of a target tracking method provided in another embodiment of this application. Step S110 specifically includes the following steps: Step S111: Identify the current image frame and determine the first category of the tracked target in the current image frame.

[0067] Step S112: Obtain the second category of the tracking target based on the first preset number of historical image frames, wherein the historical image frames are image frames in the video stream that precede the current image frame and are continuous with the current image frame, and the second category is obtained by identifying the historical image frames.

[0068] Step S113: Determine the current category information of the tracked target in the current image frame based on the first category and the second category.

[0069] The tracking system utilizes computer vision algorithms to extract features and recognize patterns from the current image frame in the video stream, thereby identifying the semantic category to which the tracked target belongs. The first category is the recognition result obtained from the current image frame, which is temporarily stored or labeled as a candidate category. The first category represents the tracking system's intuitive judgment of the tracked target's identity at the current moment.

[0070] The first category can be determined by a separate lightweight classification network, such as MobileNet or ShuffleNet. Specifically, the tracking system crops the target bounding box predicted by the tracker from the current image frame, resizes it, and inputs it into the classification network. The network then outputs probability vectors for each category, and the category with the highest probability is taken as the first category. Alternatively, the first category can be determined by the tracking network, such as OSTrack. Specifically, a parallel category classification branch can be added to the output head structure of the tracking network. After the encoder features are extracted by the tracking network, they are processed through a global average pooling layer and a fully connected layer, so that the category prediction of the tracked target in the current image frame can be directly output through the newly added category classification branch.

[0071] Because visual perception is inevitably affected by environmental noise, the first category may contain accidental errors. For example, in a certain image frame, the high-speed rotation of a drone's rotor may cause blurring, leading the classification network to misclassify it as a bird. In this case, the first category would be the incorrect bird. If subsequent decisions are made solely based on this first category, the tracking system will generate false alarms.

[0072] Historical image frames refer to a series of image frames on the video stream timeline that precede the current image frame and are temporally consecutive to it. These image frames constitute the tracking context. For example, if the current image frame is frame t, then historical image frames can include frames t-1, t-2, up to frame tA, where A is the preset historical window length. Historical image frames record the motion trajectory and appearance changes of the tracked target over a past period. Due to the high temporal correlation of video streams, the target category between adjacent image frames is usually consistent. Historical image frames contain a large amount of statistical information that can prove the true identity of the tracked target. Using historical image frames for decision support can effectively improve the accuracy of category information.

[0073] The first preset number refers to the threshold number of historical image frames referenced during category statistics and fusion, defining the size of the time window. The value of the first preset number directly affects the smoothness and response speed of category information. Therefore, the first preset number can be set to 5 to 10 frames, for example, 8 frames, to avoid insufficient smoothing due to a value that is too small, making it difficult to filter out noise, and to prevent severe lag in category updates due to a value that is too large, thus failing to reflect the real changes of the tracked target in a timely manner.

[0074] The second category is category information that reflects historical trends, obtained through specific statistical rules based on the category recognition results of historical image frames. Specifically, the second category is not an independent result of a single image frame, but a statistical conclusion drawn from the analysis of historical category data of multiple consecutive image frames.

[0075] Current category information refers to the fusion of the intuitive judgment of the current image frame (i.e., the first category) with the historical conclusions of historical image frames (i.e., the second category) to generate the final current category information for subsequent decisions. This fusion process can employ strategies such as the majority voting method (constructing a historical category queue using the first and second categories and determining the category with the most votes as the current category information), exponentially weighted moving average (applying the exponentially decaying weighted average of the probability vectors of each category and taking the maximum value), Bayesian recursive update (using the posterior category of historical image frames as the prior and combining it with the category probability of the current image frame for Bayesian update), and Hidden Markov Model (modeling the category sequence as a Hidden Markov Model for online inference of the optimal category), etc., for logical judgment.

[0076] As an example, the second category is the historical category queue. The tracking system can add the first category to the historical category queue. The length of the queue is the first preset number. Then, the mode of each category in the updated queue is calculated. The mode is the current category information. When there are multiple categories with the same number, the category with the highest tracking confidence among these categories with the same number can be determined as the current category information.

[0077] In this approach, if the first category is consistent with the dominant category in the historical category queue, the current category information remains unchanged, reinforcing the original conclusion. If the first category is a newly emerging category but has not yet become the majority in the queue, the current category information remains unchanged, i.e., the dominant category in the historical category queue, thus filtering out sudden noise. Only when the first category represents a new trend (e.g., the target is moving from far to near, and multiple consecutive frames of images are identified as more granular categories) and accumulates to become the mode in the historical category queue, will the current category information be updated to the new category.

[0078] Suppose a tracking system is tracking a target flying from a distance, with a first preset number of frames of 5. In frames 1 to 5, the target is far away, its visual features are blurred, and the classification network identifies it as an object in every frame. At this point, the historical image frames are the first 5 frames, and the dominant category in the historical category queue is "object." When processing frame 6, the tracking system identifies the current image frame and determines that the first category is still "object," consistent with the dominant category in the historical category queue. Therefore, the current category is determined to be "object."

[0079] As the target approaches, starting from frame 10, the target details gradually become clearer. In frame 10, the classification network identifies the rotor feature and outputs the first category as a multi-rotor drone. At this time, the second category of the historical image frames (frames 5 to 9) is still flying objects. Based on the first and second categories, the tracking system finds that the multi-rotor drone has only appeared once in the historical category queue and has not yet occupied a majority. Therefore, the current category information remains flying objects.

[0080] Processing continues to frame 12. The first category corresponding to the three consecutive frames (frames 10, 11, and 12) is multi-rotor drones. At this point, the number of multi-rotor drones in the historical category queue (frames 8 to 12) exceeds the number of flying objects. Therefore, the tracking system determines that the second category has been updated according to statistical rules, and finally determines that the current category information of the 12th frame image is multi-rotor drone, thus achieving a smooth switch from coarse-grained to fine-grained tracking.

[0081] Furthermore, if a bird flies quickly across the target in a subsequent image (such as frame 15), causing momentary occlusion, the classification network will be affected by the bird's characteristics and output "bird" as the first category. The tracking system then reviews historical image frames (frames 10 to 14) and finds that four of these frames are classified as multi-rotor drones and one as a bird. Therefore, the tracking system can determine that the first category is noise and correct and lock the current category information as multi-rotor drone, effectively avoiding false classifications caused by single-frame occlusion.

[0082] In the above embodiments, the final current category information is determined by fusing the first category (i.e., the recognition result of the current image frame) and the second category (i.e., the recognition results of multiple historical image frames). This method of making category decisions by combining history can effectively suppress classification fluctuations caused by changes in illumination, pose occlusion, motion blur, or imaging noise in a single frame image, thereby outputting a robust and accurate category information. In this process, by analyzing the current image frame and reviewing historical image frames, the identity of the tracked target can be accurately locked. Especially when the appearance of the tracked target changes slightly or is subject to instantaneous interference, the stability of the category can be maintained based on historical trends, avoiding frequent changes in category labels.

[0083] The tracking phase may include an initiation phase and a stabilization phase. The transition from the initiation phase to the stabilization phase signifies that the tracked target has moved from an unstable approach phase to a high-quality locking phase. This application provides a specific implementation method for the transition from the initiation phase to the stabilization phase. Step S130 may include the following steps: If the first tracking phase is the initial phase, then perform the following steps: Step S131a: Determine the number of tracking frames of the tracker, and determine whether the number of tracking frames is greater than the second preset number, to obtain the first judgment result.

[0084] The tracking system can count how many frames it has continuously processed from the start of tracking to the current image frame. The number of tracking frames is a quantitative indicator of the duration of the tracking process. Specifically, the tracking system maintains a frame counter, which increments by one after each frame is processed. The number of tracking frames reflects the cumulative effect of tracking over time, because in the initial stage, the tracked target needs a certain amount of time (i.e., a certain number of frames) to accumulate enough information for its visual features to stabilize, thus meeting the basic time threshold for entering the stable period.

[0085] The tracking system verifies whether the tracking process has lasted long enough to overcome the initial unstable phase. The second preset number is a pre-defined threshold parameter, such as 20 or 30 frames. If the number of tracking frames is less than or equal to this number, it indicates that the tracking system is still in a cold start break-in period, and the target characteristics may not yet be clear. Therefore, regardless of other indicators, entering the stable phase is prohibited. Only when the number of tracking frames exceeds the second preset number is it considered that the tracking system has sufficient historical data to support the judgment of the tracking phase transition.

[0086] Step S132a: Obtain historical category information and historical tracking status information of the tracked target based on a third preset number of historical image frames.

[0087] The tracking system can collect target category and status data within a time window preceding the current image frame, obtaining historical category information and historical tracking status information to assess trends and stability. A third preset quantity defines the length of the observation window. Historical category information refers to the target category prediction results in each image frame within this observation window; historical tracking status information includes tracking confidence, target scale, and other data for each image frame within the observation window. By acquiring this historical information, the tracking system can analyze whether the target category has fluctuated or whether the tracking quality has improved, thus providing a basis for determining whether the conditions for a stable period are met.

[0088] Step S133a: Determine whether the historical category information and the current category information meet the first preset category requirements, and obtain the second judgment result.

[0089] This step verifies the stability of the target category identification. Its core purpose is to confirm that the target category is clear and no longer changes. The first preset category requirement may include that the category prediction results must be consistent across a third preset number of historical image frames (e.g., the category of multi-rotor drones must occupy 90% of the category queue composed of the first and second categories), and the category information of the current image frame must be consistent with the historical categories.

[0090] The tracking system can determine whether the tracked target has been clearly transformed from a difficult-to-distinguish coarse-grained category to a fine-grained category by using a first preset category, and whether the category has not changed recently due to interference. If the historical category information contains multiple different categories, or if the current category information is inconsistent with the historical mainstream category, it indicates that the category recognition is unstable and does not meet the first preset category requirement, and the judgment result is negative.

[0091] Step S134a: Determine whether the historical tracking status information and the current tracking status information meet the first preset tracking status requirements, and obtain the third judgment result.

[0092] This step is used to verify the tracking quality of the tracker and the target motion trend. The first preset tracking state requirements may include indicators such as tracking confidence and target scale. The tracking confidence indicator requires that the tracking confidence output by the tracker (such as the peak score of the heatmap) must be higher than a high threshold (e.g., 0.9) in every image frame in the historical window and in the current image frame. This indicates that the tracking box always accurately covers the features of the tracked target.

[0093] The target scale index requires that the rate of change of the target scale fluctuates within a small range and no longer increases continuously and significantly as in the initial stage. This indicates that the tracked target has approached a relatively stable distance or entered a relatively level-view stage. Only when both the historical and current tracking statuses show high confidence and scale stability can the first preset tracking status requirement be considered met.

[0094] Step S135a: If the first judgment result, the second judgment result, and the third judgment result are all yes, then the second tracking stage is the stable period.

[0095] In this process, the tracking system must meet the following three criteria simultaneously: sufficient tracking frame count, stable identity information (stable category information), and stable quality reliability (stable tracking status information) before it can switch from the initial phase to the stable phase. If any one of these conditions is not met, the tracking system will remain in the initial phase and continue to wait or adjust, thereby avoiding jitter and erroneous switching during the tracking phase and effectively improving the reliability of the tracking system.

[0096] It should be noted that steps S131a to S134a are used to verify the tracking process from different dimensions. There is no specific order to the steps. Therefore, the three judgment steps S131a, S133a and S134a can be performed in any order or in parallel.

[0097] In the above embodiments, the state transition is not determined by a single indicator, but by a comprehensive multi-dimensional judgment, including tracking duration, consistency between historical and current categories, and the stability of the tracking state. This ensures that the tracking target is reliably locked before entering a stable period. This rigorous judgment logic effectively avoids misjudgments caused by transient noise or accidental improvement in clarity, guarantees the robustness of state transitions, and ensures that subsequent verification strategies are initiated at the correct time.

[0098] The tracking phase may include a stabilization period and a distance-away period. After a long period of locking, the tracked target may gradually move away from the target tracking device. At this time, the tracking phase needs to transition from the stabilization period to the distance-away period to switch the tracking strategy in a timely manner when the tracked target begins to move away. This application provides a specific implementation method for the transition from the stabilization period to the distance-away period. Step S130 may include the following steps: If the first tracking phase is a stable period, then perform the following steps: Step S131b: Obtain the historical tracking status information of the target based on the fourth preset number of historical image frames.

[0099] Step S132b: Based on the historical tracking status information and the current tracking status information, determine whether the tracking target in the historical image frame and the current image frame continues to shrink.

[0100] Step S133b: If so, the second tracking phase is the distance period.

[0101] The tracking system can extract state data within a continuous time window preceding the current image frame to analyze the motion trend of the tracked target. The fourth preset quantity is a parameter that defines the length of the trend observation window, which can be set to 10 frames, 15 frames, 20 frames, etc.

[0102] Historical tracking status information can include target scale, target resolution, target motion velocity direction, etc. For example, historical tracking status information records the size of the target bounding box in each image frame and the area or scale factor calculated from it. The tracking system can plot a curve of target scale changing over time, providing a data basis for determining whether the target size is continuously shrinking.

[0103] The tracking system can also determine whether the tracked target is continuously shrinking based on an absolute threshold of the target resolution (i.e., pixel area), or determine the approaching / moving trend by combining the direction of motion speed, or use a depth estimation network to assist in determining the target distance change based on historical tracking status information and current tracking status information.

[0104] When the tracking system determines that the target exhibits a continuous scale-down characteristic in both historical and current image frames, it considers the target to have begun to leave the observation area or enter a long-range mode. At this point, the tracking system updates the second tracking phase of the current image frame from the stable phase to the distance phase. This switching of tracking phases immediately changes the system's behavior, such as stopping class verification updates, locking the current output class, and adjusting the size of the tracker's search area to accommodate the gradually shrinking target.

[0105] In the above embodiments, by performing trend analysis on a fourth preset number of historical image frames, the camera perceives the tracking target's moving away behavior, thereby capturing the dynamic changes of the tracking target to ensure timely switching of the tracking phase. This allows the tracking system to predictively adjust its strategy and freeze the category output in the early stages when the tracking target becomes smaller, avoiding misjudgments caused by changes in the tracking target's posture.

[0106] The tracking phase may include a stabilization period and a distance-away period. During the distance-away process, the tracked target may suddenly turn back, hover, or approach again. At this time, the tracking phase needs to transition from the distance-away period to the stabilization period to reactivate the verification strategy of the stabilization period. This application provides a specific implementation method for the transition conditions from the distance-away period to the stabilization period. Step S130 may include the following steps: If the first tracking phase is a distance period, then perform the following steps: Step S131c: Obtain historical category information and historical tracking status information of the tracked target based on the fifth preset number of historical image frames.

[0107] Step S132c: Based on the historical tracking status information and the current tracking status information, determine whether the tracking target in the historical image frame and the current image frame continues to increase, and obtain the fourth judgment result.

[0108] Step S133c: Determine whether the historical category information and the current category information meet the second preset category requirements, and obtain the fifth judgment result.

[0109] Step S134c: Determine whether the historical tracking status information and the current tracking status information meet the second preset tracking status requirements, and obtain the sixth judgment result.

[0110] Step S135c: If the fourth, fifth, and sixth judgment results are all yes, then the second tracking phase is the stable period.

[0111] Among them, historical category information and historical tracking status information are category data and status data extracted within a continuous time window before the current image frame. They are used to analyze whether the tracked target has the conditions to return to a stable period. Historical category information records the classification header output of each image frame within this window, while historical tracking status information includes key indicators such as the size of the bounding box of the tracked target and tracking confidence. The fifth preset quantity defines the window length for trend observation.

[0112] The tracking system can determine whether a target is turning back or approaching based on the trend of its scale changes. Specifically, the system calculates the rate of change of the target scale in each image frame within the observation window relative to the previous frame. If, in a consecutive set number of image frames, the target scale in each frame is larger than the target scale in the previous frame, the system determines that the target is continuously increasing in size. If the target exhibits a continuous increase in scale, it means that the target has the potential to re-enter a stable observation distance; if the target scale fluctuates or continues to shrink, the determination is negative, and the tracking system will remain in the distance-away phase.

[0113] The second preset category requirement is used to verify the stability of tracking identity recognition. During the distance period, the classification structure may be chaotic due to the blurriness of the target. When the tracking target approaches again, the tracking system needs to confirm whether the identity of the tracking target is clear and consistent.

[0114] The second preset category requirement can include that the category prediction results output by the classification head in historical image frames and the current image frame must maintain a high degree of consistency (e.g., a consistency ratio higher than a certain threshold), and this category should be a previously locked valid category. This is to prevent the tracked target from entering the stabilization period prematurely, even if it is just a blurry patch, due to the distance being slightly reduced, thus avoiding false locking. The fifth judgment result is only positive when the tracked target becomes clear again and the tracking system can continuously and stably identify the category of the tracked target, ensuring that the category label locked by the tracking system after entering the stabilization period is accurate and reliable.

[0115] The tracking system must simultaneously meet three criteria: target scale trend (i.e., the target scale continues to increase), category recognition (i.e., category information is clear and stable), and tracking quality (i.e., tracking confidence consistently exceeds a threshold) before it is allowed to switch from the distance phase to the stable phase. This rigorous triple check ensures that the tracking phase will not be misjudged due to temporary information fluctuations. Only when the tracked target reliably returns to the effective observation area is the stable phase strategy activated, thereby avoiding oscillations during the tracking phase and ensuring the smoothness and rigor of the entire tracking process.

[0116] Similarly, steps S132c to S134c are used to verify the tracking process from different dimensions. There is no specific order to the steps. Therefore, the three judgment steps S132c, S133c and S134c can be performed in any order or in parallel.

[0117] The tracking phase may include an initiation phase, a stabilization phase, and a distance phase. This application provides a specific implementation method for the verification strategy corresponding to different tracking phases. The verification strategy corresponding to the initiation phase may allow for the updating and smooth switching of category information. Specifically, step S140 may include the following steps: Step S141a: If the second tracking phase is the initial phase, then the tracker is corrected according to the current category information.

[0118] The second tracking phase, the initial phase, indicates that the current tracking phase of the tracking system is in the early stage of task initiation or the approach phase of the target from far to near. At this time, the tracking system has not yet accumulated enough high-quality observation data, and the appearance of the tracked target may change significantly in each image frame (e.g., from a small dot to a recognizable outline), and the recognition confidence is in the process of gradually increasing.

[0119] The initial phase is a transitional stage. Its core task is to quickly and accurately identify the true identity of the tracked target while maintaining bounding box localization. Therefore, during the initial phase, the main purpose of the verification strategy is not to defend against external interference, but to promote the improvement of internal cognition. The verification strategy corresponding to the initial phase can be set to dynamically update the category information. This verification strategy does not force the current observation results to be absolutely consistent with historical results, but allows and even expects changes in category labels. Specifically, this verification strategy stipulates that during the initial phase, the system does not establish a verification benchmark, nor does it trigger frequent category switching. Instead, the focus of verification is placed on verifying the credibility of the current category information.

[0120] During the initial phase, the tracking system writes the verified, up-to-date current category information into the tracker's state record and updates the system's output. This correction serves two purposes: firstly, the tracking system updates the currently maintained output category to reflect the current category information, ensuring that the displayed labels always reflect the latest level of understanding; secondly, if the current category information becomes more refined compared to the previous image frame, the tracking system automatically adjusts subsequent tracking expectations. For example, when the category information is updated from an aerial object to a multi-rotor drone, the tracker can increase the attention weight for specific features such as the rotor in feature matching of subsequent image frames, thereby improving positioning accuracy.

[0121] Suppose the tracking system has just started tracking a distant black drone and is currently in the initial phase. In the first 10 frames of the image, the target appears as a black dot with only a few pixels. The current category information identified by the tracking system may be an unidentified flying object. The tracking system executes the initial phase verification strategy, considers the result reasonable, and corrects the tracker's output category to unidentified flying object.

[0122] As the drone approaches, starting from frame 15, the rotor structure of the target becomes faintly visible, and the current category information identified by the tracking drone begins to refine to multi-rotor drone. At this point, the tracking system is in the initial phase and executes the initial phase verification strategy. Finding that this change is in line with expectations, the tracking system corrects the tracker based on the current category information "multi-rotor drone," that is, it switches the output category of the tracking system from unknown flying object to multi-rotor drone. Using this more precise semantic information, it fine-tunes the feature extraction weights of the tracker, making it focus more on the rotor region, thereby obtaining a more accurate target bounding box in subsequent tracking processes.

[0123] In the above embodiments, leveraging the dynamic characteristics of the target's features during the initial tracking phase—from vague to clear and from far to near—a category update strategy is employed. This strategy corrects the tracker's output based on the current category information, effectively enriching the information content and accuracy of the output. This correction mechanism, based on dynamic verification, ensures that the tracking system, in the initial stage with insufficient information, neither blindly locks into the wrong category nor frequently jumps due to noise. This lays a solid and accurate data foundation for subsequent tracking stages, enhancing the practicality and intelligence of the tracking system in complex dynamic scenarios.

[0124] Furthermore, during the stabilization period, a verification strategy of category locking and anomaly detection is employed to ensure the accuracy of the tracked target and prevent interception. Specifically, step S140 may include the following steps: If the second tracking phase is a stable period, then perform the following steps: Step S141b: If the first tracking phase is the initial phase, then the current category information is determined as the target category information.

[0125] Step S142b: Obtain historical category information of the tracked target based on the sixth preset number of historical image frames.

[0126] Step S143b: If the current category information and the historical category information do not match the target category information, trigger re-detection to reinitialize the tracker.

[0127] During the stable period, the tracking system considers the target to have been fully identified and its appearance characteristics to be relatively stable. Therefore, it no longer allows the category label to be changed arbitrarily. The core task during the stable period changes from recognizing the target to adhering to the target, that is, maintaining a continuous lock on the confirmed target to prevent drift.

[0128] Specifically, if the first tracking phase is the initial phase and the second tracking phase is the stable phase, then the current image frame is the critical point for the transition of the tracking phase. That is, the tracking system has just completed the switch from the initial phase to the stable phase. This means that the current category information has just undergone sufficient smoothing and verification and is qualified as a subsequent verification benchmark. Therefore, the category information determined by the current image frame can be officially locked as the verification benchmark of the tracking system, that is, the current category information is determined as the target category information.

[0129] When determining whether tracker correction is needed, the tracking system reviews classification results over a past period to distinguish between random errors and persistent anomalies. The sixth preset quantity defines the review window length, which can be set to 5 frames, 10 frames, or more. Historical category information includes category information identified in the most recent few frames. The historical dimension is introduced to prevent re-detection from being triggered by instantaneous classification errors caused by occlusion, sudden changes in lighting, or detection jitter in a single frame. Only when the error has a certain degree of persistence is it considered a true tracking anomaly.

[0130] Specifically, the tracking system first checks whether the current category information of the current image frame is not equal to the locked target category information. For example, assuming the target category information is a multi-rotor drone, if the current image frame is identified as a bird, the tracking system will check whether the historical category information (such as the category information identified in the last five images) is also not equal to the target category information. Only when both the current category information and the historical category information are different from the target category information will the tracking be determined to be abnormal.

[0131] When the tracking system confirms that the bounding box has been locked onto an incorrect target (such as being intercepted by a bird), it takes remedial measures to trigger re-detection. This involves pausing the current tracking loop, reactivating the global object detection network, and performing a full-image scan or local region scan of the current image frame to find a target that matches the target category information (or historical trajectory). Once a credible target is detected at a new location, the tracking system uses the target's location and features to reinitialize the tracker, including updating the template image, resetting the tracking phase back to the beginning, and clearing the category voting queue.

[0132] In the above embodiments, by establishing target category information upon entering the stable period, a verification benchmark is constructed for the tracking system. Simultaneously, a dual mismatch judgment logic based on current and historical category information is employed to achieve high-precision anomaly detection. This not only filters single-frame noise but also captures continuous drift, effectively improving the robustness of the tracking system. Furthermore, the introduction of a re-detection mechanism endows the tracking system with self-correcting capabilities. Once mistracking occurs, the tracking system can autonomously detect and promptly retrieve the correct target through re-detection, avoiding continuous mistracking and ensuring long-term tracking reliability in complex dynamic environments (such as those with interference or occlusion).

[0133] Furthermore, during the distance period, a category freeze is employed, determining whether to trigger re-detection solely based on the tracking confidence level in the tracking status information to avoid introducing noise. Specifically, step S140 may include the following steps: Step S141c: If the second tracking stage is a distance-away period, then obtain the historical tracking status information of the tracking target based on the seventh preset number of historical image frames.

[0134] Step S142c: If the tracking confidence in the historical tracking status information is lower than a preset threshold, then trigger re-detection.

[0135] During the distance phase, the projected area of ​​the tracked target in the image continuously decreases, resulting in the loss of a large amount of pixel details. This causes previously effective features (such as texture and edges) to become blurred or even invisible. At the same time, due to the decrease in imaging resolution, the category information output by the classification branch is often filled with noise or degenerates into a blurry coarse-grained category, which no longer has any verification value. Therefore, the tracking system needs to adopt a verification logic different from that used in the initial and stable phases to adapt to this low signal-to-noise ratio tracking environment.

[0136] Tracking confidence is a scalar value output by the tracking network model, reflecting the probability or accuracy of locating the tracked target using the current bounding box. In Transformer-based tracking algorithms, this typically corresponds to the maximum response score in the location heatmap. This score does not depend on the specific identity of the target, but rather measures the strength of the similarity match between the target region features and the template features. Even when the target is not clearly visible, as long as the bounding box is still within the target region, the tracking confidence remains high. Conversely, if the target is lost, the tracking confidence will drop significantly. Therefore, tracking confidence is the most reliable basis for determining whether tracking is ongoing.

[0137] Although category information is unreliable during the long-distance period, historical tracking status information still contains tracking confidence records for each image frame. To determine whether the tracking confidence is below a preset threshold, an average judgment can be used: calculating the average tracking confidence across a seventh preset number of historical image frames. If this average is below the preset threshold, the overall tracking quality is considered substandard. Alternatively, a continuous low-value judgment can be used: if a preset number of consecutive historical image frames have tracking confidence below the preset threshold, tracking is considered to have failed. The preset threshold is typically set lower than the threshold during the stable period to accommodate the inherent difficulty of long-distance tracking; for example, it can be set to 0.4 or 0.5.

[0138] When tracking confidence remains persistently low and effective tracking can no longer be maintained, the tracking system can trigger re-detection to resume operation. Since the tracked target may have become very small or nearly disappeared during the distance period, re-detection strategies can include expanding the search area or lowering the confidence threshold of the detection network to re-search for the target over a wider area. Once re-detection is initiated, the tracking system will reinitialize the tracker using the latest detected location and re-determine whether to enter the initiation phase or another appropriate phase based on changes in the scale of the tracked target, thereby attempting to reconstruct tracking.

[0139] In the above embodiments, during the distance period, the verification of unstable category information is abandoned, and instead, the tracking confidence feature is used as the criterion. This avoids false double detection caused by classification errors, ensuring that even when the tracked target is difficult to identify, the tracking system can still maintain a clear self-awareness, neither blindly outputting erroneous data nor easily giving up when there is still hope of maintaining tracking. This achieves complete and reliable coverage of the target's entire lifecycle. Simultaneously, by introducing a seventh preset number of historical image frames for trend analysis, the anti-interference capability of the tracking system is effectively enhanced.

[0140] In addition to the three tracking phases—initial, stable, and distant—the tracking phase can also be configured as a continuous state variable, using a tracking maturity continuous variable of 0 to 2 to control the category update weights; additional states such as occlusion phase and rapid maneuver phase can also be added. The verification strategy for each tracking phase can employ pre-set rules or automatically select the optimal action using a reinforcement learning-based policy network.

[0141] The occlusion period is when the tracked target is obscured by objects such as trees, buildings, and billboards. During the occlusion period, the tracking system can use trajectory prediction to predict the tracking path of the target so as to quickly lock onto it when it reappears. Alternatively, it can calculate the time it takes for the target to pass through the obstruction based on the target's historical movement speed and set a waiting time accordingly. When the target is obscured, the tracking system starts detecting the target, and if the target is not detected after the waiting time has elapsed, it triggers a re-detection to avoid unnecessary waiting.

[0142] The rapid maneuver phase is the process in which the tracked target moves at a random speed and in a random direction. During this phase, the trajectory of the tracked target is difficult to predict. In this tracking phase, the tracking system can adjust the bounding box to expand the search range and effectively prevent the target from being lost.

[0143] According to a second aspect of the embodiments of this application, a target tracking device is provided, such as... Figure 3 As shown, Figure 3The diagram shows a structural schematic of a target tracking device provided in an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the target tracking device.

[0144] like Figure 3 As shown, the target tracking device 1 may include a processor 11 and a memory 12.

[0145] The memory 12 is used to store the computer program 13. The memory 12 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device. The computer program 13 may include computer-executable instructions.

[0146] The processor 11 is used to execute the computer program 13 to implement the above-described target tracking method embodiment.

[0147] The processor 11 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The target tracking device 1 includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0148] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described target tracking method embodiment.

[0149] This application provides a computer program that can be executed by a processor to implement the target tracking method described above.

[0150] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described target tracking method embodiment.

[0151] In the several embodiments provided in this application, any function, if implemented as a software functional module / unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of this application can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or other electronic device) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this application are not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0153] It should be noted that the above embodiments are illustrative of this application and not restrictive, and those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In claims enumerating several means, several units or modules of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

[0154] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target tracking method, characterized in that, The method includes pre-setting multiple tracking stages, each with its own verification strategy and tracking stage transition conditions, and includes: The tracker tracks the target in the video stream and analyzes the current image frame of the video stream to determine the current category information and current tracking status information of the target in the current image frame. Obtain the tracking stage corresponding to the previous image frame to obtain the first tracking stage; The second tracking stage corresponding to the current image frame is determined based on the current category information, the current tracking status information, and the tracking stage transition condition corresponding to the first tracking stage. The tracking target in the current image frame is verified using the verification strategy corresponding to the second tracking stage, and the tracker is corrected based on the verification result.

2. The target tracking method according to claim 1, characterized in that, The step of analyzing the current image frame of the video stream to determine the current category information and current tracking status information of the tracked target in the current image frame specifically includes: The current image frame is identified to determine the first category of the tracked target in the current image frame; The second category of the tracking target is obtained based on a first preset number of historical image frames, wherein the historical image frames are image frames in the video stream that precede the current image frame and are consecutive to the current image frame, and the second category is obtained by identifying the historical image frames; The current category information of the tracked target in the current image frame is determined based on the first category and the second category.

3. The target tracking method according to claim 1, characterized in that, Before tracking the target in the video stream using a tracker, the method further includes: Acquire a video stream and perform target detection on the video stream to determine the tracking target in the video stream, as well as the initial category information and initial tracking state information of the tracking target; Set the first tracking phase corresponding to the video stream as the start period; The tracker is initialized based on the initial category information and the initial tracking status information.

4. The target tracking method according to any one of claims 1 to 3, characterized in that, The tracking phase includes an initial phase and a stable phase; The step of determining the second tracking stage corresponding to the current image frame based on the current category information, the current tracking state information, and the tracking stage transition condition corresponding to the first tracking stage specifically includes: If the first tracking phase is the initial phase, then the following steps are performed: The number of tracking frames of the tracker is determined, and it is determined whether the number of tracking frames is greater than a second preset number, to obtain a first determination result; Based on a third preset number of historical image frames, obtain historical category information and historical tracking status information of the tracked target; Determine whether the historical category information and the current category information meet the first preset category requirements to obtain a second determination result; Determine whether the historical tracking status information and the current tracking status information meet the first preset tracking status requirements to obtain a third determination result; If the first judgment result, the second judgment result, and the third judgment result are all yes, then the second tracking phase is a stable period.

5. The target tracking method according to any one of claims 1 to 3, characterized in that, The tracking phase includes a stabilization period and a divergence period; The step of determining the second tracking stage corresponding to the current image frame based on the current category information, the current tracking state information, and the tracking stage transition condition corresponding to the first tracking stage specifically includes: If the first tracking phase is a stable period, then the following steps are performed: The historical tracking status information of the target is obtained based on a fourth preset number of historical image frames; Based on the historical tracking status information and the current tracking status information, determine whether the tracking target in the historical image frame and the current image frame continues to shrink; If so, then the second tracking phase is the distance period.

6. The target tracking method according to any one of claims 1 to 3, characterized in that, The tracking phase includes a stabilization period and a divergence period; The step of determining the second tracking stage corresponding to the current image frame based on the current category information, the current tracking state information, and the tracking stage transition condition corresponding to the first tracking stage specifically includes: If the first tracking phase is a distance period, then perform the following steps: Based on a fifth preset number of historical image frames, obtain historical category information and historical tracking status information of the tracked target; Based on the historical tracking status information and the current tracking status information, it is determined whether the tracking target in the historical image frame and the current image frame continues to increase, and a fourth determination result is obtained; Determine whether the historical category information and the current category information meet the second preset category requirements to obtain a fifth determination result; Determine whether the historical tracking status information and the current tracking status information meet the second preset tracking status requirements to obtain the sixth determination result; If the fourth, fifth, and sixth judgment results are all yes, then the second tracking phase is a stable period.

7. The target tracking method according to any one of claims 1 to 3, characterized in that, The tracking phase includes an initiation period; The step of verifying the tracking target in the current image frame using the verification strategy corresponding to the second tracking stage, and correcting the tracker based on the verification result, specifically includes: If the second tracking phase is the initial phase, the tracker is corrected based on the current category information.

8. The target tracking method according to any one of claims 1 to 3, characterized in that, The tracking phase includes an initial phase and a stable phase; The step of verifying the tracking target in the current image frame using the verification strategy corresponding to the second tracking stage, and correcting the tracker based on the verification result, specifically includes: If the second tracking phase is a stable period, then perform the following steps: If the first tracking phase is the initial phase, then the current category information is determined as the target category information; Historical category information of the tracked target is obtained based on a sixth preset number of historical image frames; If both the current category information and the historical category information do not match the target category information, a re-detection is triggered to reinitialize the tracker.

9. The target tracking method according to any one of claims 1 to 3, characterized in that, The tracking phase includes a distance period, and the tracking status information includes tracking confidence level; The step of verifying the tracking target in the current image frame using the verification strategy corresponding to the second tracking stage, and correcting the tracker based on the verification result, specifically includes: If the second tracking phase is a distance period, then the historical tracking status information of the tracking target is obtained according to the seventh preset number of historical image frames; If the tracking confidence level in the historical tracking status information is lower than a preset threshold, a re-detection is triggered.

10. A target tracking device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the target tracking method according to any one of claims 1 to 9.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the target tracking method according to any one of claims 1 to 9.

Images