Adaptive target tracking method and system based on timing supervision and full-link occlusion perception

Abstract

The application discloses a kind of based on timing supervision and whole link occlusion perception's self-adapting target tracking method and system, the method constructs double-flow parallel prediction architecture, utilizes the Mamba network with linear calculation complexity to carry out nonlinear motion supervision to Kalman filter, when monitoring that prediction deviation exceeds threshold value, the process noise of filter or reset speed state and covariance matrix is adaptively adjusted, to eliminate cumulative inertia error;In detection stage, the intelligent non-maximum suppression strategy based on minimum area intersection-over-union (IoS) and depth difference is used, and the occluded effective target is retained;In association and update stage, appearance feature fusing is triggered by detecting trajectory collision risk, and feature update cooling lock is started for occluded or re-found target, and feature library pollution is blocked.The application can effectively improve the tracking accuracy of nonlinear maneuvering target, and significantly reduce the identity switching rate in dense crowded scene.

Description

Technical Field

[0001] This invention relates to the field of computer vision, specifically to an adaptive target tracking method and system based on temporal supervision and full-link occlusion perception, aiming to solve the problems of Kalman filter divergence and feature contamination in dense scenes during long-term tracking. Background Technology

[0002] Multi-object tracking (MOT) is a key task in computer vision, widely used in scenarios such as intelligent surveillance, autonomous driving, and human-computer interaction. Its core objective is to locate multiple targets of interest in a continuous video sequence, assign a unique identifier (ID) to each target, and maintain the continuity of its motion trajectory.

[0003] Current mainstream multi-object tracking algorithms typically employ a "tracking-by-detection" paradigm, which first uses an object detector to obtain the bounding box of the current frame, and then performs data association through motion models and appearance features. Despite significant progress in this field in recent years, the following major technical bottlenecks still exist when processing long video sequences, non-linear moving targets, and densely occluded scenes:

[0004] First, there are limitations of traditional linear motion models and filter divergence issues. Most current trackers (such as SORT, DeepSORT, and BoTSORT) rely on Kalman filtering (KF) for state estimation. Kalman filtering typically assumes the target follows a linear motion model with uniform velocity or uniform acceleration. However, in real-world long-video scenarios, targets frequently perform nonlinear maneuvers such as sudden stops and sharp turns. As tracking time increases, the error covariance matrix of the Kalman filter often converges to a small value, leading to a decrease in gain and making the filter insensitive to new observation data (i.e., overfitting to historical trajectories). When the target suddenly changes its motion state, the filter cannot respond in time, resulting in a large deviation between the predicted and actual positions, leading to target loss.

[0005] Secondly, there is the issue of computational overhead in deep learning motion models. To address nonlinear motion problems, some studies have attempted to introduce recurrent neural networks (RNNs) or Transformers to model trajectories. However, RNNs suffer from long-sequence forgetting issues, while the computational complexity of Transformers increases quadratically with sequence length, consuming enormous computational resources when handling long-term tracking and failing to meet real-time requirements. Therefore, there is an urgent need for a motion modeling scheme that possesses both long-term memory capabilities and linear computational complexity.

[0006] 3. Detection Suppression and Feature Contamination Issues under Dense Occlusion: In crowded scenes, occlusion frequently occurs between targets. At the detection level, traditional Non-Maximum Suppression (NMS) primarily relies on Intersection over Union (IoU) to remove redundant boxes. When a distant target is partially occluded by a nearby target, the IoU can be very high, leading to the incorrect suppression of valid distant targets by NMS and resulting in missed detections. At the association and update level, existing appearance matching mechanisms (Person Re-identification, ReID) lack dynamic awareness of occlusion states. When two targets collide or intersect, traditional algorithms often continue updating the appearance feature library, causing features of background targets to be incorrectly fused into the feature library of foreground targets (i.e., feature contamination). Once targets are separated, due to the contaminated feature library, identity switching is highly likely.

[0007] Therefore, designing a tracking method that can adaptively correct linear filtering errors and effectively handle occlusion and feature contamination throughout the entire tracking chain (detection, association, and update) is a pressing technical challenge that needs to be addressed. Summary of the Invention

[0008] Purpose of the invention: This invention provides an adaptive target tracking method and system based on temporal supervision and full-link occlusion awareness, aiming to solve the problems of divergence and lag of Kalman filtering in long-term tracking, high computational complexity of deep learning motion models, and false suppression and feature contamination in dense occlusion scenarios in the prior art, so as to achieve robust tracking in complex scenarios.

[0009] Technical Solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0010] An adaptive target tracking method based on temporal supervision and end-to-end occlusion awareness includes the following steps:

[0011] Step S1: In the motion prediction stage, a dual-stream parallel motion prediction architecture of Kalman filtering and Mamba network is used to predict the target trajectory of the input video frame. The deviation between the two prediction states is calculated and compared with a preset threshold. Based on the comparison result, the process noise covariance of Kalman filtering is adaptively increased or a state reset mechanism is triggered. The state reset refers to retaining the position state, forcibly clearing the velocity state, and resetting the error covariance matrix.

[0012] Step S2: In the target detection stage, perform intelligent non-maximum suppression based on depth perception: calculate the minimum area intersection-union ratio (IoS) between candidate detection boxes, and at the same time calculate the vertical distance difference between the bottom edge coordinates of the candidate boxes. When the IoS values ​​of two candidate boxes are greater than the preset overlap threshold and the vertical distance difference is greater than the preset depth threshold, it is determined that the two are in a front-to-back occlusion relationship rather than redundant detection, the two candidate boxes are retained, and the candidate box with the smaller bottom edge coordinate is marked with an occlusion status flag.

[0013] Step S3: In the data association stage, calculate the intersection-union ratio (IoU) of all predicted trajectories. If the IoU value of any pair of trajectories exceeds the collision warning threshold, it is determined that the pair of trajectories is in a collision aliasing state. Set the appearance feature distance of the trajectory pair in the collision aliasing state as the blocking value so that the process only relies on the motion distance for matching.

[0014] Step S4: In the state update phase, the trajectory state is updated according to the association result. It is checked whether the successfully matched trajectory has an occlusion status flag, or is in a collision aliasing state, or is a trajectory that has just been recovered from a lost state. If any of the above conditions are met, the feature update cooling lock is activated. During the cooling period, the image features of the current frame are prohibited from being updated to the historical feature library of the trajectory. Only the position state update of Kalman filtering is performed.

[0015] An adaptive target tracking system based on temporal supervision and end-to-end occlusion awareness includes:

[0016] The first processing module is used to predict the target trajectory of the input video frame in the motion prediction stage using a dual-stream parallel motion prediction architecture of Kalman filtering and Mamba network, calculate the deviation between the two prediction states and compare it with a preset threshold, and adaptively increase the process noise covariance of Kalman filtering or trigger a state reset mechanism based on the comparison result. The state reset refers to retaining the position state, forcibly clearing the velocity state, and resetting the error covariance matrix.

[0017] The second processing module is used to perform depth-aware intelligent non-maximum suppression during the target detection stage: calculate the minimum area intersection-union ratio (IoS) between candidate detection boxes, and at the same time calculate the vertical distance difference between the bottom edge coordinates of the candidate boxes. When the IoS values ​​of two candidate boxes are greater than a preset overlap threshold and the vertical distance difference is greater than a preset depth threshold, it is determined that the two are in a front-to-back occlusion relationship rather than redundant detection, the two candidate boxes are retained, and the candidate box with the smaller bottom edge coordinate is marked with an occlusion status flag.

[0018] The third processing module is used to calculate the intersection-union ratio (IoU) of all predicted trajectories during the data association stage. If the IoU value of any pair of trajectories exceeds the collision warning threshold, it is determined that the pair of trajectories is in a collision and aliasing state. The appearance feature distance of the pair of trajectories in the collision and aliasing state is set as the blocking value, so that the process only relies on the motion distance for matching.

[0019] The fourth processing module is used to update the trajectory status according to the association results during the status update phase. It checks whether the successfully matched trajectory has an occlusion status flag, or is in a collision aliasing state, or is a trajectory that has just been recovered from a lost state. If any of the above conditions are met, the feature update cooling lock is activated. During the cooling period, the image features of the current frame are prohibited from being updated to the historical feature library of the trajectory. Only the position status update of Kalman filtering is performed.

[0020] The present invention also provides an electronic device, characterized in that it includes: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion awareness as described above.

[0021] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion perception as described above.

[0022] The present invention also provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion perception as described above.

[0023] Beneficial Effects: This invention effectively overcomes the shortcomings of traditional Kalman filtering in long-term tracking due to covariance convergence lag in response to nonlinear maneuvering targets by constructing a dual-stream prediction architecture based on Mamba temporal supervision. Utilizing the unique linear computational complexity of the Mamba model, it achieves accurate capture and adaptive filter correction of complex motion trends such as target acceleration and sudden stops while ensuring industrial-grade real-time performance. This fundamentally solves the technical bottleneck of quadratic growth in computational power consumption during long-sequence modeling in traditional deep learning models. Furthermore, for densely populated scenes, this invention innovatively establishes a full-link spatiotemporal occlusion perception and feature protection mechanism, employing an intelligent suppression strategy based on IoS and depth differences during the detection phase (Smart... The NMS (Non-Maintained Search and Recall) breaks through the limitations of traditional NMS, which relies solely on planar overlap for elimination. It significantly improves the detection recall rate of occluded small targets. Furthermore, in the association and update stages, it constructs a dynamic defense system against the appearance feature library through collision feature melting and feature update cooling lock technology. This effectively blocks the risk of feature contamination during target aliasing, collision, and re-retrieval, thereby greatly reducing the frequency of identity switching while ensuring long-term tracking continuity. It has extremely high robustness and practical application value. Attached Figure Description

[0024] Figure 1 This is a flowchart of the adaptive target tracking method based on temporal supervision and full-link occlusion perception in an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram illustrating the principle of the dual-stream motion prediction and Kalman filter adaptive correction mechanism in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram illustrating the principle of depth-sensing intelligent suppression (Smart NMS) and collision feature circuit breaking logic in an embodiment of the present invention. Detailed Implementation

[0027] To provide a clearer understanding of the features and advantages of the technical solution of the present invention, the composition and implementation of the specific solution are described below in conjunction with the accompanying drawings.

[0028] This invention first provides an adaptive target tracking method based on temporal supervision and full-link occlusion awareness. This method introduces a Mamba network with long-term memory for nonlinear motion supervision on top of linear prediction using Kalman filtering, and combines depth-aware suppression, collision feature melting, and feature update cooling mechanisms to achieve robust tracking in complex scenes.

[0029] Reference Figure 1 An adaptive target tracking method based on temporal supervision and full-link occlusion awareness includes the following steps:

[0030] Step S1: In the motion prediction stage, a dual-stream parallel motion prediction architecture using Kalman filtering and Mamba network is used to predict the target trajectory of the input video frame, and a state reset is performed based on the deviation between the two prediction states.

[0031] This invention constructs a dual-stream parallel motion prediction architecture. The main path prediction unit employs a Kalman filter algorithm to output the linear predicted state of the target; the auxiliary path supervision unit uses a Mamba deep neural network based on a state space model. This network takes the target's historical trajectory coordinate sequence as input and uses a selective scanning mechanism to extract long-term temporal motion features, outputting the target's nonlinear predicted state.

[0032] (1) Main path prediction (Kalman filter)

[0033] Let the state vector of the target be ,in With the center coordinates, Aspect ratio, For height, This represents the corresponding rate of change. Position status, The state is velocity. The state prediction equation for the Kalman filter is:

[0034]

[0035]

[0036] in, Here is the state transition matrix. Let be the error covariance matrix. Let be the process noise covariance matrix. The prior state estimate (motion inertia, velocity, etc.) at time k is calculated under the condition of time k-1. The prior error covariance matrix at time k is calculated under the condition of time k-1.

[0037] (2) Auxiliary road supervision (Mamba network construction)

[0038] To address the linearity limitations of Kalman filtering, this invention introduces Mamba (Selective State Space Model) as a supervisory network. The Mamba network takes the target's coordinate change rate sequence from past frames as input, dynamically adjusts the state transition parameters through a selective scanning module, and outputs the predicted position for the next frame.

[0039] Model input: Select target past Frames (e.g.) Historical center point velocity sequence , Let be the velocity at time t.

[0040] Mamba core processing: Processing sequences using selective state-space equations.

[0041]

[0042]

[0043] in, It is based on the input data The parameters obtained through dynamic discretization. Among them, The source data observed at time t includes velocity and the coordinates of the detection box. Let C be the hidden state at time t-1, and let C be the state to be hidden. Transform into The mapping matrix allows the model to dynamically adjust its parameters based on the current motion context (such as a "deceleration trend"), thus enabling long-term memory and non-linear fitting capabilities.

[0044] Model output: Outputs the non-linear position prediction value for the next frame. .

[0045] During the motion prediction phase, the deviation between the two predicted states is calculated in real time. If the deviation exceeds a preset threshold, the process noise covariance of the Kalman filter is adaptively increased, or a state reset mechanism is triggered. The state reset refers to retaining the position state, forcibly clearing the velocity state, and resetting the error covariance matrix. In this embodiment, the Euclidean distance between the main path predicted state and the auxiliary path predicted state is calculated in real time as the nonlinear deviation. This deviation is compared with a preset confidence threshold: when the deviation is less than the threshold, the current parameters of the Kalman filter are maintained; when the deviation is greater than or equal to the threshold, it is determined that the target has undergone a sudden maneuver or the filter has diverged. At this time, the process noise covariance matrix of the Kalman filter is dynamically increased according to the magnitude of the deviation to improve the filter's sensitivity to the current observation. If the deviation exceeds the threshold for several consecutive frame periods, a forced reset mechanism is triggered: the current position state component of the Kalman filter is retained, the velocity state component is forcibly cleared to zero, and the error covariance matrix is ​​reset to its initial large value, thereby eliminating the inertial error accumulated over long-term tracking.

[0046] According to an embodiment of the present invention, referring to Figure 2 Calculate main road prediction With auxiliary road prediction Euclidean distance error .

[0047]

[0048] Set correction threshold and reset threshold (and ):

[0049] Scenario 1 (Slight Deviation) Noise injection is performed. The process noise of the Kalman filter is dynamically adjusted. :

[0050]

[0051] in For adjustment coefficients, This is the base process noise covariance matrix of the Kalman filter (i.e., the system's default noise level). This increases the filter's estimation of system uncertainty, making it more inclined to trust the observations in subsequent updates.

[0052] Scenario 2 (Severe divergence) ): Execution state reset. This corresponds to filter over-convergence caused by the target undergoing violent maneuvers or prolonged tracking. The specific execution logic is as follows:

[0053] Step a: Preserve position status constant.

[0054] Step b: Force the speed state Set to zero to eliminate incorrect inertial direction.

[0055] Step c: Convert the covariance matrix Reset to the initial large numerical matrix For example, setting the diagonal elements to 1000 or a larger value indicates that the current state has extremely high uncertainty, causing the filter to re-enter the fast convergence phase.

[0056] Step S1 finally outputs the supervised adjusted predicted trajectory. .

[0057] Step S2: In the target detection stage, perform Smart Non-Maximum Suppression (Smart NMS) based on depth perception: calculate the minimum area intersection-union ratio (IoS) between candidate detection boxes, and at the same time calculate the vertical distance difference between the bottom edge coordinates of the candidate boxes. When the IoS values ​​of two candidate boxes are greater than a preset overlap threshold and the vertical distance difference is greater than a preset depth threshold, it is determined that the two are in a front-to-back occlusion relationship rather than redundant detection, the two candidate boxes are retained, and the candidate box with the smaller bottom edge coordinate is marked with an occlusion status flag.

[0058] In this invention, the IoS calculation formula is the intersection area of ​​two detection boxes divided by the area of ​​the smaller detection box, thereby identifying the situation where a large target occludes a small target.

[0059] Specifically, in the target detection phase, any target detection method such as Figure 1 The YoLo network in the middle obtains the object detection box. Subsequently, to prevent the accidental deletion of distant targets in dense crowds, an IoS-based suppression strategy was adopted. For any two detection boxes... Calculate its minimum area intersection-to-union ratio (IoS):

[0060]

[0061] , These represent the detection boxes. Calculate the area of ​​both, and also calculate the difference in vertical distance between their bases:

[0062]

[0063] , These are the detection boxes The vertical height of the base is the ordinate of the base.

[0064] Reference Figure 3 Based on a preset overlap threshold and depth threshold The intelligent nonmaximum suppression determination logic of the present invention is as follows: If (High overlap), and (Depth difference is large) If the coefficient is 0.05, then the two are determined to be in a front-to-back occlusion relationship. In this case, do not suppress: retain the box with the lower score and mark occlusion: mark the target with the smaller bottom coordinate (visually located behind) as occluded, is_occluded = True.

[0065] Step S3: In the data association stage, a comprehensive cost matrix including motion distance and appearance distance is constructed. During this process, the intersection-over-union (IoU) ratio between each pair of predicted trajectory boxes is first calculated. If the IoU value of any pair of trajectories exceeds the collision warning threshold, the pair of trajectories is determined to be in a collision aliasing state. For trajectory pairs in a collision aliasing state, their appearance feature distance is forcibly set to a blocking value, and matching is performed solely based on motion distance to prevent identity swapping caused by similar appearance features in the aliasing area.

[0066] Specifically, refer to Figure 3 This invention designs a collision-sensing feature-based circuit breaking mechanism. A cost matrix is ​​constructed. For Hungarian matching. Before matching, the IoU between the Kalman prediction boxes of all trajectories is calculated, and a collision warning threshold is preset. If the predicted trajectories of targets i and j output in step S1 are... and of (For example, 0.35) then both are marked as "collision and aliasing state". Dynamic cost matrix construction equation:

[0067]

[0068] in, Let the IoU distance be the predicted trajectory bounding boxes of target i and target j. Let be the feature similarity distance between target i and target j.

[0069] Feature-based circuit breaker mechanism: If the trajectory and If the system is in a "collision aliasing state" or a "masking state" (masking state is also called masking feature locking state), then a forced command will be executed. (infinity).

[0070] At this point, the associated cost degenerates into This means that matching is based solely on spatial location, completely avoiding the risk of ID swapping caused by extremely similar appearance features in overlapping areas.

[0071] Step S4: In the state update phase, update the trajectory state based on the association results. Check whether the successfully matched trajectory has an occlusion status flag, is in a collision / aliasing state, or is a trajectory just recovered from a lost state. If any of the above conditions are met, activate the feature update cooling lock. During the cooling period, it is prohibited to update the image features of the current frame to the trajectory's historical feature library. Only the position state update of the Kalman filter is performed to avoid the features of occluders or interference contaminating the main features of the target.

[0072] The feature update cooling lock mechanism of the present invention is as follows: the counter is reset to a preset number of frames, and the exponential moving average (EMA) update of the feature library is skipped before the count reaches zero, and only the position update of the Kalman filter is performed.

[0073] Specifically, once the trajectory successfully matches the detection box, a status update is performed.

[0074] 1) Kalman state update: Update the state vector using the standard Kalman gain formula. Covariance .

[0075] 2) Appearance feature update (with cooling lock): Maintain a feature update cooling counter feature_lock_counter.

[0076] Triggering condition: If the currently matched trajectory meets any of the following conditions:

[0077] i. During the detection phase, it is marked as "occluded state", that is, the is_occluded flag is true;

[0078] ii. The association phase is marked as "collision aliasing state";

[0079] iii. The trajectory has just recovered from the Lost state (Refined). The Lost state means that the tracking was lost, that is, it was not associated at some point.

[0080] Then the action to be performed is: reset feature_lock_counter to Frames (e.g., 10 frames).

[0081] The update logic of this mechanism is as follows:

[0082] If feature_lock_counter > 0, skip the feature smoothing update step and preserve the historical feature vector. It remains unchanged, but the counter is decremented by 1.

[0083] If feature_lock_counter == 0, then perform an exponential moving average (EMA) update:

[0084]

[0085] in, Update the weights for the features. This is the image feature vector of the current frame. This refers to the image feature vector of a historical frame. A feature vector specifically refers to the features of a target (such as a pedestrian) in an image. During computer processing, it is stored in vector form; therefore, image features, image feature vectors, and image feature vectors all mean the same thing.

[0086] This mechanism ensures that the appearance features are only included in the long-term memory bank when the target is clear, independent, and moving smoothly.

[0087] This method introduces a Mamba network with long-term memory for nonlinear motion supervision on the basis of linear prediction by Kalman filtering, and combines depth-aware suppression, collision feature melting and feature update cooling mechanism to achieve robust tracking in complex scenes.

[0088] Based on the same inventive concept, this invention provides an adaptive target tracking system based on temporal supervision and end-to-end occlusion perception, comprising:

[0089] The first processing module is used to predict the target trajectory of the input video frame in the motion prediction stage using a dual-stream parallel motion prediction architecture of Kalman filtering and Mamba network, calculate the deviation between the two prediction states and compare it with a preset threshold, and adaptively increase the process noise covariance of Kalman filtering or trigger a state reset mechanism based on the comparison result. The state reset refers to retaining the position state, forcibly clearing the velocity state, and resetting the error covariance matrix.

[0090] The second processing module is used to perform depth-aware intelligent non-maximum suppression during the target detection stage: calculate the minimum area intersection-union ratio (IoS) between candidate detection boxes, and at the same time calculate the vertical distance difference between the bottom edge coordinates of the candidate boxes. When the IoS values ​​of two candidate boxes are greater than a preset overlap threshold and the vertical distance difference is greater than a preset depth threshold, it is determined that the two are in a front-to-back occlusion relationship rather than redundant detection, the two candidate boxes are retained, and the candidate box with the smaller bottom edge coordinate is marked with an occlusion status flag.

[0091] The third processing module is used to calculate the intersection-union ratio (IoU) of all predicted trajectories during the data association stage. If the IoU value of any pair of trajectories exceeds the collision warning threshold, it is determined that the pair of trajectories is in a collision and aliasing state. The appearance feature distance of the pair of trajectories in the collision and aliasing state is set as the blocking value, so that the process only relies on the motion distance for matching.

[0092] The fourth processing module is used to update the trajectory status according to the association results during the status update phase. It checks whether the successfully matched trajectory has an occlusion status flag, or is in a collision aliasing state, or is a trajectory that has just been recovered from a lost state. If any of the above conditions are met, the feature update cooling lock is activated. During the cooling period, the image features of the current frame are prohibited from being updated to the historical feature library of the trajectory. Only the position status update of Kalman filtering is performed.

[0093] In the first processing module, the main path prediction unit of the dual-stream parallel motion prediction architecture uses the Kalman filter algorithm to output the linear prediction state of the target; the auxiliary path supervision unit uses a Mamba deep neural network based on the state space model. This network takes the historical trajectory coordinate sequence of the target as input, uses a selective scanning mechanism to extract long-term motion features, and outputs the nonlinear prediction state of the target.

[0094] The first processing module calculates the Euclidean distance between the predicted states of the main road and the auxiliary road in real time as the nonlinear deviation. This deviation is compared to a preset confidence threshold: when the deviation is less than the threshold, the current parameters of the Kalman filter are maintained; when the deviation is greater than or equal to the threshold, it is determined that the target has undergone a sudden maneuver or the filter has diverged. At this time, the process noise covariance matrix of the Kalman filter is dynamically increased according to the magnitude of the deviation to improve the filter's sensitivity to the current observation. If the deviation exceeds the threshold for several consecutive frame periods, a forced reset mechanism is triggered: the current position state component of the Kalman filter is retained, the velocity state component is forcibly cleared to zero, and the error covariance matrix is ​​reset to its initial large value, thereby eliminating the inertial error accumulated during long-term tracking. The first processing module corresponds to... Figure 1 The dual-stream prediction supervision module in the middle.

[0095] It should be understood that the adaptive target tracking system based on temporal supervision and full-link occlusion perception in the embodiments of the present invention can realize all the technical solutions in the above method embodiments. The functions of each functional module can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above embodiments, which will not be repeated here.

[0096] The present invention also provides an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion awareness as described above.

[0097] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion perception as described above.

[0098] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus (systems), computer devices, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] This invention is described with reference to a flowchart of a method according to embodiments of the invention. It should be understood that each step in the flowchart and combinations thereof can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A device for a function specified in one or more processes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. An adaptive target tracking method based on temporal supervision and full-link occlusion perception, characterized in that, Includes the following steps: Step S1: In the motion prediction stage, a dual-stream parallel motion prediction architecture of Kalman filtering and Mamba network is used to predict the target trajectory of the input video frame. The deviation between the two prediction states is calculated and compared with a preset threshold. Based on the comparison result, the process noise covariance of Kalman filtering is adaptively increased or a state reset mechanism is triggered. The state reset refers to retaining the position state, forcibly clearing the velocity state, and resetting the error covariance matrix. Step S2: In the target detection stage, perform intelligent non-maximum suppression based on depth perception: calculate the minimum area intersection-union ratio (IoS) between candidate detection boxes, and at the same time calculate the vertical distance difference between the bottom edge coordinates of the candidate boxes. When the IoS values ​​of two candidate boxes are greater than the preset overlap threshold and the vertical distance difference is greater than the preset depth threshold, it is determined that the two are in a front-to-back occlusion relationship rather than redundant detection, the two candidate boxes are retained, and the candidate box with the smaller bottom edge coordinate is marked with an occlusion status flag. Step S3: In the data association stage, calculate the intersection-union ratio (IoU) of all predicted trajectories. If the IoU value of any pair of trajectories exceeds the collision warning threshold, it is determined that the pair of trajectories is in a collision aliasing state. Set the appearance feature distance of the trajectory pair in the collision aliasing state as the blocking value so that the process only relies on the motion distance for matching. Step S4: In the state update phase, the trajectory state is updated according to the association result. It is checked whether the successfully matched trajectory has an occlusion status flag, or is in a collision aliasing state, or is a trajectory that has just been recovered from a lost state. If any of the above conditions are met, the feature update cooling lock is activated. During the cooling period, the image features of the current frame are prohibited from being updated to the historical feature library of the trajectory. Only the position state update of Kalman filtering is performed.

2. The method according to claim 1, characterized in that, In step S1, calculating the deviation between the two prediction states includes: calculating in real time the Euclidean distance between the Kalman filter's prediction state of the target trajectory of the input video frame and the Mamba network's prediction state of the target trajectory of the input video frame as a nonlinear deviation, and using the deviation as the bias.

3. The method according to claim 2, characterized in that, In step S1, the adaptive increase of the Kalman filter process noise covariance or the triggering of the state reset mechanism based on the comparison result includes: comparing the deviation with a preset confidence threshold; when the deviation is less than the preset confidence threshold, maintaining the current parameters of the Kalman filter; when the deviation is greater than or equal to the preset confidence threshold, determining that the target has undergone a sudden maneuver or the filter has diverged, dynamically increasing the Kalman filter process noise covariance matrix according to the magnitude of the deviation; if the deviation exceeds the preset confidence threshold for multiple consecutive frame periods, triggering a forced reset mechanism: retaining the current position state component of the Kalman filter, forcibly clearing the velocity state component to zero, and resetting the error covariance matrix to a specified value.

4. The method according to claim 1, characterized in that, In step S2, the minimum area intersection-union ratio (IoS) between candidate detection boxes is calculated by dividing the intersection area of ​​two candidate detection boxes by the area of ​​the smaller candidate detection box.

5. The method according to claim 1, characterized in that, Step S3 specifically includes: Calculate the IoU between the Kalman prediction boxes of all trajectories, if the trajectory and of If the collision warning threshold is exceeded, the two will be marked as a collision overlap state; Constructing the dynamic cost matrix: ,in The IoU distance between the predicted trajectory bounding boxes of target i and target j. The feature similarity distance between target i and target j; If trajectory and If the object is in a collision-overlapping or occlusion state, then force it to... At this point, the associated cost degenerates into This means that matching is based solely on spatial location.

6. The method according to claim 1, characterized in that, Step S4 specifically includes: Maintain a feature update cooldown counter, feature_lock_counter, if the currently matched trajectory satisfies any of the following conditions: i. The detection phase is marked as an occluded state; ii. The association phase is marked as a collision aliasing state; iii. The trajectory has just been recovered from a lost state; Then the action to be performed is: reset feature_lock_counter to Frame, K is a preset value; Feature updates are performed according to the following logic: if feature_lock_counter > 0, the feature smoothing update step is skipped, and the historical feature vector is preserved. The feature update cooldown counter remains unchanged, and is decremented by 1; if feature_lock_counter == 0, an exponential moving average update is performed. , Update the weights for the features. The image feature vector of the current frame. This refers to the image feature vector of the historical frame, i.e., the historical feature vector.

7. An adaptive target tracking system based on temporal supervision and full-link occlusion perception, characterized in that, include: The first processing module is used to predict the target trajectory of the input video frame in the motion prediction stage using a dual-stream parallel motion prediction architecture of Kalman filtering and Mamba network, calculate the deviation between the two prediction states and compare it with a preset threshold, and adaptively increase the process noise covariance of Kalman filtering or trigger a state reset mechanism based on the comparison result. The state reset refers to retaining the position state, forcibly clearing the velocity state, and resetting the error covariance matrix. The second processing module is used to perform depth-aware intelligent non-maximum suppression during the target detection stage: calculate the minimum area intersection-union ratio (IoS) between candidate detection boxes, and at the same time calculate the vertical distance difference between the bottom edge coordinates of the candidate boxes. When the IoS values ​​of two candidate boxes are greater than a preset overlap threshold and the vertical distance difference is greater than a preset depth threshold, it is determined that the two are in a front-to-back occlusion relationship rather than redundant detection, the two candidate boxes are retained, and the candidate box with the smaller bottom edge coordinate is marked with an occlusion status flag. The third processing module is used to calculate the intersection-union ratio (IoU) of all predicted trajectories during the data association stage. If the IoU value of any pair of trajectories exceeds the collision warning threshold, it is determined that the pair of trajectories is in a collision and aliasing state. The appearance feature distance of the pair of trajectories in the collision and aliasing state is set as the blocking value, so that the process only relies on the motion distance for matching. The fourth processing module is used to update the trajectory status according to the association results during the status update phase. It checks whether the successfully matched trajectory has an occlusion status flag, or is in a collision aliasing state, or is a trajectory that has just been recovered from a lost state. If any of the above conditions are met, the feature update cooling lock is activated. During the cooling period, the image features of the current frame are prohibited from being updated to the historical feature library of the trajectory. Only the position status update of Kalman filtering is performed.

8. An electronic device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion awareness as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion perception as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the adaptive target tracking method based on temporal supervision and full-link occlusion perception as described in any one of claims 1-6.

Images