An anti-occlusion target tracking method fusing multi-granularity dynamic appearance

Abstract

The application belongs to the field of multi-target tracking. An anti-occlusion target tracking method fusing multi-granularity dynamic appearance is provided. The purpose is to solve the technical problems of target ID jump and tracking effect affected in the occlusion scene in the prior art. The main scheme includes obtaining an original image. A target detector is called on the original image to obtain a target detection result, and a target appearance feature is extracted from the target image in the target detection result; a target appearance contribution factor alpha of the video frame is calculated; based on the target detection result of the previous frame, a target motion feature of the current frame is obtained. The target appearance feature, the target appearance contribution factor alpha and the target motion feature are data fused to obtain a target similarity. The targets in two frames are cascade matched and IOU matched, and each target is assigned an ID. The positions of the targets associated with the same ID in each frame are obtained, the running track of the same target ID in the image sequence is obtained, and the target tracking result is output.

Description

Technical Field

[0001] This application proposes a method for multi-object tracking (MOT) in computer vision. It provides an occlusion-resistant target tracking method that integrates multi-granularity dynamic appearance. Background Technology

[0002] Video tracking is an important task in computer vision, referring to the continuous inference of the state of a target in a video sequence. Its task is to locate the target in each frame of the video to generate the target's trajectory. Video tracking technology has a wide range of applications in both military and civilian fields. Military applications include unmanned aerial vehicles, airborne early warning, and battlefield surveillance; civilian applications include mobile robots, intelligent video surveillance, intelligent transportation systems, human-computer interaction, and virtual reality.

[0003] When a target is occluded by other objects, interfered with by the background, or moves out of the field of view, it will cause partial or complete loss of the target's features, making it impossible to associate and match between consecutive frames, resulting in target loss and tracking failure. When the occlusion is restored and re-identification occurs, the target ID will change, which will affect the tracking results. This is a challenge in target tracking tasks.

[0004] Prior art solutions similar to this proposal

[0005] SORT algorithm (Simple Online and Realtime Tracking)

[0006] The basic principle of SORT is to obtain target bounding boxes (Detections) based on target detection algorithms, predict target behavior (Tracks) using Kalman filtering, and perform IOU matching between Detections and Tracks using the Hungarian algorithm.

[0007] (1) Kalman Filter

[0008] Kalman filtering can predict the state of the target bounding box in the next frame based on the current frame's state. The target's motion state is defined as a vector of eight normally distributed elements. The process of predicting the next frame's state based on the current frame's target state is as follows:

[0009] First, it is assumed that both the predicted and observed values ​​contain errors, and that these errors conform to a Gaussian distribution.

[0010] Then assume the error of the predicted value is Q, and the error of the observed value is R;

[0011] The Kalman gain is calculated by combining Q and R, and the optimal estimate is obtained by integrating the predicted and observed values.

[0012] (2) Hungarian algorithm

[0013] The Hungarian algorithm solves an assignment problem by constructing a similarity matrix through the Interchange of Union (IoU) between consecutive frames, performing data matching, and determining the target ID based on the matching results. Its main processing steps are as follows:

[0014] Perform IOU matching between the detected bounding box and the track's predicted bounding box.

[0015] Then, calculate its cost matrix based on the IOU matching results.

[0016] The cost matrix is ​​used as input to the Hungarian algorithm to obtain the matching results, which include three cases:

[0017] Unmatched Track: Matching failed. Detection and Track could not be matched. If the match fails for Tost consecutive times, the target ID will be removed from the image.

[0018] Unmatched Detections: This means that no track can match the detection, so a new track needs to be assigned to this detection.

[0019] Matched Tracks: Indicates a successful match, and then the matched Tracks are updated based on the Kalman Filter.

[0020] DeepSORT algorithm

[0021] DeepSORT adds a CNN network model to SORT to extract the appearance features of the detection box. It then fuses the appearance and motion features to calculate the cost matrix and uses cascaded matching and new trajectory state confirmation to reduce the number of target ID switches.

[0022] Add a CNN network to extract target appearance features

[0023] DeepSORT incorporates a deep learning CNN network to extract the appearance features of the target bounding box. After each frame of detection and tracking, the target appearance features are extracted and saved.

[0024] A CNN is built and trained to distinguish different objects of the same category. The intermediate layers of the network are used to construct an appearance feature extractor. After obtaining the feature vectors of each object, the minimum cosine distance between each pair of objects can be calculated.

[0025] Similarity is calculated by combining appearance features and motion features.

[0026] Motion features provide information about the possible location of the target, which is very effective in short-term predictions. Appearance features consider the differences in appearance between the predicted information and the trajectory information, and are more effective in identifying targets after occlusion when the displacement of the tracked object is small. Therefore, when calculating the similarity matrix, the appearance model (Cosine Distance of ReID) and the motion model (Mahalanobis Distance) are used to calculate the similarity in a weighted manner.

[0027] Added Matching Cascade and status confirmation

[0028] The cascaded matching process is a loop process that matches the trajectory from missing age=0 to missing age=70 with the Detection. Trajectories that have not been lost are matched first, while those that have been lost for a long time are matched later. In addition, a status confirmation judgment is added during the matching process to reduce the problem of target jump.

[0029] Disadvantages of existing technical solutions:

[0030] SORT: In the Sort algorithm, although matching solely based on IOU is very fast, ID switching (target ID changes due to tracking algorithm misjudgment for the same target) is very significant. Especially in occluded scenes, the Kalman filter continuously predicts but does not update, causing the cost matrix to shrink. This easily leads to unspecified tracks incorrectly matching other detectors, resulting in frequent ID switching.

[0031] Deep-SORT:

[0032] 1. The tracking algorithm's performance is highly dependent on the detector, but it does not perform any algorithmic judgment or processing on the confidence of the target box output by the detector, which will affect the tracking performance in certain scenarios (e.g., occlusion).

[0033] 2. Appearance features were introduced when extracting target features, but the CNN network used had a large number of parameters and low performance. Summary of the Invention

[0034] The purpose of this invention is to solve the technical problems existing in the prior art, such as target ID jumps and the impact of tracking performance on occlusion scenarios.

[0035] To achieve the above objectives, the present invention employs the following technical solution:

[0036] An occlusion-resistant target tracking method incorporating multi-granularity dynamic appearance includes the following steps:

[0037] Step 1: Read the images frame by frame from the video stream to obtain the original images.

[0038] Step 2: Call the object detector on the original image to obtain the object detection results, including: confidence score and object image.

[0039] Step 3: Input the target image from the target detection results into a pre-trained multi-granularity network model to extract the target appearance features;

[0040] Step 4: Based on the confidence level in the target detection results, calculate the target appearance contribution factor α of the video frame;

[0041] Step 5: Based on the target detection results of the previous frame, Kalman filtering is used to predict and update the target motion features of the current frame.

[0042] Step 6: Perform data fusion on target appearance features, target appearance contribution factor α, and target motion features, and comprehensively calculate the target similarity in two adjacent frames.

[0043] Step 7: Based on the target similarity in the two adjacent frames, the Hungarian algorithm is used to perform concatenated matching and IOU matching on all targets in the two frames, and an ID is assigned to each target.

[0044] Step 8: Associate the position of the target with the same ID in each frame to obtain the running trajectory of the target ID in the image sequence, and thus output the target tracking result.

[0045] In the above technical solution, step 3 includes the following steps:

[0046] Step 3.1, the pre-training process of the multi-granularity network model, includes the following steps:

[0047] 3.1.1: Perform data augmentation on the ReID dataset to obtain a simulated occlusion ReID dataset, where the ReID dataset refers to the dataset that has been matted with the target bounding box;

[0048] Step 3.1.1 specifically includes the following steps: Input object: ReID original dataset. The GridMask method is used to perform gridded deletion of the target boxes in the ReID dataset, and the density, size and shape of the deleted area are controlled to ensure that the image area is not overly masked and is preserved, thus achieving a reasonable balance.

[0049] After deleting a portion of the ReID dataset in step 3.1.1, pixel values ​​from images in other ReID datasets are randomly extracted and filled into the deleted area to obtain the pseudo-occlusion ReID dataset.

[0050] 3.1.2 Divide the simulated occlusion ReID dataset into a test set and a training set;

[0051] 3.1.3 Constructing a multi-granularity feature fusion network for a multi-granularity network model includes the following steps:

[0052] 3.1.3.1. ResNet50 is used as the backbone network, and the first three layers extract basic image features through convolution;

[0053] 3.1.3.2 After the third layer of the backbone network, four multi-granularity feature branches are constructed, namely: global coarse-grained feature extraction branch, horizontal segmentation fine-grained feature extraction branch, vertical segmentation fine-grained feature extraction branch, and channel segmentation fine-grained feature extraction branch. Based on the basic image features, the multi-granularity features of the image are extracted through each granularity branch.

[0054] 3.1.3.3 Connect the multi-granularity features of the image to the max pooling layer, and calculate the maximum value to obtain the image appearance features that fuse global and local features;

[0055] 3.1.3.4 After max pooling, the image is connected to a fully connected layer and outputs the image re-identification and classification results based on the image appearance features.

[0056] 3.1.4. The multi-granularity network model is trained using the training set. During the training process, the loss function is used as the signal for supervised learning. The global coarse-grained feature extraction branch uses the Softmax Loss cross-entropy loss function and the Triplet Loss triplet loss function to calculate the loss. The three local fine-grained feature extraction branches use the Softmax Loss cross-entropy function to calculate the loss. The model weights are obtained through training, which gives the trained multi-granularity network model.

[0057] 3.1.5. Use test set images as input to verify the output effect of the multi-granularity network model. The main purpose is to use the test set to verify the model effect and prevent overfitting, where the model performs well on the training set but is not suitable for the test set.

[0058] 3.2. Call process during online tracing

[0059] Since only the appearance features of the image are needed during online tracking, the fully connected layer is removed during the call. The target image in the target detection result is used as input, and the trained multi-granularity network model is called to obtain the target appearance features.

[0060] In the above technical solution, the target appearance contribution factor α in step 4 is:

[0061]

[0062] Where Sdet Let S be the confidence score of the target detection result in a video frame at a certain moment, and Ω be the confidence threshold for filtering noise detection. d叶 When Ω = 1, a is 1, which means the dynamic factor reaches its maximum value; when Sdet ≤ Ω, it reaches its minimum value of 0.

[0063] In the above technical solution, step 5:

[0064] Target motion characteristics: Kalman filtering defines the target's motion characteristics as eight normally distributed vectors, namely: (u, v, γ, h, u · v · γ · h · ), representing the position coordinates (u, v) of the center point of the bounding box, the aspect ratio γ, the height h, and the velocity information obtained after differentiation (u). · v · γ · h · );

[0065] Prediction process: When the target moves, the Kalman filter calculates and predicts the motion characteristics of the target in the current frame, namely its position and velocity, based on the position and velocity parameters of the target image in the previous frame.

[0066] Update process: The optimal motion state estimation result for the current frame is obtained by linearly weighting the predicted value based on the previous frame and the actual observed value of the current frame, where the observed value is the motion feature actually observed in the current frame.

[0067] In the above technical solution, step 6: target similarity is measured by distance; the distance is smaller when they belong to the same target, and larger when they are different targets.

[0068] The target's appearance features are represented by cosine distance, denoted as d1;

[0069] The target appearance contribution factor is obtained from step 4 and is denoted as a;

[0070] The target motion characteristics are represented by Mahalanobis distance, denoted as d2;

[0071] The similarity D of the target is calculated by weighting appearance features and motion features:

[0072] D = (θ*a)*d1 + (1 - θ*a)*d2

[0073] Where θ is a hyperparameter with a value range of 0 to 1, used to set the influence of appearance features and motion features; a is a dynamic parameter for each frame used to assign different weights to targets with different confidence levels.

[0074] In the above technical solution, in step 7: the Hungarian algorithm constructs a similarity matrix based on all detected target sets in the current frame and all trajectory sets in the current frame predicted by Kalman filtering, and performs concatenated matching on the detected target set and trajectory set. When the concatenated matching fails, IOU matching is used to solve the target ID allocation problem.

[0075] Each trajectory in the trajectory set contains a trajectory id, which represents the ID of the target assigned to this trajectory, and state, which represents the state of the trajectory.

[0076] Cascaded matching of detection targets and trajectories: When using similarity matrices for matching, a priority strategy of cascaded matching is adopted, prioritizing the matching of trajectories that have not been lost and matching those that have been lost for a long time later.

[0077] Matching results: The matching results between the target set and the trajectory set can be categorized into the following types:

[0078] (1) Detecting target and trajectory matching, also known as Matched Tracks, indicates a successful match;

[0079] (2) No matching trajectory was found for the detected target, also known as Unmatched Detections, which means that a new target suddenly appears in the image and the detected target cannot find a matching target in the previous trajectory;

[0080] (3) Tracks that do not find a matching target, also known as Unmatched Tracks, means that the continuously tracked target moves out of the image area and the track cannot match any of the current detection targets.

[0081] Trajectory State Confirmation: To enhance the algorithm's robustness, the creation of new trajectories and the deletion of lost trajectories must be confirmed before execution. Taking trajectory deletion as an example: for cases where a successful match has been confirmed previously, the trajectory is retained even if consecutive matches no longer occur; deletion is only performed when the threshold is exceeded. The confirmation process is implemented through the State information of the trajectory, which includes the following three states:

[0082] (1) Tentative: Uncertain state. This state is assigned when a trajectory is initialized. It will only transition to a deterministic state after consecutively matching n_init frames. If no detection target is matched while in an uncertain state, it will transition to a deletion state. n_init is an algorithm parameter. An ID is only assigned if several consecutive matches are successful. This is to avoid ID jump problems caused by mismatches.

[0083] (2) Confirmed: This indicates that the trajectory is indeed in a matching state. If the current trajectory is in the confirmed state, but the number of consecutive mismatches reaches max_age, it will switch to the deleted state. The default value of max_age is generally 70, and the threshold can be manually adjusted. The purpose of setting this parameter is to avoid the ID jump problem caused by the reassignment of IDs when the target reappears after being accidentally deleted, in case the target is occluded for a period of time and the matching fails.

[0084] (3) Deleted: Deleted state, indicating that the trajectory has become invalid.

[0085] Update the trajectory set: Based on the trajectory matching results, update the state of each trajectory in the trajectory set using a Kalman filter.

[0086] Assign a target ID: Assign a unique ID to the identified target based on the trajectory matching results.

[0087] Because the present invention adopts the above technical solution, it has the following beneficial effects:

[0088] I. Appearance Contribution Factor Based on Confidence Level

[0089] This invention sets up a target appearance contribution factor based on confidence level, which can adaptively and dynamically adjust the appearance weight when calculating target similarity. When the confidence level is low, the weight of appearance features is reduced, and when the confidence level is high, the weight is increased, thereby improving the robustness of the target tracking algorithm.

[0090] II. Multi-granularity feature fusion network model

[0091] This invention replaces the CNN network in the original DeepSORT algorithm with a multi-granularity feature fusion network MGN, which makes full use of global coarse-grained features, horizontal segmentation fine-grained features, vertical segmentation fine-grained features, and channel segmentation fine-grained features. It can effectively extract structural global information and highly discriminative local information, significantly improving the feature extraction effect under occlusion conditions.

[0092] III. Occlusion Simulation Dataset

[0093] This invention integrates two data augmentation techniques, GridMask and Cutmix, to obtain a simulated occlusion dataset, providing a data foundation for optimization algorithms. GridMask controls the density, size, and shape of the deleted regions when obtaining new data through gridded deletion, ensuring that image regions are neither over-occluded nor preserved, achieving a reasonable balance. Cutmix uses other image filling methods to simulate the effect of target occlusion, while allowing the model to learn the features of two targets simultaneously more efficiently.

[0094] Fourth, deep learning network models are used to extract target appearance features, and these features are combined with motion-based cues to establish associations between targets. The target association task is modeled as a query matching problem. Attached Figure Description

[0095] Figure 1 This is a flowchart of the present invention;

[0096] Figure 2 This is a diagram illustrating the pre-training process of a multi-granularity network model.

[0097] Figure 3 This is an example diagram of the detection results of the present invention. Detailed Implementation

[0098] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0099] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0100] An occlusion-resistant target tracking method incorporating multi-granularity dynamic appearance includes the following steps:

[0101] Step 1: Read the images frame by frame from the video stream to obtain the original images.

[0102] Step 2: Call the object detector on the original image to obtain the object detection results, including: confidence score, bounding box, and object image features.

[0103] Step 3: After outlining the target detection results according to the target bounding box, feed them into a pre-trained multi-granularity network model to extract the appearance features of the target.

[0104] Step 3 specifically includes the following steps:

[0105] Step 3.1 Pre-training process of multi-granularity network model

[0106] 3.1.1 Data augmentation processing is performed on the original ReID dataset to obtain a simulated occlusion dataset.

[0107] Input object: ReID raw dataset

[0108] means:

[0109] The GridMask method is used to perform gridded deletion of the target bounding box, controlling the density, size, and shape of the deleted region. This ensures that the image region is neither excessively masked nor excessively preserved, achieving a reasonable balance.

[0110] After deleting a portion of the target bounding box dataset in step 1, and then, drawing on the idea of ​​Cuxmix technology, pixel values ​​from other target bounding box image datasets are randomly extracted and filled into the deleted region, thus obtaining a pre-training dataset for occlusion simulation.

[0111] Results: Simulated occlusion target bounding box dataset.

[0112] 3.1.2 Divide the simulated occlusion ReID dataset into a test set and a training set.

[0113] 3.1.3 Constructing a multi-granularity feature fusion network

[0114] 3.1.3.1. ResNet50 is used as the backbone network, and the first three layers are used to extract the basic features of the target image.

[0115] 3.1.3.2. Extracting multi-granularity features from the target image using a multi-granularity feature branch layer. After the third layer of the backbone network ResNeSt-50, it is divided into four branches. These are:

[0116] Global coarse-grained feature extraction branch

[0117] Fine-grained feature extraction branch of horizontal segmentation

[0118] Fine-grained feature extraction branch of vertical segmentation

[0119] Fine-grained feature extraction branch for channel segmentation

[0120] 3.1.3.3 After the outputs of each branch of the multi-granularity feature are connected to the max pooling layer, the maximum value is extracted to obtain the feature representation of the image.

[0121] 3.1.3.4 After max pooling, connect to the fully connected layer to identify the final result.

[0122] 3.1.3.5 To obtain comprehensive recognition capabilities, global and local information are ultimately concatenated to form a comprehensive target appearance feature. A loss function is used as the signal for supervised learning. The global branch uses the Softmax Loss cross-entropy loss function and the Triplet Loss triplet loss function to calculate the loss, while the three local fine-grained branches use the Softmax Loss cross-entropy function to calculate the loss.

[0123] 3.1.4 Train the model using the training set to obtain the model weights.

[0124] Using ReID test set images as input, the multi-granularity network model is fed in to obtain a list of images that match the test images.

[0125] Step 3.2 Invocation process during online tracing

[0126] Since only the appearance features of the image are needed during online tracking, the fully connected layer is removed during the call. The target image in the target detection result is used as input, and the trained multi-granularity network model is called to obtain the target appearance features.

[0127] Step 4: Based on the confidence level in the target detection results, calculate the target appearance contribution factor α of the video frame.

[0128]

[0129] Where S det Let S be the confidence score of the target detection result in a video frame at a certain moment, and Ω be the confidence threshold for filtering noise detection. det When α = 1, the dynamic factor reaches its maximum value, and when Sdet < Ω, it reaches its minimum value of 0.

[0130] Step 5: Based on the target detection results of the previous frame, Kalman filtering is used to predict and update the target motion features of the current frame.

[0131] Target motion characteristics: Kalman filtering defines the target's motion characteristics as eight normally distributed vectors, namely: (u, v, γ, h, u...). · v · γ · h · ), representing the position of the boundingbox center, aspect ratio, height, and corresponding velocity information in the image coordinates, respectively.

[0132] Prediction process: When the target moves, the Kalman filter calculates and predicts the position, velocity, and other running status of the target box in the current frame based on parameters such as the target box and velocity in the previous frame.

[0133] Update process: The predicted values ​​and observed values ​​are linearly weighted to obtain the optimal state estimation result.

[0134] The target's appearance features are represented by cosine distance, denoted as d1;

[0135] The target appearance contribution factor is obtained from step 4 and is denoted as a;

[0136] The target motion characteristics are represented by Mahalanobis distance, denoted as d2;

[0137] The similarity D of the target is calculated by weighting appearance features and motion features:

[0138] D=(θ*a)*d1+(1-θ*α)*d2

[0139] Where θ is a hyperparameter with a value range of 0 to 1, used to set the influence of appearance features and motion features; a is a dynamic parameter for each frame used to assign different weights to targets with different confidence levels.

[0140] Step 7: Based on the target similarity in the two adjacent frames, the Hungarian algorithm is used to perform concatenated matching and IOU matching on all targets in the two frames, and an ID is assigned to each target.

[0141] In step 7: The Hungarian algorithm constructs a similarity matrix based on all detected target sets in the current frame and all trajectory sets predicted by Kalman filtering, using a target similarity calculation method. It then performs concatenated matching on the detected target set and trajectory set. If the concatenated matching fails, IOU matching is then used to solve the target ID allocation problem.

[0142] Each trajectory in the trajectory set contains a trajectory id, which represents the ID of the target assigned to this trajectory, and state, which represents the state of the trajectory.

[0143] Cascaded matching of detection targets and trajectories: When using similarity matrices for matching, a priority strategy of cascaded matching is adopted, prioritizing the matching of trajectories that have not been lost and matching those that have been lost for a long time later.

[0144] Matching results: The matching results between the target set and the trajectory set can be categorized into the following types:

[0145] (1) Detecting target and trajectory matching, also known as Matched Tracks, indicates a successful match;

[0146] (2) No matching trajectory was found for the detected target, also known as Unmatched Detections, which means that a new target suddenly appears in the image and the detected target cannot find a matching target in the previous trajectory;

[0147] (3) Tracks that do not find a matching target, also known as Unmatched Tracks, means that the continuously tracked target moves out of the image area and the track cannot match any of the current detection targets.

[0148] Trajectory State Confirmation: To enhance the algorithm's robustness, the creation of new trajectories and the deletion of lost trajectories must be confirmed before execution. Taking trajectory deletion as an example: for cases where a successful match has been confirmed previously, the trajectory is retained even if consecutive matches no longer occur; deletion is only performed when the threshold is exceeded. The confirmation process is implemented through the State information of the trajectory, which includes the following three states:

[0149] (1) Tentative: Uncertain state. This state is assigned when a trajectory is initialized. It will only transition to a deterministic state after consecutively matching n_init frames. If no detection target is matched while in an uncertain state, it will transition to a deletion state. n_init is an algorithm parameter. An ID is only assigned if several consecutive matches are successful. This is to avoid ID jump problems caused by mismatches.

[0150] (2) Confirmed: This indicates that the trajectory is indeed in a matching state. If the current trajectory is in the confirmed state, but the number of consecutive mismatches reaches max_age, it will switch to the deleted state. The default value of max_age is generally 70, and the threshold can be manually adjusted. The purpose of setting this parameter is to avoid the ID jump problem caused by the reassignment of IDs when the target reappears after being accidentally deleted, in case the target is occluded for a period of time and the matching fails.

[0151] (3) Deleted: Deleted state, indicating that the trajectory has become invalid.

[0152] Update the trajectory set: Based on the trajectory matching results, update the state of each trajectory in the trajectory set using a Kalman filter.

[0153] Assign a target ID: Assign a unique ID to the identified target based on the trajectory matching results.

[0154] Step 8: Output the target tracking result, that is, the target's trajectory in the image sequence.

Claims

1. A method for anti-occlusion target tracking that integrates multi-granularity dynamic appearance, characterized in that, Includes the following steps: Step 1: Read the images frame by frame from the video stream to obtain the original images; Step 2: Apply the object detector to the original image to obtain the object detection results, including: confidence score and object image; Step 3: Feed the target image from the target detection results into a pre-trained four-branch multi-granularity network model to extract the target appearance features; The four-branch multi-granularity network model includes: a global coarse-grained feature extraction branch, a horizontal segmentation fine-grained feature extraction branch, a vertical segmentation fine-grained feature extraction branch, and a channel segmentation fine-grained feature extraction branch. The four-branch multi-granularity network model is deployed in parallel after the third layer of the backbone network. The outputs of each branch are fused by the max pooling layer to generate a target appearance feature vector that combines global coarse-grained features and local fine-grained features. Step 3.1: The pre-training process of the multi-granularity network model includes the following steps: Step 3.1.1: Perform data augmentation on the ReID dataset to obtain a simulated occlusion ReID dataset, where the ReID dataset refers to the dataset that has already been matted with the target bounding boxes; Step 3.1.1 specifically includes the following steps: Input object: ReID original dataset. The GridMask method is used to perform gridded deletion of the target boxes in the ReID dataset, and the density, size and shape of the deleted area are controlled to ensure that the image area is not overly masked and is preserved, thus achieving a reasonable balance. After deleting a portion of the ReID dataset in step 3.1.1, pixel values ​​from images in other ReID datasets are randomly extracted and filled into the deleted area to obtain the simulated occlusion ReID dataset. Step 3.1.2 Divide the simulated occlusion ReID dataset into a training set and a test set; Step 3.1.3 Constructing a multi-granularity feature fusion network for the multi-granularity network model, specifically including the following steps: Step 3.1.3.1: Use ResNet50 as the backbone network. The first three layers extract basic image features through convolution. Step 3.1.3.2: After the third layer of the backbone network, four multi-granularity feature branches are constructed, namely: global coarse-grained feature extraction branch, horizontal segmentation fine-grained feature extraction branch, vertical segmentation fine-grained feature extraction branch, and channel segmentation fine-grained feature extraction branch. Based on the basic image features, the multi-granularity features of the image are extracted through each granularity branch. Step 3.1.3.3: Connect the multi-granularity features of the image to the max pooling layer, and calculate the maximum value to obtain the image appearance features that fuse global and local features; Step 3.1.3.4: After max pooling, connect to the fully connected layer and output the image re-identification and classification result based on the image appearance features; Step 3.1.4: Train the multi-granularity network model using the training set. During training, a loss function is used as a signal for supervised learning. Specifically, the global coarse-grained feature extraction branch uses the Softmax Loss cross-entropy loss function and the Triplet Loss triplet loss function to calculate the loss, while the three local fine-grained feature extraction branches use the Softmax Loss cross-entropy function to calculate the loss. The model weights are obtained through training, thus yielding the trained multi-granularity network model. Step 3.1.5: Use the test set images as input to verify the output performance of the multi-granularity network model; Step 3.2, Invocation process during online tracing Since only the image appearance features need to be obtained during online tracking, the fully connected layer is removed during the call, and the target image in the target detection result is used as input to call the trained multi-granularity network model to obtain the target appearance features; Step 4: Based on the confidence level in the target detection results, calculate the target appearance contribution factor of the video frame. : ； in Let Ω represent the confidence level of the target detection result in a video frame at a certain moment, and Ω be the confidence threshold for filtering noise detection. When α is 1, the dynamic factor reaches its maximum value. The minimum value of 0 is obtained at that time; Step 5: Based on the target detection results of the previous frame, Kalman filtering is used to predict and update the target motion features of the current frame; Step 6: Perform data fusion on target appearance features, target appearance contribution factor α, and target motion features, and comprehensively calculate the target similarity in two adjacent frames; Step 7: Based on the target similarity in the two adjacent frames, the Hungarian algorithm is used to perform concatenated matching and IOU matching on all targets in the two frames, and an ID is assigned to each target; Step 8: Associate the position of the target with the same ID in each frame to obtain the running trajectory of the target ID in the image sequence, and thus output the target tracking result.

2. The anti-occlusion target tracking method based on multi-granularity dynamic appearance as described in claim 1, characterized in that, In step 5: Target motion characteristics: The Kalman filter algorithm defines the target motion characteristics as eight normally distributed vectors: (u,v,γ,h,u˙,v˙,γ˙,h˙), which represent the position coordinates (u,v), aspect ratio γ, height h, and velocity information obtained after differentiation (u˙,v˙,γ˙,h˙) of the bounding box center point, respectively. Among them, u˙,v˙,γ˙,h˙ refer to the derivatives of u,v, γ, and h, respectively. Prediction process: When the target moves, the Kalman filter calculates and predicts the motion characteristics of the target in the current frame, namely its position and velocity, based on the position and velocity parameters of the target image in the previous frame. Update process: The optimal motion state estimation result for the current frame is obtained by linearly weighting the predicted value based on the previous frame and the actual observed value of the current frame, where the observed value is the motion feature actually observed in the current frame.

3. The anti-occlusion target tracking method based on multi-granularity dynamic appearance as described in claim 1, characterized in that, In step 6: Target similarity is measured by distance; the distance is smaller when they belong to the same target, and larger when they are different targets. The target's appearance features are represented by cosine distance, denoted as . ; The target appearance contribution factor is obtained from step 4 and is denoted as ; The target motion characteristics are represented by Mahalanobis distance, denoted as . ; The similarity D of the target is calculated by weighting appearance features and motion features: ； in, This is a hyperparameter with a value range of 0 to 1, used to set the influence of appearance and motion features; The dynamic parameters for each frame are used to assign different weights to targets with different confidence levels.

4. The anti-occlusion target tracking method based on multi-granularity dynamic appearance as described in claim 1, characterized in that, In step 7: The Hungarian algorithm constructs a similarity matrix based on all detected target sets in the current frame and all trajectory sets predicted by Kalman filtering, using a target similarity calculation method. It then performs concatenated matching on the detected target set and trajectory set. If the concatenated matching fails, IOU matching is then used to solve the target ID allocation problem. Each trajectory in the trajectory set contains a trajectory id, which represents the ID of the target assigned to this trajectory, and state, which represents the state of the trajectory. Cascaded matching of detection targets and trajectories: When using similarity matrices for matching, a priority strategy of cascaded matching is adopted, prioritizing the matching of trajectories that have not been lost and matching those that have been lost for a long time later. Matching results: The matching results between the target set and the trajectory set can be categorized into the following types: (1) Detecting target and trajectory matching, also known as Matched Tracks, indicates a successful match; (2) No matching trajectory was found for the detected target, also known as Unmatched Detections, which means that a new target suddenly appears in the image and the detected target cannot find a matching target in the previous trajectory; (3) No matching target was found for the trajectory, also known as Unmatched Tracks, which means that the continuously tracked target moves out of the image area and the trajectory cannot match any of the current detection targets; Trajectory State Confirmation: To enhance the algorithm's robustness, the creation of new trajectories and the deletion of lost trajectories must be confirmed before execution. Taking trajectory deletion as an example: for cases where a successful match has been confirmed previously, the trajectory is retained even if consecutive matches no longer occur; deletion is only performed when the threshold is exceeded. The confirmation process is implemented through the State information of the trajectory, which includes the following three states: (1) Tentative: Uncertain state. This state is assigned when a trajectory is initialized. It will only change to a deterministic state after consecutively matching n_init frames. If no detection target is matched while in an uncertain state, it will change to a deletion state. n_init is an algorithm parameter. It is assigned an ID only if it matches successfully several times in a row. This is to avoid ID jump problems caused by mismatches. (2) Confirmed: The confirmed state indicates that the trajectory is indeed in a matching state. If the current trajectory is in the confirmed state, but the mismatch reaches the max_age number of consecutive times, it will be changed to the deletion state. The default value of max_age is generally 70, and the threshold can be manually adjusted. The purpose of setting this parameter is to avoid the matching failure caused by the target being occluded for a period of time, and the ID jump problem caused by the reassignment of ID when the target reappears after being mistakenly deleted. (3) Deleted: This indicates that the track has been deleted and is no longer valid. Update the trajectory set: Based on the trajectory matching results, update the state of each trajectory in the trajectory set using a Kalman filter; Assign a target ID: Assign a unique ID to the identified target based on the trajectory matching results.

Images