Target tracking method for multi-scale ReID network and double-domain joint measurement in complex scene

Abstract

The invention discloses a multi-scale ReID network and double-domain joint measurement target tracking method for a complex scene. The method comprises the following steps: S1, constructing and training a target detection model YOLOv5; s2, designing an improved ReID network IncepSPA-DSC fusing a depth separable convolution and a spatial pyramid channel attention mechanism; s3, a DeepTrack-SPAE tracking framework is constructed, and a DeepTrack-SPAE tracking framework is According to the invention, by constructing a multi-scale feature fusion mechanism and an attention enhancement module, on the premise of maintaining the real-time processing speed, an anti-interference feature vector with strong discrimination is generated, and the cooperative capture capability of the network on local features and global context information is significantly improved. A space-feature double-domain joint measurement method is innovatively proposed, and by establishing a feature similarity matrix fused with Euclidean distance constraint, the spatial proximity and feature consistency of a target are considered in cost calculation, so that the problem of trajectory breakage caused by short-time shielding is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and computer vision technology and can be used in scenarios such as video surveillance, autonomous driving, and human-computer interaction. It specifically relates to a target tracking method using a multi-scale ReID network and dual-domain joint measurement for complex scenarios. Background Art

[0002] Multi-object tracking (MOT), a core technology in fields such as autonomous driving and video surveillance, relies on accurately detecting and estimating the trajectories of multiple objects in a video stream. In recent years, tracking-by-detection methods have become the mainstream approach to solving MOT tasks. These methods first perform object detection and then track the objects. The tracking process is primarily divided into two parts: motion modeling and state estimation, and the associated tasks.

[0003] The Kalman filter (KF), with its constant velocity assumption, has become a common tool in motion modeling and state estimation. The KF not only predicts the target's trajectory bounding box in subsequent frames but also predicts the trajectory state even when the target is occluded or missed.

[0004] The association task is to match the new frame detection with the current trajectory group. This step is mainly achieved through two methods. The first is to locate based on the intersection over union (IoU) between the predicted trajectory bounding box and the detection bounding box. This method is simple and intuitive, but in complex scenes, such as when the target position changes due to camera motion, the accuracy of IoU may be affected. The second is to combine the use of deep appearance cues for re-identification (Re-ID). The ReID feature extraction network can extract deep features of the target. These features usually have high discrimination and robustness, which helps to accurately identify the target in complex environments.

[0005] However, in crowded scenes or when the target is partially occluded, the effectiveness of the ReID feature extraction network may be limited, as occlusion and crowding may cause incomplete or disturbed feature information. Therefore, how to improve the accuracy and robustness of the ReID feature extraction network in such situations, as well as how to balance IoU and appearance features in different scenarios and design more robust association methods are important challenges faced by current research. Summary of the Invention

[0006] To address key issues in existing object tracking technologies, particularly the challenges posed by object occlusion and appearance similarity in complex scenes, this paper proposes a target tracking method for complex scenes that combines a multi-scale ReID network with a dual-domain joint metric. This innovative StrongSORT target tracking method, DeepTrack-SPAE (DeepTrack-SpatialPyramid Attention Mechanism and Euclidean Distance), is presented. This method offers in-depth optimization and improvements based on the existing StrongSORT framework, embodied in two key innovations. First, addressing the limitations of small feature extraction networks for ReID, this paper improves them by introducing depthwise separable convolutions and a spatial pyramid channel attention mechanism, significantly enhancing the feature extraction capabilities of the ReID network in complex environments and ensuring stable and accurate target tracking. Second, this paper designs a novel association method that innovatively increases the original two-matching step to three-matching. Using the neighborhood distance (Euclidean distance) between two boxes as a key criterion for the three-matching step, this method simultaneously considers the spatial proximity and feature consistency of the target in the cost calculation, improving target tracking performance in multiple scenarios. By combining these two approaches, the present invention improves efficiency while maintaining performance, enabling real-time multi-target tracking. Training and evaluation were conducted using two examples. Experimental results show that the present invention demonstrates superior performance in scenarios with large crowds and frequent occlusions.

[0007] The technical solutions of the present invention are as follows:

[0008] A target tracking method for complex scenes using a multi-scale ReID network and dual-domain joint metric includes the following steps:

[0009] S1: Construction and training of the YOLOv5 object detection model. This model is constructed based on the YOLOv5 architecture. By performing normalized coordinate transformation and adaptive anchor box optimization on the annotated dataset, the model's generalization capability is improved in conjunction with the Mosaic data augmentation strategy. Bounding box regression training is performed using the CIoU loss function, ultimately generating a detection model that outputs target location coordinates and confidence scores, providing an initial positioning benchmark for subsequent processing.

[0010] S2: Design an improved ReID network IncepSPA-DSC that integrates depthwise separable convolution and spatial pyramid channel attention mechanism. It uses the trained object detection model to extract the target image region in the video sequence and construct a temporally correlated target feature sample set.

[0011] S3: Build the DeepTrack-SPAE tracking framework, including the following steps S31-S32:

[0012] S31: For existing trajectories, a Kalman filter (KF) is used to predict and update the trajectory state. A linear motion model is constructed using the target's historical state vector to infer the predicted trajectory for the current frame. In each frame, the KF module first outputs the target predicted state based on the linear optimal estimate. Then, the observation value from the detection module is input, and the trajectory is dynamically corrected using the Kalman gain. The estimated value and error covariance matrix are updated to obtain the optimal target state estimate.

[0013] S32: A three-stage multi-dimensional matching strategy is implemented, integrating appearance feature similarity, geometric position overlap (IoU), and neighborhood distance information in sequence, and gradually completing the association between the trajectory and the detection results.

[0014] In the method described above, in step S2, an improved IncepSPA-DSC is constructed, which uses Inception as the backbone network and introduces the depthwise separable convolution and spatial pyramid attention mechanisms on this basis, as follows:

[0015] S21: At the input end, the target detected by the target detection model is processed to obtain a target recognition dataset as the input of the network;

[0016] S22: The first stage first passes through two convolution blocks. The specific calculation process includes: the input image first passes through a 32-channel, 3*3 convolution kernel, and a stride of 2*2, followed by a Relu convolution. Then, the same convolution and Relu convolution are repeated. The result is the output of this stage.

[0017] S23: The second stage consists of 12 blocks. The specific calculation process is as follows: First, the input of the first block is the output of the first stage. The result is obtained by passing through the block layer. Then, a 1×1 convolution is performed on the output of the previous block. The result is channel-wise concatenated with the output of the current block. The concatenated result will be used as the input of the next block, thus realizing residual connection and avoiding loss of semantic information.

[0018] S24: The input of the third stage comes from the output of the previous stage. The specific calculation process is: first, the number of channels is changed to 2048 through ordinary convolution, then batch normalization and ReLU activation are performed; then global average pooling is performed, and finally a 2048-dimensional feature vector f is output through the fully connected layer. det (d j ).

[0019] The method described above is as follows: first, a ReLU activation function is performed, and then a depth-wise separable convolution block (SeparableConv) is applied; then, two ReLU activation and depth-wise separable convolution block (SeparableConv) operations are performed; then, a maximum pooling (MaxPooling) operation is performed, and finally, an SPA block is applied as an attention mechanism module; after these steps, the result of the Block is output.

[0020] In the described method, SeparableConv represents a depth-wise separable convolution block, and its specific calculation process is as follows: first, a convolution with the number of input channels equal to the number of output channels is used to process each channel independently, which is called depth-wise convolution; then, a 1×1 convolution is used to convert the number of output channels of the depth-wise convolution into the required number of output channels, which is called point-by-point convolution.

[0021] The method described above states that the SPA block is a module for the spatial pyramid pooling channel attention mechanism. The specific calculation process is as follows: first, a normal convolution is performed to match the number of channels; then, the input feature map is pooled into three different sizes through three adaptive average pooling layers of different scales; then, the outputs of the three pooling layers are spliced ​​in the channel dimension for feature fusion; next, the spliced ​​feature vector is mapped to a lower dimension through a fully connected layer and processed using the ReLU activation function; then, another fully connected layer is used to map the feature vector back to the original dimension, and the Sigmoid activation function is used to limit the output to the range of [0,1] to generate a channel attention weight vector; each element in the vector represents the weight of the corresponding channel in the input feature map; finally, the channel attention weight vector is multiplied element-by-element with the input feature map and output, thereby completing the application of the channel attention mechanism.

[0022] The method described, in step S32, the steps are as follows:

[0023] S321: Appearance feature f based on the detection frame det (d j ) and the appearance features of the trajectory estimation box f trk (t i ) Calculate the cosine distance and construct the cost matrix C using the minimum cost matching method cos , where C cos [i,j] is the trajectory t i With the detection box d j The cosine distance between them, the cost matrix C cos The construction formula is:

[0024]

[0025] C cos (i,j)=1-cosine_similarity(f det (d j ),f trk (t i ))

[0026] Among them, cosine_similarity(f det (d j ),f trk (t i )) is the cosine similarity formula;

[0027] S322: Calculate the Mahalanobis distance using the motion features of the detection frame and the trajectory estimation frame, and C cos Perform weighted combination to obtain the new cost matrix C mahal , its construction formula is:

[0028] S=P i +P j

[0029]

[0030]

[0031] Among them, d M (z i ,z j ) represents the Mahalanobis distance calculation formula, z i and z j They represent the state vector of the i-th trajectory and the state vector of the j-th detection, respectively, P i and P j They represent the covariance matrix of the i-th trajectory and the covariance matrix of the j-th detection respectively, gated_cost is the gating cost used to indicate an infeasible match, and gating_threshold represents the gating threshold;

[0032] S323: Using the Hungarian algorithm to calculate the cost matrix C cost =C mahal (i,j) solves the optimal allocation problem and obtains the trajectory index set and detection index collection pass The matched trajectory and detection box set M can be obtained (1) ={(t i ,d j )}, and the set of unmatched trajectories and unmatched detection box set The formula is:

[0033]

[0034] S324: For unmatched trajectory sets and unmatched detection box set Use the IoU between the track box and the detection box to construct the cost matrix C IoU , use the Hungarian matching algorithm to perform the second matching to obtain the matching pair M (2) ={(t i ,d j )} and unmatched trajectory set and unmatched detection box set C IoU The construction formula is:

[0035]

[0036] in, and Respectively represent the tth i The bounding box of the dth track (expressed in tlwh form) and the dth track j The formula for obtaining the matching set, the unmatched trajectory set, and the unmatched detection frame set is the same as that in S323;

[0037] S325: Use Euclidean distance to compare the above unmatched trajectory sets and unmatched detection box set Perform the third matching, calculate the Euclidean distance between multiple candidate target frames, set the distance threshold, and when the distance between the two is less than the specified threshold, it is considered to be a matching target, and the final matching pair M is obtained accordingly. (3) ={(t i ,d j )} and unmatched trajectory set and unmatched detection box set The Euclidean distance construction formula is:

[0038]

[0039] Among them, d E (t i ,d j ) represents the Euclidean distance calculation formula, t i and d j denote the state vector of the i-th trajectory and the state vector of the j-th detection respectively;

[0040] S326: Unmatched detection box d j A new trajectory is generated and set to the unconfirmed state Tentative, without matching the trajectory t i Will increase the unmatched age When its unmatched age exceeds the maximum age threshold max Or when it is in an unconfirmed state, the track will be destroyed. The formula is:

[0041]

[0042] Through the above steps, the detection frame and trajectory frame of the current frame are matched, and the matching results can be used for subsequent motion model updates; unmatched detections and trajectories will be processed accordingly. The Euclidean distance matching mechanism of the third matching shows excellent robustness when dealing with complex dynamic scenes such as large numbers of targets, target occlusions, and target size changes.

[0043] By adopting the above solution, the present invention solves the following technical problems:

[0044] (1) Addressing the difficulty of feature representation in complex scenarios: Traditional pedestrian re-identification (ReID) networks are prone to feature confusion and representation failure in complex scenarios such as target occlusion and dense crowds. This solution constructs a multi-scale feature fusion mechanism and an attention enhancement module to generate highly discriminative and anti-interference feature vectors while maintaining real-time processing speed, significantly improving the network's ability to collaboratively capture local features and global contextual information.

[0045] (2) Association robustness in dynamic scenes: Traditional association methods based on Intersection over Union (IoU) are prone to mismatching when the target deforms or the view angle changes. This solution innovatively proposes a spatial-feature dual-domain joint measurement method. By establishing a feature similarity matrix that incorporates Euclidean distance constraints, it simultaneously considers the spatial proximity and feature consistency of the target in the cost calculation, effectively solving the problem of trajectory interruption caused by short-term occlusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 : DeepTrack-SPAE model structure diagram;

[0047] Figure 2 : Overall flow chart;

[0048] Figure 3 : Structure diagram of the appearance feature extraction model IncepSPA-DSC;

[0049] Figure 4 : Block structure diagram;

[0050] Figure 5 : SeparableConv block structure diagram;

[0051] Figure 6 : SPA block structure diagram;

[0052] Figure 7 : Three-way matching flow chart; DETAILED DESCRIPTION

[0053] The present invention is described in detail below with reference to specific embodiments.

[0054] A target tracking method based on multi-scale ReID network and dual-domain joint measurement for complex scenes. The overall process is as follows Figure 1 The process includes steps S1-S3, which are as follows:

[0055] S1: Construction and training of the YOLOv5 object detection model. This model is built based on the YOLOv5 architecture. By performing normalized coordinate transformation and adaptive anchor box optimization on the annotated dataset, combined with the Mosaic data augmentation strategy, the model's generalization capability is improved. Bounding box regression training is performed using the CIoU loss function, ultimately generating a detection model that outputs object location coordinates and confidence scores, providing an initial positioning benchmark for subsequent processing.

[0056] S2: Design an improved ReID network, IncepSPA-DSC, that fuses deep separable convolution with a spatial pyramid channel attention mechanism. Leveraging a trained object detection model, the network extracts target image regions from video sequences and constructs a temporally correlated set of target feature samples. The key improvements include the construction of a multi-scale feature fusion mechanism and an attention enhancement module. While maintaining real-time processing speed, the network generates a highly discriminative, interference-resistant, and cross-frame consistent 2048-dimensional feature vector, enabling efficient encoding and matching of target identity features.

[0057] S3: DeepTrack-SPAE tracking framework is proposed, and the model structure is shown in the figure below. Figure 2 The figure shows the overall processing flow of this method in multi-target tracking tasks, integrating core modules such as Kalman filter prediction, appearance feature extraction, three-stage matching association and trajectory update to achieve stable tracking of targets in complex scenarios. The process specifically includes the following key steps:

[0058] S31: A Kalman filter (KF) is used to predict and update the trajectory state for existing trajectories. A linear motion model is constructed using the target's historical state vector (including position, size, and velocity components) to infer the predicted trajectory for the current frame. In each frame, the KF module first outputs the predicted target state based on a linear optimal estimate. The observations from the detection module are then input, and the trajectory is dynamically corrected using the Kalman gain. The estimated value and error covariance matrix are updated to obtain the optimal target state estimate.

[0059] S32: An innovative three-stage multi-dimensional matching strategy is designed, which sequentially integrates appearance feature similarity, geometric position overlap (IoU), and neighborhood distance information to gradually associate trajectories with detection results.

[0060] Finally, successfully matched tracks will be updated with the KF, unmatched detection results will be used to create new tracks, and tracks that have not been confirmed for a long time will be deleted. This process enables continuous and robust tracking of targets under complex occlusions and multi-target interference.

[0061] Specifically, in step S2, an improved ReID network model (Inception with SpatialPyramid Attention and Depthwise Separable Convolution Model, IncepSPA-DSC) is constructed. The IncepSPA-DSC network structure is as follows: Figure 3 As shown in the figure, Inception is used as the backbone network, and the depth-wise separable convolution and spatial pyramid attention mechanisms are introduced on this basis. The specific design is as follows:

[0062] S21: At the input end, the target detected by the target detection model is processed to obtain a target recognition dataset as the input of the network.

[0063] S22: The first stage begins with two convolution blocks. The specific computational process involves: the input image first undergoes a 32-channel, 3x3 kernel, and 2x2 stride convolution, followed by a Relu convolution. This convolution and Relu convolution are then repeated again. The result is the output of this stage.

[0064] S23: The second stage consists of 12 blocks. The specific calculation process is as follows: First, the input of the first block is the output of the first stage. The result is obtained through the block layer. Then, a 1×1 convolution is performed on the output of the previous block. The result is channel-wise concatenated with the output of the current block. The concatenated result serves as the input of the next block, thus implementing a residual connection and avoiding loss of semantic information.

[0065] Among them, the structure of each Block is as follows Figure 4 As shown in the figure, the specific calculation process is as follows: first, a ReLU activation function is performed, followed by a depthwise separable convolution block (SeparableConv). Next, two more ReLU activations and depthwise separable convolution blocks (SeparableConv) are performed. After that, a max pooling operation is performed, and finally, a SPA block is applied as an attention mechanism module. After these steps, the result of the block is output.

[0066] Among them, SeparableConv represents the depth-separable convolution block, the structure is as follows Figure 5 As shown in the figure, the specific calculation process is as follows: First, a convolution with the number of input channels equal to the number of output channels is used to process each channel independently, which is called depthwise convolution. Then, a 1×1 convolution is used to convert the number of output channels of the depthwise convolution to the required number of output channels, which is called pointwise convolution.

[0067] The SPA block is a module for the spatial pyramid pooling channel attention mechanism, and its structure is as follows Figure 6 As shown. The specific calculation process is: first, a normal convolution is performed to match the number of channels; then, the input feature map is pooled into three different sizes through three adaptive average pooling layers of different scales. Next, the outputs of the three pooling layers are spliced ​​in the channel dimension for feature fusion. Next, the spliced ​​feature vector is mapped to a lower dimension through a fully connected layer and processed using the ReLU activation function. Subsequently, the feature vector is mapped back to the original dimension through another fully connected layer, and the output is limited to the range of [0,1] through the Sigmoid activation function to generate a channel attention weight vector. Each element in this vector represents the weight of the corresponding channel in the input feature map. Finally, the channel attention weight vector is multiplied element-by-element with the input feature map and output, thereby completing the application of the channel attention mechanism.

[0068] S24: The input of the third stage comes from the output of the previous stage. The specific calculation process is: first, the number of channels is changed to 2048 through ordinary convolution, then batch normalization and ReLU activation are performed; then global average pooling is performed, and finally a 2048-dimensional feature vector f is output through the fully connected layer. det (d j ).

[0069] Regarding step S32, the present invention innovatively adds a new matching process based on neighborhood distance (Euclidean distance) on the basis of the StrongSORT matching process, and the structure is as follows: Figure 7 As shown, the specific design is as follows:

[0070] S321: Appearance feature f based on the detection frame det (d j ) and the appearance features of the trajectory estimation box f trk (t i ) Calculate the cosine distance and construct the cost matrix C using the minimum cost matching method cos , where C cos [i,j] is the trajectory t i With the detection box d j The cosine distance between them, the cost matrix Ccos The construction formula is:

[0071]

[0072] C cos (i,j)=1-cosine_similarity(f det (d j ),f trk (t i ))

[0073] Among them, cosine_similarity(f det (d j ),f trk (t i )) is the cosine similarity formula.

[0074] S322: Calculate the Mahalanobis distance using the motion features of the detection frame and the trajectory estimation frame, and C cos Perform weighted combination to obtain the new cost matrix C mahal , its construction formula is:

[0075] S=P i +P j

[0076]

[0077] Among them, d M (z i ,z j ) represents the Mahalanobis distance calculation formula, z i and z j They represent the state vector of the i-th trajectory and the state vector of the j-th detection, respectively, P i and P j They represent the covariance matrix of the i-th trajectory and the covariance matrix of the j-th detection, respectively. gated_cost is the gating cost used to indicate an infeasible match, and gating_threshold represents the gating threshold.

[0078] S323: Using the Hungarian algorithm to calculate the cost matrix C cost =C mahal (i,j) solves the optimal allocation problem and obtains the trajectory index set and detection index collection pass The matched trajectory and detection box set M can be obtained (1) ={(t i ,d j )}, and the set of unmatched trajectories and unmatched detection box set The formula is:

[0079]

[0080] S324: For unmatched trajectory sets and unmatched detection box set Use the IoU between the track box and the detection box to construct the cost matrix C IoU , use the Hungarian matching algorithm to perform the second matching to obtain the matching pair M (2) ={(t i ,d j )} and unmatched trajectory set and unmatched detection box set C IoU The construction formula is:

[0081]

[0082] in, and Respectively represent the tth i The bounding box of the dth track (expressed in tlwh form) and the dth track j The formula for obtaining the matching set, the unmatched track set, and the unmatched detection box set is the same as that in S323.

[0083] S325: Use Euclidean distance to compare the above unmatched trajectory sets and unmatched detection box set Perform the third matching, calculate the Euclidean distance between multiple candidate target frames, set the distance threshold, and when the distance between the two is less than the specified threshold, it is considered to be a matching target, and the final matching pair M is obtained accordingly. (3) ={(t i ,d j )} and unmatched trajectory set and unmatched detection box set The Euclidean distance construction formula is:

[0084]

[0085] Among them, d E (t i ,d j ) represents the Euclidean distance calculation formula, t i and d j denote the state vector of the i-th trajectory and the state vector of the j-th detection, respectively.

[0086] S326: Unmatched detection box d jA new trajectory is generated and set to the unconfirmed state Tentative, without matching the trajectory t i Will increase the unmatched age When its unmatched age exceeds the maximum age threshold max Or when it is in an unconfirmed state, the track will be destroyed. The formula is:

[0087]

[0088] Through these steps, the detection bounding box of the current frame is matched to the trajectory bounding box, and the matching results are used to update the motion model. Unmatched detections and trajectories are handled accordingly. The third matching mechanism, Euclidean distance matching, demonstrates excellent robustness when handling complex dynamic scenes such as large numbers of objects, object occlusions, and object size changes.

[0089] The following describes in detail the implementation process of the present invention for tracking multiple fish targets in the MOT17[1] and MOT20[2] datasets with reference to the examples.

[0090] Example 1: Real-time multi-target tracking on the MOT17 dataset

[0091] The specific steps of this embodiment are as follows: First, use ImageNet [3] A pre-trained YOLOv5s object detection model was used. The IncepSPA-DSC model was then trained on the object recognition dataset converted from the MOT17 dataset to obtain an appearance feature extraction model. During the object tracking process, the video was passed to the object tracking model and processed frame by frame. The steps for processing each frame are as follows:

[0092] (1) Target detection stage: Perform target detection on the current frame and obtain the target detection frame d k =[x k ,y k ,a k ,h k ] and its confidence score C k .

[0093] (2) Appearance feature extraction stage: Use the pre-trained IncepSPA-DSC to extract the appearance features f of the detection target det (d j ).

[0094] (3) Trajectory initialization and prediction stage: 1) Create a new trajectory. For the newly detected target, we define its detection box d k , confidence C k and appearance features f det (d j) is passed to DeepTrack-SPAE; 2) Kalman filter initialization: inside DeepTrack-SPAE, the detection box and confidence information are used to initialize the target's state estimation vector X0 and state covariance matrix P0; 3) Motion state estimation: based on the state estimation X of the current frame k-1 and the covariance matrix P k-1 , the Kalman filter predicts the state estimation vector of the target in the next frame and covariance matrix The formula is:

[0095] (4) Inter-frame matching stage: 1) Preliminary matching, calculate the cosine distance between the appearance features of the detection frame and the trajectory estimation frame to obtain the cosine distance cost matrix C cos , combined with the cosine distance cost matrix to perform Mahalanobis distance C mahal The calculation of the Hungarian algorithm is used to obtain the preliminary matching trajectory detection set M (1) ={(t i ,d j )} and the unmatched trajectory set and unmatched detection box set 2) Secondary matching: for unmatched items, use the IoU cost matrix C IoU Perform secondary matching and use the Hungarian algorithm again to obtain the successfully matched trajectory M (2) ={(t i ,d j )} and the remaining unmatched trajectories and detection box 3) Three-way matching: for the unmatched items in the second matching, three-way matching is performed using the Euclidean distance, and the Hungarian algorithm is used to obtain the final successful matching trajectory M (3) ={(t i ,d j )} and the remaining unmatched trajectories and detection box

[0096] (5) Trajectory management stage: for unmatched trajectories Its "age" attribute increases by 1. If the maximum age is exceeded or the track is in the "unconfirmed" state, the track is deleted; for unmatched detection frames Generate a new track and mark it as "unconfirmed". When the track successfully matches the detection box for three consecutive frames, change the track status to "confirmed".

[0097] (6) State update stage: The successfully matched trajectory and detection box M (3) ={(t i ,d j)}, calculate the observation matrix H of the detection box k and the observation covariance matrix R k , using the calculated Kalman gain K k State estimation and the state estimation covariance matrix Update and finally get the updated trajectory state vector X k and the state estimation covariance matrix P k , the formula is:

[0098] (7) Continuous iteration phase: For each frame of the video, the above steps of target detection, feature extraction, trajectory initialization and prediction, inter-frame matching, trajectory management, and state update are repeated until the video processing is completed.

[0099] Finally, the record of each trajectory is saved in .txt format and compared with the gt.txt file in the MOT17 dataset. Indicators such as MOTA, IDF1, HOTA, and ID Switch are calculated to evaluate the performance of the target tracking model.

[0100] The experimental results of the DeepTrack-SPAE algorithm in this embodiment are shown in Table 1.

[0101] Table 1 Experimental results of DeepTrack-SPAE on the MOT17 dataset

[0102]

[0103] Experimental results on the MOT17 dataset demonstrate that the DeepTrack-SPAE algorithm demonstrates significant advantages over StrongSORT in multi-target tracking performance. Its HOTA, MOTA, and IDF1 improve by 0.9, 2.1, and 1.2 percentage points, respectively, demonstrating superior performance in overall tracking accuracy and detection and association consistency. Furthermore, the number of ID switches decreases from 942 to 770, demonstrating a significant improvement in the algorithm's identity preservation stability. These data fully demonstrate the effectiveness of the DeepTrack-SPAE model in feature modeling and target association, validating its potential for practical application in complex scenarios.

[0104] Example 2: Real-time multi-target tracking on the MOT20 dataset

[0105] The specific steps of this embodiment are consistent with those of Example 1, except that MOT17 is replaced with a more complex dataset with denser targets, MOT20, and target tracking is performed according to the process in Example 1. The final experimental results are shown in Table 2.

[0106] Table 2 Experimental results of DeepTrack-SPAE on the MOT20 dataset

[0107]

[0108] According to the experimental results on the MOT20 dataset in Table 2, DeepTrack-SPAE still demonstrates superior multi-target tracking performance compared to StrongSORT in complex scenes with frequent target occlusion. DeepTrack-SPAE improves the three core indicators of HOTA, MOTA, and IDF1 by 1.4, 1.6, and 2.0 percentage points, respectively, demonstrating significant improvements in the consistency of target detection and association, overall tracking accuracy, and identity preservation. In particular, in the MOT20 dataset with frequent occlusion, the number of ID switches (IDsw) dropped significantly from 1950 to 1194, a reduction of approximately 39%, indicating that the method has greater robustness and stability when dealing with target interactions, overlaps, and occlusions. This further verifies the effectiveness of DeepTrack-SPAE in modeling target spatial-appearance features and optimizing matching strategies, enabling more reliable multi-target tracking tasks in high-density and complex environments.

[0109] References:

[0110] [1]A.Milan, L.Leal-Taix′e, I.Reid, S.Roth, and K.Schindler.Mot16:Abenchmark for multi-object tracking.arXiv preprint arXiv:1603.00831,2016.1,5

[0111] [2] P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taix′e. Mot20: A benchmark for multi object tracking incrowded scenes. arXiv preprint arXiv:2003.09003, 2020.1,5

[0112] [3]Krizhevsky A,Sutskever I,Hinton G E.Imagenet classification with deep convolutional neural networks[C] / / Advances in neural informationprocessing systems.2012:1097-1105.

[0113] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A target tracking method using a multi-scale ReID network and dual-domain joint metric in complex scenes, characterized by: The following steps are involved: S1: Construction and training of the YOLOv5 object detection model. This model is constructed based on the YOLOv5 architecture. By performing normalized coordinate transformation and adaptive anchor box optimization on the annotated dataset, the model's generalization capability is improved in conjunction with the Mosaic data augmentation strategy. Bounding box regression training is performed using the CIoU loss function, ultimately generating a detection model that outputs target location coordinates and confidence scores, providing an initial positioning benchmark for subsequent processing. S2: Design an improved ReID network IncepSPA-DSC that integrates depthwise separable convolution and spatial pyramid channel attention mechanism. It uses the trained object detection model to extract the target image region in the video sequence and construct a temporally correlated target feature sample set. S3: Build the DeepTrack-SPAE tracking framework, including the following steps S31-S32: S31: For existing trajectories, a Kalman filter (KF) is used to predict and update the trajectory state. A linear motion model is constructed using the target's historical state vector to infer the predicted trajectory for the current frame. In each frame, the KF module first outputs the target predicted state based on the linear optimal estimate. Then, the observation value from the detection module is input, and the trajectory is dynamically corrected using the Kalman gain. The estimated value and error covariance matrix are updated to obtain the optimal target state estimate. S32: A three-stage multi-dimensional matching strategy is implemented, integrating appearance feature similarity, geometric position overlap (IoU), and neighborhood distance information in sequence, and gradually completing the association between the trajectory and the detection results.

2. The target tracking method according to claim 1, characterized in that In step S2, an improved IncepSPA-DSC is constructed, using Inception as the backbone network. On this basis, the depthwise separable convolution and spatial pyramid attention mechanisms are introduced, as follows: S21: At the input end, the target detected by the target detection model is processed to obtain a target recognition dataset as the input of the network; S22: The first stage first passes through two convolution blocks. The specific calculation process includes: the input image first passes through a 32-channel, 3*3 convolution kernel, and a stride of 2*2, and then undergoes ReLu. Then, the same convolution and ReLu are repeated again. The result is used as the output of this stage. S23: The second stage consists of 12 blocks. The specific calculation process is as follows: First, the input of the first block is the output of the first stage. The result is obtained by passing through the block layer. Then, a 1×1 convolution is performed on the output of the previous block. The result is channel-wise concatenated with the output of the current block. The concatenated result will be used as the input of the next block, thus realizing residual connection and avoiding loss of semantic information. S24: The input of the third stage comes from the output of the previous stage; the specific calculation process is: first, the number of channels is changed to 2048 through ordinary convolution, then batch normalization and ReLU activation are performed; then global average pooling is performed, and finally a 2048-dimensional feature vector f is output through the fully connected layer. det (d j ).

3. The target tracking method according to claim 2, characterized in that The specific calculation process of each Block block is as follows: first, a ReLU activation function is performed, and then a depth-wise separable convolution block (SeparableConv) is applied; then, two ReLU activation and depth-wise separable convolution block (SeparableConv) operations are performed; then, a maximum pooling (MaxPooling) operation is performed, and finally, the SPA block is applied as the attention mechanism module; After these steps, the result of the Block is output.

4. The target tracking method according to claim 3, characterized in that SeparableConv represents a depth-wise separable convolution block. Its specific calculation process is as follows: first, a convolution with the number of input channels equal to the number of output channels is used to process each channel independently, which is called depth-wise convolution; then, a 1×1 convolution is used to convert the number of output channels of the depth-wise convolution to the required number of output channels, which is called point-wise convolution.

5. The target tracking method according to claim 3, characterized in that: The SPA block is a module for the spatial pyramid pooling channel attention mechanism. The specific calculation process is as follows: first, a normal convolution is performed to match the number of channels; then, the input feature map is pooled into three different sizes through three adaptive average pooling layers of different scales; then, the outputs of the three pooling layers are spliced ​​in the channel dimension for feature fusion; next, the spliced ​​feature vector is mapped to a lower dimension through a fully connected layer and processed using the ReLU activation function; then, another fully connected layer is used to map the feature vector back to the original dimension, and the Sigmoid activation function is used to limit the output to the range of [0,1] to generate a channel attention weight vector; each element in the vector represents the weight of the corresponding channel in the input feature map; finally, the channel attention weight vector is multiplied element-by-element with the input feature map and output, thereby completing the application of the channel attention mechanism.

6. The target tracking method according to claim 3, characterized in that: In step S32, the specific steps are as follows: S321: Appearance feature f based on the detection frame det (d j ) and the appearance features of the trajectory estimation box f trk (t i ) Calculate the cosine distance and construct the cost matrix C using the minimum cost matching method cos , where C cos [i,j] is the trajectory t i With the detection box d j The cosine distance between them, the cost matrix C cos The construction formula is: C cos (i,j)=1-cosine_similarity(f det (d j ),f trk (t i )) Among them, cosine_similarity(f det (d j ),f trk (t i )) is the cosine similarity formula; S322: Calculate the Mahalanobis distance using the motion features of the detection frame and the trajectory estimation frame, and C cos Perform weighted combination to obtain the new cost matrix C mahal , its construction formula is: S=P i +P j Among them, d M (z i ,z j ) represents the Mahalanobis distance calculation formula, z i and z j They represent the state vector of the i-th trajectory and the state vector of the j-th detection, respectively, P i and P j They represent the covariance matrix of the i-th trajectory and the covariance matrix of the j-th detection respectively, gated_cost is the gating cost used to indicate an infeasible match, and gating_threshold represents the gating threshold; S323: Using the Hungarian algorithm to calculate the cost matrix C cost =C mahal (i,j) solves the optimal allocation problem and obtains the trajectory index set and detection index collection pass The matched trajectory and detection box set M can be obtained (1) ={(t i ,d j )}, and the set of unmatched trajectories and unmatched detection box set The formula is: S324: For unmatched trajectory sets and unmatched detection box set Use the IoU between the track box and the detection box to construct the cost matrix C IoU , use the Hungarian matching algorithm to perform the second matching to obtain the matching pair M (2) ={(t i ,d j )} and unmatched trajectory set and unmatched detection box set C IoU The construction formula is: in, and Respectively represent the tth i The bounding box of the dth track (expressed in tlwh form) and the dth track j The formula for obtaining the matching set, the unmatched trajectory set, and the unmatched detection frame set is the same as that in S323; S325: Use Euclidean distance to compare the above unmatched trajectory sets and unmatched detection box set Perform the third matching, calculate the Euclidean distance between multiple candidate target frames, set the distance threshold, and when the distance between the two is less than the specified threshold, it is considered to be a matching target, and the final matching pair M is obtained accordingly. (3) ={(t i ,d j )} and unmatched trajectory set and unmatched detection box set The Euclidean distance construction formula is: Among them, d E (t i ,d j ) represents the Euclidean distance calculation formula, t i and d j denote the state vector of the i-th trajectory and the state vector of the j-th detection respectively; S326: Unmatched detection box d j A new trajectory is generated and set to the unconfirmed state Tentative, without matching the trajectory t i Will increase the unmatched age When its unmatched age exceeds the maximum age threshold max Or when it is in an unconfirmed state, the track will be destroyed. The formula is: Through the above steps, the detection frame of the current frame is matched with the trajectory frame, and the matching result can be used for subsequent motion model updates.

Images