Multi-target tracking identity recovery method based on lightweight feature embedding

Abstract

The invention discloses a multi-target tracking identity recovery method based on lightweight feature embedding, and the method comprises the following steps: inputting video frame data into an enhanced detector of a newly-added lightweight multi-scale feature fusion sub-network in a check part, carrying out the target detection and feature extraction, and outputting a result; and inputting the result into an improved tracker for processing, performing two-stage matching on the result by the improved tracker to obtain an activated track, performing third-stage matching on the activated track to obtain a matched lost track, and finally outputting a tracking result of a track state in the matched lost track. And the lost ID feature library carries out real-time cleaning on the lost track. According to the method, feature extraction and detection are synchronously carried out through the enhanced detector, the newly added lightweight feature fusion sub-network flexibly adapts to various detection-based tracking algorithms, the problem of trajectory loss is effectively solved, and the robustness of trajectory data is enhanced.

Description

Technical Field

[0001] This invention relates to the field of multi-target tracking, and in particular to a multi-target tracking identity recovery method based on lightweight feature embedding. Background Technology

[0002] Multi-target tracking technology is widely used in scenarios such as intelligent monitoring and autonomous driving. Detection-based tracking paradigms are currently the mainstream, and the combination of YOLO (a computer vision detection framework) series detectors with trackers such as ByteTrack (a tracking framework) is highly favored due to its good balance.

[0003] However, in complex scenarios, when targets are severely occluded or move intersecting, existing technologies are prone to target loss and misassignment of IDs upon target reappearance, a phenomenon known as "ID hopping." This severely disrupts the continuity of the trajectory and reduces the usability of tracking data. Current solutions primarily involve introducing complex ReID (Re-identification) models. One common approach is to introduce independent, complex re-identification networks to extract highly discriminative features. For example, accuracy can be improved by fusing global and high-frequency features. However, this significantly increases computational burden, making it difficult to meet the demands of real-time applications. Another approach is to abandon purely geometric ReID methods, such as FastTracker, and attempt to rely entirely on motion models, geometric constraints, and scene priors for tracking. While this improves speed, its discriminative ability is limited in scenarios with densely packed, similar-looking targets and irregular motion patterns. In summary, existing technologies still suffer from the following specific problems:

[0004] 1. Computational redundancy problem: Independent ReID models and detection models usually have independent backbone networks, resulting in the extraction of features twice for the same frame of image, which generates huge computational waste. This is the main source of the real-time bottleneck.

[0005] 2. Feature alignment issue: There is a risk of spatiotemporal misalignment between the detection box and ReID features during asynchronous processing. Even a slight offset of the detection box may cause the extracted features to not belong to the same target, affecting the matching accuracy.

[0006] 3. Feature efficiency problem: General ReID models pursue generalization, and their feature dimensions are high (usually 512 dimensions or more) and information is redundant. However, for identity recognition in specific tracking scenarios, there are a large number of unnecessary feature dimensions.

[0007] 4. Engineering integration complexity issues: Maintaining two independent models and processing the data flow between them increases the complexity of system deployment, debugging and optimization. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a multi-target tracking identity recovery method based on lightweight feature embedding.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] A multi-target tracking identity recovery method based on lightweight feature embedding includes the following steps:

[0011] S1. Construct a three-level cascaded network architecture of “backbone-neck-head”, and add a lightweight multi-scale fusion feature extraction module in the neck part to obtain an enhanced detector;

[0012] Acquire video frame data, input the video frame data into the enhanced detector for target detection and feature extraction, and output the detection results and feature extraction results;

[0013] S2. Input the detection results and feature extraction results into the improved tracker for processing. The improved tracker performs two-level matching on the detection results and feature extraction results to obtain the activated trajectory. Then, it performs a third-level matching on the activated trajectory to determine whether the trajectory status of the activated trajectory is active or lost and whether the trajectory ID has been added. When the trajectory status of the activated trajectory is active and the trajectory ID has not been added, the tracking result of the trajectory status of the activated trajectory is output.

[0014] When the activated trajectory is in the state of being lost and a new trajectory ID is added, obtain the lost trajectory and the trajectory with the new ID, and proceed to step S3;

[0015] S3. Store the trajectory ID of the lost trajectory into the lost ID feature library, extract the feature vector of the target feature in the lost trajectory, obtain the extraction result, compare the extraction result with the lost ID feature library, obtain the unmatched detection box, calculate the cosine similarity between the unmatched detection box and the trajectory of the newly added ID, select the highest cosine similarity, judge and match the highest cosine similarity, obtain the matched lost trajectory, and finally output the tracking result of the trajectory state in the matched lost trajectory.

[0016] S4. The automatic cleanup mechanism of the lost ID feature library determines in real time whether the track ID in the lost track has reached the cleanup period. When the cleanup period is reached, the expired track ID in the lost track is cleaned up and then the next frame is processed. Otherwise, the next frame data is processed directly.

[0017] In step S1, the lightweight multi-scale fusion feature extraction module is as follows: the fifth CSP2Conv layer is a convolutional layer with 256 input channels and 128 output channels. It receives the 40×40×256 feature map output from the third CSP2Conv layer as input, and reduces the number of channels from 256 to 128 through feature compression, outputting a 40×40×128 feature map. Then, the first Concat layer combines the 40×40×128 feature map output from the fifth CSP2Conv layer with the feature information from the third CSP2Conv layer, the fourth CSP2Conv layer, and the second Usample layer. The process involves fusion, connecting along the channel dimension to form a 40×40×(128+3) enhanced feature map; then, through the first Usample layer (upsampling), the size of the 40×40×(128+3) enhanced feature map is enlarged from 40×40 to 80×80 while maintaining 128 channels, resulting in an 80×80×128 feature map; the sixth Conv layer is a convolutional layer with 128 output channels, further refining and extracting features from the 80×80×128 feature map output by the first Usample layer, maintaining the 80×80×128 feature map output size; finally, the Global... The Pooling layer compresses the 80×80×128 feature map output from the sixth Conv layer in spatial dimension, eliminating spatial location information and outputting a 1×1×128 feature vector. Then, the 1×1×128 feature vector output from the Global Pooling layer is input into the second Concat layer to ensure the dimensionality consistency of the output features, outputting a 1×1×128 feature vector. Finally, the L2 Normalization layer performs L2 normalization on the 1×1×128 feature vector output from the second Concat layer, converting the feature vector to unit length, and finally outputting a normalized 128-dimensional feature vector as the feature extraction result.

[0018] In step S1, the neck part is a multi-scale feature fusion module, specifically: the second Usample layer receives the 20×20×1024 feature map output by the fourth CSP2Conv layer as input, enlarges the size of the 20×20×1024 feature map by 2 times, and outputs a 40×40×1024 feature map; the third Concat layer concatenates the 40×40×1024 feature map output by the second Usample layer and the 40×40×512 feature map output by the third CSP2Conv layer in the channel dimension, and outputs a fused feature map of 40×40×1024; the sixth CSP2Conv layer performs feature refinement processing on the fused feature map of 40×40×1024 output by the third Concat layer, and outputs a fused feature map with 512 channels and a feature map size of 40×40×512.

[0019] The fused features output from the sixth CSP2Conv layer are input into the third Usample layer for upsampling, which enlarges the feature map size of the fused features by a factor of 2, resulting in an 80×80×512 feature map. The fourth Concat layer concatenates the 80×80×512 feature map output from the third Usample layer with the 80×80×256 feature map output from the second CSP2Conv layer along the channel dimension, forming an 80×80×512 fused feature map. The seventh CSP2Conv layer refines the 80×80×512 fused feature map output from the fourth Concat layer, outputting a fused feature map with 256 channels and a feature map size of 80×80×256.

[0020] The fused features output from the seventh CSP2Conv layer are input into the seventh Conv layer for processing. The seventh Conv layer is a convolutional layer with 256 output channels. The fused features output from the seventh CSP2Conv layer are further compressed, resulting in a 40×40×256 feature map. The fifth Concat layer concatenates the 40×40×256 feature map output from the seventh Conv layer with the 40×40×512 fused features output from the sixth CSP2Conv layer, forming a 40×40×768 fused feature map. The eighth CSP2Conv layer refines the 40×40×768 fused feature map output from the fifth Concat layer, outputting a fused feature map with 512 channels and a size of 40×40×512.

[0021] The fused features output from the eighth CSP2Conv layer are input into the eighth Conv layer for processing. The eighth Conv layer is a convolutional layer with 512 output channels. The fused features output from the eighth CSP2Conv layer are further compressed, resulting in a 20×20×512 feature map. The sixth Concat layer concatenates the 20×20×512 feature map output from the eighth Conv layer with the 20×20×1024 feature map output from the fourth CSP2Conv layer to form a 20×20×1024 fused feature map. The ninth CSP2Conv layer refines the 20×20×1024 fused feature map output from the sixth Concat layer, resulting in a 20×20×1024 fused feature map.

[0022] In step S3, the formula for calculating the cosine similarity is as follows:

[0023] ;

[0024] in, The feature vector of the unmatched detection box in the lost trajectory. The feature vector of the trajectory ID in the trajectory with the newly added ID;

[0025] Calculate the cosine similarity between the unmatched detection boxes and the trajectories of all newly added IDs, and select the K with the highest cosine similarity as candidate matches; where K=1, that is, only the highest cosine similarity match is considered; based on the highest cosine similarity threshold, decide whether to recover the trajectory ID of the lost trajectory, and finally obtain the matched lost trajectory: if the highest cosine similarity is greater than or equal to the threshold, recover the original trajectory of the trajectory ID in the lost trajectory, reset the life cycle of the trajectory ID, and remove the trajectory ID record from the lost ID feature library; if it is less than the threshold, reassign a new trajectory ID to the unmatched detection boxes and create a new trajectory, and record the features of the new trajectory; finally, output the tracking result of the trajectory state in the matched lost trajectory.

[0026] Step S4 specifically involves: In the lost ID feature database, the loss time of all lost trajectories is checked in real time. The loss time of a lost trajectory is obtained by subtracting the time frame number of its last occurrence from the current frame number of the lost trajectory. If the loss time of a lost trajectory exceeds the maximum loss time, the trajectory ID, trajectory, and target features of the lost trajectory are removed from the lost ID feature database. The formula for calculating the maximum loss time is:

[0027] ;

[0028] ;

[0029] Where vx is the instantaneous horizontal velocity of the activated trajectory, vy is the instantaneous vertical velocity of the activated trajectory, w is the predicted bounding box length of the activated trajectory, and h is the predicted bounding box width of the activated trajectory.

[0030] Meanwhile, this invention provides:

[0031] A server includes a processor and a memory, the memory storing at least one program that is loaded and executed by the processor to implement the above-described multi-target tracking identity recovery method based on lightweight feature embedding.

[0032] A computer-readable storage medium storing at least one program that is loaded and executed by a processor to implement the above-described multi-target tracking identity recovery method based on lightweight feature embedding.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] 1. This invention, through an enhanced detector and an improved tracker, can accurately recover the original ID when the target reappears after a brief occlusion, thereby recovering lost trajectories and enhancing the usability of tracking trajectories.

[0035] 2. The enhanced detector of this invention enables the feature extraction and detection processes to be performed simultaneously, avoiding the repeated calculations of independent ReID models. The lightweight feature dimension is controlled within 128 dimensions, and the computational overhead increases by less than 5%, meeting the requirements of real-time applications.

[0036] 3. This invention has strong generalization ability and easy integration. The newly added lightweight multi-scale fusion feature extraction module serves as an enhancement plug-in for the tracker. It can flexibly adapt to a variety of detection-based tracking algorithms, does not depend on a specific detection model, and can be used in conjunction with mainstream detectors such as the YOLO series and Faster R-CNN. The parameter adjustment is simple and intuitive (similarity threshold, maximum loss time), which facilitates application deployment in different scenarios.

[0037] 4. This invention has high practical value, solves the most prominent ID jump problem in practical applications, improves the intelligence level of the monitoring system, and has broad application prospects in fields such as security monitoring, autonomous driving, and intelligent transportation. It provides an optimized solution that balances accuracy and efficiency for real-time multi-target tracking systems. Attached Figure Description

[0038] Figure 1 This is the overall flowchart.

[0039] Figure 2 This is a diagram of the enhanced detector network architecture.

[0040] Figure 3 To improve the two-stage matching flowchart of the tracker.

[0041] Figure 4 To improve the third-level matching flowchart of the tracker. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0043] like Figure 1 As shown, a multi-target tracking identity recovery method based on lightweight feature embedding includes the following steps:

[0044] S1. Construct a three-level cascaded network architecture of “backbone-neck-head”, and add a lightweight multi-scale fusion feature extraction module in the neck part to obtain an enhanced detector;

[0045] Acquire video frame data, input the video frame data into the enhanced detector for target detection and feature extraction, and output the detection results and feature extraction results;

[0046] S2. Input the detection results and feature extraction results into the improved tracker for processing. The improved tracker performs two-level matching on the detection results and feature extraction results to obtain the activated trajectory. Then, it performs a third-level matching on the activated trajectory to determine whether the trajectory status of the activated trajectory is active or lost and whether the trajectory ID has been added. When the trajectory status of the activated trajectory is active and the trajectory ID has not been added, the tracking result of the trajectory status of the activated trajectory is output.

[0047] When the activated trajectory is in the state of being lost and a new trajectory ID is added, obtain the lost trajectory and the trajectory with the new ID, and proceed to step S3;

[0048] S3. Store the trajectory ID of the lost trajectory into the lost ID feature library, extract the feature vector of the target feature in the lost trajectory, obtain the extraction result, compare the extraction result with the lost ID feature library, obtain the unmatched detection box, calculate the cosine similarity between the unmatched detection box and the trajectory of the newly added ID, select the highest cosine similarity, judge and match the highest cosine similarity, obtain the matched lost trajectory, and finally output the tracking result of the trajectory state in the matched lost trajectory.

[0049] S4. The automatic cleanup mechanism of the lost ID feature library determines in real time whether the track ID in the lost track has reached the cleanup period. When the cleanup period is reached, the expired track ID in the lost track is cleaned up and then the next frame is processed. Otherwise, the next frame data is processed directly.

[0050] In step S1, such as Figure 2 As shown, a three-level cascaded network of "backbone-neck-head" is constructed. A lightweight multi-scale fusion feature extraction module is added to the neck part. The backbone part is a multi-scale feature extraction module, the neck part is a multi-scale feature fusion module, and the head part is a multi-scale feature decoupling and prediction module.

[0051] A lightweight multi-scale fusion feature extraction module is used to generate compact 128-dimensional features. The fifth CSP2Conv layer is a convolutional layer with 256 input channels and 128 output channels. It receives the 40×40×256 feature map output from the third CSP2Conv layer as input, and reduces the number of channels from 256 to 128 through feature compression, outputting a 40×40×128 feature map. Then, the first Concat layer combines the 40×40×128 feature map output from the fifth CSP2Conv layer with the features from the third, fourth, and second Usample layers. Information is fused and connected along the channel dimension to form a 40×40×(128+3) enhanced feature map. Then, through the first Usample layer (upsampling), the size of the 40×40×(128+3) enhanced feature map is enlarged from 40×40 to 80×80, while maintaining 128 channels, resulting in an 80×80×128 feature map. The sixth Conv layer is a convolutional layer with 128 output channels, further refining and extracting features from the 80×80×128 feature map output by the first Usample layer, maintaining the 80×80×128 feature map output size. Finally, the Global... The Pooling layer compresses the 80×80×128 feature map output from the sixth Conv layer in spatial dimension, eliminating spatial location information and outputting a 1×1×128 feature vector. Then, the 1×1×128 feature vector output from the Global Pooling layer is input into the second Concat layer to ensure the dimensionality consistency of the output features, outputting a 1×1×128 feature vector. Finally, the L2Normalization layer performs L2 normalization on the 1×1×128 feature vector output from the second Concat layer, converting the feature vector to unit length, and finally outputting a normalized 128-dimensional feature vector as the feature extraction result.

[0052] The multi-scale feature extraction module is used to extract multi-scale feature representations from the input image. Through progressive downsampling and the application of the CSP2Conv layer, it realizes the progressive extraction of features from low-level to high-level, providing multi-scale feature representations for subsequent feature fusion and prediction. Specifically, the first Conv layer is a convolutional layer with 3 input channels and 64 output channels. It uses a 3×3 convolutional kernel to perform preliminary feature extraction on the input image, and the output feature map size is 320×320×64. The first Conv layer is responsible for capturing low-level visual features such as edges and textures.

[0053] The second Conv layer is a convolutional layer with 64 input channels and 128 output channels. The 320×320×64 feature map output by the first Conv layer is input into the second Conv layer for downsampling and feature extraction. Then, the first CSP2Conv layer is used to enhance the feature reuse capability and improve the computational efficiency. Finally, the feature map output by the first CSP2Conv layer is 160×160×128.

[0054] The third Conv layer is a convolutional layer with 128 input channels and 256 output channels. The 160×160×128 feature map output by the first CSP2Conv layer is input into the third Conv layer for downsampling and feature extraction. The feature reuse capability is then enhanced by the second CSP2Conv layer. Finally, the feature map output by the second CSP2Conv layer is 80×80×256. The third Conv layer captures mid-level semantic features and is used to detect mid-scale targets.

[0055] The fourth Conv layer is a convolutional layer with 256 input channels and 512 output channels. The 80×80×256 feature map output by the second CSP2Conv layer is input into the fourth Conv layer for downsampling and feature extraction. The feature reuse capability is then enhanced by the third CSP2Conv layer. Finally, the feature map output by the third CSP2Conv layer is 40×40×512. The fourth Conv layer captures high-level semantic features and is used to detect large-scale targets.

[0056] The fifth Conv layer is a convolutional layer with 512 input channels and 1024 output channels. The 40×40×512 feature map output by the third CSP2Conv layer is input into the fifth Conv layer for downsampling and feature extraction. Then, the feature reuse capability is enhanced by the fourth CSP2Conv layer. Finally, the feature map output by the fourth CSP2Conv layer has a size of 20×20×1024. The fifth Conv layer captures global contextual information, which is used to detect extremely large-scale targets or provide background information.

[0057] The multi-scale feature fusion module integrates feature maps of different scales output by the multi-scale feature extraction module. It enhances feature representation through upsampling and concatenation operations. Specifically, the second Usample layer receives the 20×20×1024 feature map output by the fourth CSP2Conv layer as input, enlarges the size of the 20×20×1024 feature map by 2 times, and outputs a 40×40×1024 feature map. The third Concat layer concatenates the 40×40×1024 feature map output by the second Usample layer with the 40×40×512 feature map output by the third CSP2Conv layer in the channel dimension, and outputs a fused feature map of 40×40×1024. The sixth CSP2Conv layer refines the fused feature map of 40×40×1024 output by the third Concat layer, and outputs a fused feature map with 512 channels and a feature map size of 40×40×512.

[0058] The fused features output from the sixth CSP2Conv layer are input into the third Usample layer for upsampling, which enlarges the feature map size of the fused features by a factor of 2, resulting in an 80×80×512 feature map. The fourth Concat layer concatenates the 80×80×512 feature map output from the third Usample layer with the 80×80×256 feature map output from the second CSP2Conv layer along the channel dimension, forming an 80×80×512 fused feature map. The seventh CSP2Conv layer refines the 80×80×512 fused feature map output from the fourth Concat layer, outputting a fused feature map with 256 channels and a feature map size of 80×80×256.

[0059] The fused features output from the seventh CSP2Conv layer are input into the seventh Conv layer for processing. The seventh Conv layer is a convolutional layer with 256 output channels. The fused features output from the seventh CSP2Conv layer are further compressed, resulting in a 40×40×256 feature map. The fifth Concat layer concatenates the 40×40×256 feature map output from the seventh Conv layer with the 40×40×512 fused features output from the sixth CSP2Conv layer, forming a 40×40×768 fused feature map. The eighth CSP2Conv layer refines the 40×40×768 fused feature map output from the fifth Concat layer, outputting a fused feature map with 512 channels and a size of 40×40×512.

[0060] The fused features output from the eighth CSP2Conv layer are input into the eighth Conv layer for processing. The eighth Conv layer is a convolutional layer with 512 output channels. The fused features output from the eighth CSP2Conv layer are further compressed, resulting in a 20×20×512 feature map. The sixth Concat layer concatenates the 20×20×512 feature map output from the eighth Conv layer with the 20×20×1024 feature map output from the fourth CSP2Conv layer to form a 20×20×1024 fused feature map. The ninth CSP2Conv layer refines the 20×20×1024 fused feature map output from the sixth Concat layer, resulting in a 20×20×1024 fused feature map.

[0061] The multi-scale feature decoupling and prediction module is used to decouple the feature map output by the multi-scale feature fusion module and predict the target bounding box coordinates and class probabilities respectively. Specifically, the 80×80×256 fused feature output from the seventh CSP2Conv layer is input into the ninth Conv layer with a 3×3 kernel and a stride of 1, and the output feature size remains unchanged, which is an 80×80×256 feature map. The 80×80×256 feature map output from the ninth Conv layer is input into the first Conv2d layer with a 1×1 kernel and a stride of 0 to perform bounding box coordinate regression and class prediction, and obtain the first prediction tensor.

[0062] The 40×40×512 fused feature output from the eighth CSP2Conv layer is input into the tenth Conv layer with a 3×3 kernel and a stride of 1, and the output feature size remains unchanged, resulting in a 40×40×512 feature map. The 40×40×512 feature map output from the tenth Conv layer is then input into the second Conv2d layer with a 1×1 kernel and a stride of 0 to perform bounding box coordinate regression and class prediction, thus obtaining the second prediction tensor.

[0063] The fused features of 20×20×1024 output from the ninth CSP2Conv layer are input into the eleventh Conv layer with a 3×3 kernel and a stride of 1, keeping the feature size unchanged, and the output is a 20×20×1024 feature map. The 20×20×1024 feature map output from the eleventh Conv layer is then input into the third Conv2d layer with a 1×1 kernel and a stride of 0 for bounding box coordinate regression and class prediction, resulting in the third prediction tensor. The first, second, and third prediction tensors are then output to obtain the detection result.

[0064] In step S2, such as Figure 3As shown, the improved tracker performs a two-stage matching of the detection results and feature extraction results output by the enhanced detector to obtain the activated trajectory. The trajectory information of the activated trajectory includes trajectory ID, trajectory state, target features, and the current frame. The lifecycle of the activated trajectory is managed, and eight-dimensional motion data is obtained through Kalman filtering, Hungarian algorithm, and other method libraries. The eight-dimensional motion data includes the predicted bounding box length w, predicted bounding box width h, center point x-coordinate position, center point y-coordinate position, lateral instantaneous velocity vx, longitudinal instantaneous velocity vy, width change rate vw, and height change rate vh of the activated trajectory. The eight-dimensional motion data list is accessed externally and used to improve the tracker's trajectory position prediction for the next frame.

[0065] like Figure 4 As shown, a third-level matching is performed on the activated trajectory: determining whether the trajectory state of the activated trajectory is active or lost, and whether the trajectory ID of the activated trajectory has been added. When the trajectory state is active and the trajectory ID has not been added, the tracking result of the trajectory state in the activated trajectory is output; when the trajectory state is lost and the trajectory ID has been added, the lost trajectory and the trajectory with the added ID are obtained. Among them, the lost trajectory includes bounding box information, confidence, current frame, and target features. The bounding box information and confidence of the lost trajectory are inherent attributes of the object returned by the improved tracker.

[0066] In step S3, the lost trajectory is input into the lost ID feature library. The target feature information of the lost trajectory is stored in key-value pairs: {track_id:{"feature":vector,"last_frame": frame_num}}. The time frame number of the last occurrence of the lost trajectory is recorded periodically to facilitate the removal of expired lost trajectories. Next, the feature vector of the target feature in the lost trajectory is extracted to obtain the extraction result. The extracted result is compared with the target feature information of the lost trajectory in the lost ID feature library to obtain unmatched detection boxes. The cosine similarity between the unmatched detection boxes and the trajectory with the newly added ID is calculated. The formula for calculating the cosine similarity is as follows:

[0067] ;

[0068] in, The feature vector of the unmatched detection box in the lost trajectory. The feature vector of the trajectory ID in the trajectory with the newly added ID;

[0069] Calculate the cosine similarity between the unmatched detection boxes and the trajectories of all newly added IDs, and select the K with the highest cosine similarity as candidate matches; where K=1, that is, only the highest cosine similarity match is considered; based on the highest cosine similarity threshold, decide whether to recover the trajectory ID of the lost trajectory, and finally obtain the matched lost trajectory. The highest cosine similarity threshold is the average of the scores of the target feature information of all stored lost trajectories, to prevent the matching result from being affected by the low or high quality of the scene image, and associate the lost trajectory with the corresponding trajectory ID of the lost trajectory; when the highest cosine similarity is greater than or equal to the threshold (e.g., 0.7), recover the original trajectory of the trajectory ID in the lost trajectory, reset the life cycle of the trajectory ID, and remove the trajectory ID record from the lost ID feature library; when it is less than the threshold, reassign a new trajectory ID to the unmatched detection box and create a new trajectory, and record the features of the new trajectory; finally, output the tracking result of the trajectory state in the matched lost trajectory.

[0070] Step S4 specifically involves: In the lost ID feature database, the loss time of all lost trajectories is checked in real time. The loss time of a lost trajectory is obtained by subtracting the time frame number of the last occurrence from the current frame number. If the loss time of a lost trajectory exceeds the maximum loss time, the trajectory ID, trajectory, and target features of the lost trajectory are removed from the lost ID feature database. The formula for calculating the maximum loss time is:

[0071] ;

[0072] ;

[0073] Where vx is the instantaneous horizontal velocity of the activated trajectory, vy is the instantaneous vertical velocity of the activated trajectory, w is the predicted bounding box length of the activated trajectory, and h is the predicted bounding box width of the activated trajectory.

[0074] Meanwhile, this invention provides:

[0075] A server includes a processor and a memory, the memory storing at least one program that is loaded and executed by the processor to implement the above-described multi-target tracking identity recovery method based on lightweight feature embedding.

[0076] A computer-readable storage medium storing at least one program that is loaded and executed by a processor to implement the above-described multi-target tracking identity recovery method based on lightweight feature embedding.

[0077] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A multi-target tracking identity recovery method based on lightweight feature embedding, characterized in that, Includes the following steps: S1. Construct a three-level cascaded network architecture of "backbone-neck-head", and add a lightweight multi-scale fusion feature extraction module in the neck part to obtain an enhanced detector; Acquire video frame data, input the video frame data into the enhanced detector for target detection and feature extraction, and output the detection results and feature extraction results; S2. Input the detection results and feature extraction results into the improved tracker for processing. The improved tracker performs two-level matching on the detection results and feature extraction results to obtain the activated trajectory. Then, it performs a third-level matching on the activated trajectory to determine whether the trajectory status of the activated trajectory is active or lost and whether the trajectory ID has been added. When the trajectory status of the activated trajectory is active and the trajectory ID has not been added, the tracking result of the trajectory status of the activated trajectory is output. When the activated trajectory is in the state of being lost and a new trajectory ID is added, obtain the lost trajectory and the trajectory with the new ID, and proceed to step S3; S3. Store the trajectory ID of the lost trajectory into the lost ID feature library, extract the feature vector of the target feature in the lost trajectory, obtain the extraction result, compare the extraction result with the lost ID feature library, obtain the unmatched detection box, calculate the cosine similarity between the unmatched detection box and the trajectory of the newly added ID, select the highest cosine similarity, judge and match the highest cosine similarity, obtain the matched lost trajectory, and finally output the tracking result of the trajectory state in the matched lost trajectory. S4. The automatic cleanup mechanism of the lost ID feature library determines in real time whether the track ID in the lost track has reached the cleanup period. When the cleanup period is reached, the expired track ID in the lost track is cleaned up and then the next frame is processed. Otherwise, the next frame data is processed directly.

2. The multi-target tracking identity recovery method based on lightweight feature embedding according to claim 1, characterized in that, In step S1, the lightweight multi-scale fusion feature extraction module is as follows: the fifth CSP2Conv layer is a convolutional layer with 256 input channels and 128 output channels. It receives the 40×40×256 feature map output from the third CSP2Conv layer as input, and reduces the number of channels from 256 to 128 through feature compression, outputting a 40×40×128 feature map. Then, the first Concat layer combines the 40×40×128 feature map output from the fifth CSP2Conv layer with the feature information from the third CSP2Conv layer, the fourth CSP2Conv layer, and the second Usample layer. The process involves fusion, connecting along the channel dimension to form a 40×40×(128+3) enhanced feature map; then, through the first Usample layer (upsampling), the size of the 40×40×(128+3) enhanced feature map is enlarged from 40×40 to 80×80 while maintaining 128 channels, resulting in an 80×80×128 feature map; the sixth Conv layer is a convolutional layer with 128 output channels, further refining and extracting features from the 80×80×128 feature map output by the first Usample layer, maintaining the 80×80×128 feature map output size; finally, the Global... The Pooling layer compresses the 80×80×128 feature map output from the sixth Conv layer in spatial dimension, eliminating spatial location information and outputting a 1×1×128 feature vector. Then, the 1×1×128 feature vector output from the Global Pooling layer is input into the second Concat layer to ensure the dimensionality consistency of the output features, outputting a 1×1×128 feature vector. Finally, the L2 Normalization layer performs L2 normalization on the 1×1×128 feature vector output from the second Concat layer, converting the feature vector to unit length, and finally outputting a normalized 128-dimensional feature vector as the feature extraction result.

3. The multi-target tracking identity recovery method based on lightweight feature embedding according to claim 1, characterized in that, In step S1, the neck part is a multi-scale feature fusion module, specifically: the second Usample layer receives the 20×20×1024 feature map output by the fourth CSP2Conv layer as input, enlarges the size of the 20×20×1024 feature map by 2 times, and outputs a 40×40×1024 feature map. The third Concat layer concatenates the 40×40×1024 feature map output from the second Usample layer with the 40×40×512 feature map output from the third CSP2Conv layer in the channel dimension, and outputs a fused feature map of 40×40×1024. The sixth CSP2Conv layer refines the 40×40×1024 fused feature map output by the third Concat layer, and outputs a fused feature map with 512 channels and a feature map size of 40×40×512. The fused features output from the sixth CSP2Conv layer are input into the third Usample layer for upsampling, which enlarges the feature map size of the fused features by 2 times, and outputs a feature map of 80×80×512. The fourth Concat layer concatenates the 80×80×512 feature map output from the third Usample layer with the 80×80×256 feature map output from the second CSP2Conv layer in the channel dimension to form a fused feature map of 80×80×512. The seventh CSP2Conv layer refines the 80×80×512 fused feature map output by the fourth Concat layer, and outputs a fused feature map with 256 channels and a feature map size of 80×80×256. The fused features output from the seventh CSP2Conv layer are input into the seventh Conv layer for processing. The seventh Conv layer is a convolutional layer with 256 output channels. The fused features output from the seventh CSP2Conv layer are further compressed to output a 40×40×256 feature map. The fifth Concat layer concatenates the 40×40×256 feature map output from the seventh Conv layer with the 40×40×512 fused feature map output from the sixth CSP2Conv layer to form a 40×40×768 fused feature map. The eighth CSP2Conv layer refines the 40×40×768 fused feature map output by the fifth Concat layer, and outputs a fused feature map with 512 channels and a feature map size of 40×40×512. The fused features output from the eighth CSP2Conv layer are input into the eighth Conv layer for processing. The eighth Conv layer is a convolutional layer with 512 output channels. The fused features output from the eighth CSP2Conv layer are further compressed to output a 20×20×512 feature map. The sixth Concat layer concatenates the 20×20×512 feature map output from the eighth Conv layer with the 20×20×1024 feature map output from the fourth CSP2Conv layer to form a 20×20×1024 fused feature map. The ninth CSP2Conv layer refines the 20×20×1024 fused feature map output by the sixth Concat layer, and outputs a fused feature map with a size of 20×20×1024.

4. The multi-target tracking identity recovery method based on lightweight feature embedding according to claim 1, characterized in that, In step S3, the formula for calculating the cosine similarity is as follows: ； in, The feature vector of the unmatched detection box in the lost trajectory. The feature vector of the trajectory ID in the trajectory with the newly added ID; Calculate the cosine similarity between the unmatched detection boxes and the trajectories of all newly added IDs, and select the K with the highest cosine similarity as candidate matches; where K=1, that is, only the highest cosine similarity match is considered; based on the highest cosine similarity threshold, decide whether to recover the trajectory ID of the lost trajectory, and finally obtain the matched lost trajectory: if the highest cosine similarity is greater than or equal to the threshold, recover the original trajectory of the trajectory ID in the lost trajectory, reset the life cycle of the trajectory ID, and remove the trajectory ID record from the lost ID feature library; if it is less than the threshold, reassign a new trajectory ID to the unmatched detection boxes and create a new trajectory, and record the features of the new trajectory; finally, output the tracking result of the trajectory state in the matched lost trajectory.

5. The multi-target tracking identity recovery method based on lightweight feature embedding according to claim 1, characterized in that, Specifically, step S4 involves checking the loss time of all lost trajectories in the lost ID feature database in real time. The loss time of a lost trajectory is obtained by subtracting the time frame number of the last occurrence from the current frame number of the lost trajectory. If the loss time of the lost trajectory exceeds the maximum loss time, the trajectory ID, trajectory, and target features of the lost trajectory will be removed from the lost ID feature library. The formula for calculating the maximum time lost is: ； ； Where vx is the instantaneous horizontal velocity of the activated trajectory, vy is the instantaneous vertical velocity of the activated trajectory, w is the predicted bounding box length of the activated trajectory, and h is the predicted bounding box width of the activated trajectory.

6. A server, characterized in that, The server includes a processor and a memory, the memory storing at least one program that is loaded and executed by the processor to implement the multi-target tracking identity recovery method based on lightweight feature embedding as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is loaded and executed by a processor to implement the multi-target tracking identity recovery method based on lightweight feature embedding as described in any one of claims 1 to 5.

Images