Vehicle tracking method in highway scene based on time-space fusion network

By constructing the STFN-ADM-MTHS network model and combining techniques such as adaptive decay memory, repulsion loss, and spatial-guided cross-temporal feature aggregation, the problem of missed detection in vehicle tracking in highway scenarios was solved, the vehicle tracking accuracy was improved, and a foundation was laid for multi-target vehicle tracking.

CN119919896BActive Publication Date: 2025-11-18SOUTHEAST UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510009903.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-01-03
Publication Date
2025-11-18
Estimated Expiration
2045-01-03

AI Technical Summary

Technical Problem

Existing technologies for vehicle tracking in highway scenarios suffer from vehicle omissions, especially under the influence of external conditions such as camera vibration and changes in lighting, resulting in insufficient vehicle tracking accuracy.

Method used

An STFN-ADM-MTHS network model was constructed, and the vehicle tracking model was optimized by combining techniques such as adaptive decay memory, repulsion loss, spatial-guided cross-temporal feature aggregation, and adaptive decay memory, thereby improving the accuracy of vehicle detection and tracking.

Benefits of technology

It effectively solves the problem of missed detection in the vehicle tracking process, improves vehicle tracking accuracy, and provides a foundation for multi-target vehicle tracking in highway scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119919896B_ABST
    Figure CN119919896B_ABST
Patent Text Reader

Abstract

The application discloses a vehicle tracking method in a highway scene based on a time-space fusion network, comprising the following steps: constructing a vehicle tracking data set in a highway scene; for the joint decoding structure of detection and tracking in a specific scene, a self-attention new target detection query module is constructed, the detection query range is narrowed, and the detection and tracking precision is improved; for the occlusion problem, repulsion loss is added in the tracking model; a space-guided cross-time feature aggregation module is constructed, random motion change rules caused by vibration are learned by using inter-frame trajectory difference, the occlusion of vehicles in different trajectories at the same time is considered, and self-adaptive fading memory is constructed to adaptively update the fading rate of long-term memory. The application can effectively track the vibration and occluded vehicles in the highway scene, solve the vehicle missing detection problem in the tracking process, improve the vehicle tracking precision, and provide a basis for subsequent realization of multi-target vehicle tracking in the highway scene.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the research field of smart highways and smart perception, specifically involving a vehicle tracking method in highway scenarios based on a time-space fusion network. Background Technology

[0002] Vehicle tracking is crucial for information perception in highway scenarios. Highways, exposed to the elements, experience significant light variations throughout the day, evening, and night, leading to severe light pollution in surveillance cameras. Furthermore, cameras are susceptible to vibration due to external conditions. Vehicle tracking technology can track vehicles in real-time, quickly identify abnormal behavior or potential collision risks, and issue timely alerts to prevent accidents and improve traffic safety. Therefore, researching vehicle tracking in surveillance video streams within traffic scenarios involves, based on target detection and tracking, and considering the characteristics of traffic scenarios, achieving vehicle detection and data association to obtain vehicle trajectory information. Extensive research has been conducted both domestically and internationally on target detection and tracking based on surveillance video streams, as well as vehicle detection and tracking in traffic scenarios. Summary of the Invention

[0003] Purpose of the invention: To overcome the shortcomings of existing technologies, this invention provides a vehicle tracking method for highway scenarios based on a time-space fusion network. An STFN-ADM-MTHS network model for vehicle tracking in highway scenarios is constructed. Through model training, parameter optimization, and model comparison, the method can effectively track vehicles that are occluded by vibration in highway scenarios, solve the problem of missed vehicle detection during the tracking process, improve vehicle tracking accuracy, and provide a foundation for the subsequent realization of multi-target vehicle tracking in highway scenarios.

[0004] Technical Solution: To achieve the above objectives, this invention provides a vehicle tracking method for highway scenarios based on a time-space fusion network, comprising the following steps:

[0005] S1: Construct a vehicle tracking dataset for the highway scenario at Southeast University, including factors such as camera vibration, lighting changes, and weather changes (rain, strong winds).

[0006] S2: Based on the characteristics of the vehicle tracking dataset, a multi-vehicle tracking network model for highway scenarios (STFN-ADM-MTHS) based on spatiotemporal fusion network adaptive decay memory is constructed for vehicle tracking in highway scenarios.

[0007] S3: Train the STFN-ADM-MTHS network model and optimize its parameters, then compare it with other tracking models for vehicle tracking in highway scenarios.

[0008] Furthermore, the specific steps of step S1 are as follows:

[0009] S1-1: Construct a Multi-Vehicle Tracking in Highway Scenes Datasets of Southeast University (MTHS-SEU), incorporating factors such as camera vibration, lighting changes, and weather changes (rain, strong winds), etc. Figure 5 As shown, the dataset consists of 21,558 images across 2 scene segments and 15 time periods, divided into a training set and a test set in a 2:1 ratio.

[0010] Furthermore, the specific steps of step S2 are as follows:

[0011] S2-1: For a joint decoding structure of detection and tracking in a specific scenario (highway), a new-born target detection query with self-attention from the previous frame is constructed. By narrowing the scope of the detection query, the detection and tracking accuracy is improved. Conflicts exist between detection and tracking in the joint query, leading to poor detection performance. The detection query lacks semantic features compared to the tracking query. Specifically, during training, the tracking query is initially empty, so there is no semantic alignment issue. The tracking query updated in the next frame contains semantic information, but the detection query is not updated at this time, resulting in a semantic alignment problem. This is addressed by associating the detection query of the current frame... Detection output compared to the previous frame By using a self-attention mechanism to focus detection attention on a specific region, the scope of the detection query is reduced, and sufficient semantic information is obtained while retaining the relevance of the tracking query;

[0012] S2-2: To address the occlusion problem, a repulsion loss was added to the tracking model, improving detection and tracking accuracy. The repulsion loss, defined as the repulsion loss, consists of two parts:

[0013]

[0014] Among them, L RepGT The ground-truth exclusion loss term indicates that the predicted target box is far from other ground-truth boxes; L RepBox The prediction rejection term represents the distance the target predicted box moves away from the predicted boxes corresponding to other ground truth boxes (Ground-Truth). The weights α and β are used to balance the loss function.

[0015] There is a repulsion loss between the predicted bounding boxes of different ground truths (GTs). Analogous to the repulsive force between magnets, the repulsive effect becomes less pronounced with increasing distance. Therefore, the calculation process of the repulsion loss is simplified by defining a threshold δ, where the Interchange of Union (IOU) between the ground and truth is greater than δ. Here, δ is set to 0.3. The L... RepGT and L RepBox The loss calculation needs to consider the overlapping area, center point distance, and aspect ratio. CIOU is used to calculate the loss term.

[0016]

[0017] Where, ρ 2 R represents the Euclidean distance. p and G p Let c represent the center point of the predicted bounding box and the ground truth (GT), respectively, and let w represent the minimum diagonal length containing both the predicted bounding box and the GT. G and h G These represent the width and height of GT, respectively. R and h R These represent the width and height of the predicted bounding box, respectively. v is used to quantize the aspect ratio, which is equal to... The weight γ is used to balance v.

[0018] For the j-th ground truth frame in the i-th frame Calculate the ground-truth set of the Repulsion loss target. Prediction box set The formula for calculating Repulsion loss is as follows:

[0019]

[0020] S2-3: Construct a Spatial-Guided Cross-Temporal Feature Aggregation module. Spatially, since vehicle trajectories change linearly, the most direct method to distinguish between random motion caused by camera shake and regular changes caused by the object's own motion is to calculate the difference between inter-frame trajectories and find the correlation between trajectory features and the difference. Temporally, the tracking process itself has a temporal correlation. Since occlusion is continuous, the most direct method for occluded targets is to enhance the current frame using the vehicle position from the previous frame. Specifically, this involves using the inter-frame trajectory difference to learn the random motion change patterns caused by vibration, considering the occlusion situation of vehicles in different trajectories at the same time. To address the vibration problem, a Transformer attention module is constructed to generate Q and K based on the trajectory feature differences between adjacent frames, and the trajectory features of the current frame. Generate V, attention module f shakeBy capturing irregular motion changes caused by camera vibration and ignoring these changes through an attention mechanism, the true motion trajectory of the vehicle can be identified and analyzed more accurately.

[0021] Regarding the occlusion problem, for f shake Output By comparing the trajectory output with the previous frame Add them together to aggregate the trajectory features of adjacent frames. Enhance the trajectory representation of the current frame to prevent the loss of trajectory information, and simultaneously remove frames that have not passed f. shake Output the trajectory difference between the i-th and i-1-th frames Track output from the previous frame Adding them together yields the trajectory features of aggregated adjacent frames. Because the features of the aggregation are directly applied and Concatenation or averaging would impair the original representational power; therefore, this application constructs a gatedfusion module to adaptively fuse and aggregate features, M,M ′ M ″ MLP is represented by the following expression:

[0022]

[0023] S2-4: Considering that the occlusion situation of vehicles on different trajectories at the same time in the tracking sequence is different, the decay rate of long-term memory will also differ. Therefore, an Adaptive Decay Memory is constructed to adaptively update the decay rate of long-term memory. In a stable state, i.e., when the tracked vehicle presents continuous and stable features in the video, the decay rate of long-term memory should be increased accordingly. At this point, the current frame information can accurately reflect the current state of the vehicle, so assigning more weight to the observation of the current frame helps improve the stability and accuracy of tracking. In an unstable state, such as in occlusion situations, when some or all of the tracked vehicle's information is temporarily invisible, the decay rate of long-term memory should be slowed down accordingly. Since the observation information of the current frame may be incomplete or inaccurate during occlusion, relying more on previous long-term memory helps maintain the continuity of tracking. Specifically: Initially, the long-term memory is initialized with the output of the current frame. By establishing a connection between the trajectory output features of the current frame and the long-term memory, more accurate and stable trajectory features are obtained. The i-th frame... and V and Q are generated separately for long-term memory. Generate K, perform multi-head cross-attention, and output the query vector for the next frame. Different trajectory features correspond to different memory decay rates. This application constructs an Adaptive Decay Memory module, which takes the trajectory output features as input and processes them through MLP and Sigmoid to obtain the corresponding trajectory decay rate. The long-term memory update formula is as follows:

[0024]

[0025] in, This is the trajectory output after aggregating features in the i-th frame. This represents the decay rate of trajectory features in long-term memory. This indicates the degree to which the trajectory features in the i-th frame contribute to the long-term memory of the next frame.

[0026] S3-1: For the evaluation of the tracking model, the CLEAR evaluation metrics are used, including MOTA, MOTP, CLR_Re, CLR_Pr, FP, FN, IDF1, and HOTA.

[0027] S3-2: Analyze the effectiveness of the improved strategy in the STFN-ADM-MTHS model, compare the metrics of MeMOTR and MOTR models, and comprehensively compare and select the vehicle tracking method for highway scenarios.

[0028] The beneficial effects of this invention are as follows: by combining tracking algorithms and improvement strategies for corresponding scenario problems, an STFN-ADM-MTHS network model for vehicle tracking in highway scenarios is constructed, which can effectively track vehicles occluded in highway vibration scenarios, solve the problem of missed vehicle detection in the tracking process, improve vehicle tracking accuracy, and provide a foundation for the subsequent realization of multi-target vehicle tracking in highway scenarios. Attached Figure Description

[0029] Figure 1 Visualize different sections of the highway dataset, where (a) good lighting conditions and vibration, (b) poor lighting conditions and vibration, (c) poor lighting conditions, (d) good lighting conditions, (e) rainy day, and (f) sunny day.

[0030] Figure 2 The model structure is STFN-ADM-MTHS.

[0031] Figure 3 This is a schematic diagram of the Spatial-Guided Cross-Temporal Feature Aggregation Module.

[0032] Figure 4 This is a schematic diagram of the Adaptive Decay Memory.

[0033] Figure 5 The image shows the detection performance of STFN-ADM-MTHS on the test set. (a) is frames 1-1000, (b) is frames 1-1050, (c) is frames 3-1398, (d) is frames 4-256, (e) is frames 5-998, and (f) is frames 5-1023. Detailed Implementation

[0034] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0035] This invention provides a vehicle tracking method for highway scenarios based on a temporal-spatial fusion network, specifically including the following steps:

[0036] S1: Using video data from highway scene vehicle monitoring, a Multi-Vehicle Tracking in Highway Scenes Datasets of Southeast University (MTHS-SEU) was constructed, incorporating factors such as camera vibration, lighting changes, and weather changes (rain, strong winds), etc. Figure 5 As shown. The dataset consists of 21,558 images across 2 scene segments and 15 time periods, divided into a training set and a test set in a 2:1 ratio. A partial visualization of the highway dataset is shown below. Figure 1 As shown.

[0037] S2: Based on the characteristics of the dataset, construct an STFN-ADM-MTHS network model for vehicle tracking in highway scenarios, with the following structure: Figure 2 As shown.

[0038] S2-1: For a joint decoding structure of detection and tracking in a specific scenario (highway), a Lastframe new-born with Detect query self-attention was constructed, such as... Figure 2 As shown, by narrowing the detection query range, the detection and tracking accuracy is improved, and by associating the detection query with the current frame... Detection output compared to the previous frame By using a self-attention mechanism to focus detection attention on a specific region, the scope of the detection query is reduced, and sufficient semantic information is obtained while retaining the relevance of the tracking query;

[0039] S2-2: To address the occlusion problem, a Repulsion Loss was added to the tracking model, improving detection and tracking accuracy. The Repulsion Loss, defined to address occlusion issues in the scene, consists of two parts:

[0040]

[0041] Among them, L RepGT The Ground-Truth exclusion loss term indicates that the predicted target box is far from other Ground-Truth elements; L RepBox The prediction loss term indicates that the target predicted box moves away from other predicted boxes corresponding to the Ground-Truth. The weights α and β are used to balance the loss function.

[0042] There is a repulsion loss between the predicted bounding boxes of different ground truths (GTs). Analogous to the repulsive force between magnets, the repulsive effect becomes less pronounced with increasing distance. Therefore, the calculation process of the repulsion loss is simplified by defining a threshold δ, where the Interchange of Union (IOU) between the ground and truth is greater than δ. Here, δ is set to 0.3. The L... RepGT and L RepBox The loss calculation needs to consider the overlapping area, center point distance, and aspect ratio. CIOU is used to calculate the loss term.

[0043]

[0044] Where, ρ 2 R represents the Euclidean distance. p and G p Let c represent the center point of the predicted bounding box and the ground truth (GT), respectively, and let w represent the minimum diagonal length containing both the predicted bounding box and the GT. G and h G These represent the width and height of GT, respectively. R and h R These represent the width and height of the predicted bounding box, respectively. v is used to quantize the aspect ratio, which is equal to... The weight γ is used to balance v.

[0045] For the j-th ground truth frame in the i-th frame Calculate the ground-truth set of the Repulsion loss target. Prediction box set The formula for calculating Repulsion loss is as follows:

[0046]

[0047] S2-3: Construct Spatial-Guided Cross-Temporal Feature Aggregation, such as Figure 3As shown, the study utilizes inter-frame trajectory differences to learn the patterns of random motion changes caused by vibration, considering the occlusion of vehicles on different trajectories at the same time. To address the vibration issue, a Transformer attention module is constructed to generate Q and K based on the trajectory feature differences between adjacent frames, and the trajectory features of the current frame... Generate V, attention module f shake By capturing irregular motion changes caused by camera vibration and ignoring these changes through an attention mechanism, the true motion trajectory of the vehicle can be identified and analyzed more accurately.

[0048] Regarding the occlusion problem, for f shake Output By comparing the trajectory output with the previous frame Add them together to aggregate the trajectory features of adjacent frames. Enhance the trajectory representation of the current frame to prevent the loss of trajectory information, and simultaneously remove frames that have not passed f. shake Output the trajectory difference between the i-th and i-1-th frames Track output from the previous frame Adding them together yields the trajectory features of aggregated adjacent frames. Because the features of the aggregation are directly applied and Concatenation or averaging would impair the original representational power; therefore, this application constructs a gatedfusion module to adaptively fuse and aggregate features, M,M ′ M ″ MLP is represented by the following expression:

[0049]

[0050] S2-4: Considering that the occlusion of vehicles on different trajectories at the same time in the tracking sequence is different, the decay rate of long-term memory will also differ accordingly. Therefore, we construct an Adaptive Decay Memory, such as... Figure 4 As shown, the decay rate of long-term memory is adaptively updated. Initially, the long-term memory is initialized with the output of the current frame. By establishing a connection between the trajectory output features of the current frame and the long-term memory, more accurate and stable trajectory features are obtained. The trajectory features of the i-th frame are... and V and Q are generated separately for long-term memory. Generate K, perform multi-head cross-attention, and output the query vector for the next frame. Different trajectory features correspond to different memory decay rates. This application constructs an Adaptive Decay Memory module, which takes the trajectory output features as input and processes them through MLP and Sigmoid to obtain the corresponding trajectory decay rate. The long-term memory update formula is as follows:

[0051]

[0052] in, This is the trajectory output after aggregating features in the i-th frame. This represents the decay rate of trajectory features in long-term memory. This indicates the degree to which the trajectory features in the i-th frame contribute to the long-term memory of the next frame.

[0053] S3: To verify the tracking performance, especially the association performance, state-of-the-art methods were compared on the MTHS-SEU dataset. As shown in Table 1, on the MTHS-SEU dataset, the STFN-ADM-MTHS method achieved 72.197 HOTA, 77.659 AssA, and 88.084 IDF1, respectively. The DetA and MOTA metrics were lower than those of Tracking by Detection methods, such as BOT-SORT (DetA 68.426) and C-BIOU (MOTA 77.146). DetA and MOTA metrics emphasize detection performance. The end-to-end joint decoding approach limits detection capabilities, performing worse than independent detection models (BOT-SORT) on the detection-biased metrics of DetA and MOTA. However, compared to joint decoding methods like MOTR and MeMOTR, STFN-ADM-MTHS achieves DetA 67.704 and MOTA 76.264, demonstrating the effectiveness of Last-frame new-born with Detect query self-attention and Repulsion in detection. During decoding, the detection component is indeed enhanced. AssA and IDF1 emphasize correlation performance. The STFN-ADM-MTHS method achieves better correlation results compared to state-of-the-art methods, proving the effectiveness of Spatial-Guided Cross-Temporal Feature Aggregation and Adaptive Decay Memory.

[0054] Table 1 Performance of different tracking models on the MTHS-SEU test set

[0055]

[0056] To more intuitively demonstrate the model tracking performance, a visual comparison of the MeMOTR and STFN-ADM-MTHS models is presented on a custom MTHS-SEU dataset, such as... Figure 5As shown, in each sub-figure, the left side represents the MeMOTR visualization result, and the right side represents the STFN-ADM-MTHS visualization result. Due to camera vibration, the position and extent of the bounding box on the image plane may change significantly, which may lead to ID switches. In (b) and (f), errors exist in the MeMOTR visualization, while STFN-ADM-MTHS can accurately identify incorrect vehicles, proving that Spatial-Guided Cross-Temporal Feature Aggregation is effective. In (a) and (c), MeMOTR has a brief missed detection of occluded targets, but STFN-ADM-MTHS can correctly detect vehicles, demonstrating that Adaptive Decay Memory can effectively combine real-world scene information with temporal information. In (d), (e), and (f), it was found that MeMOTR missed small targets, while STFN-ADM-MTHS could detect them correctly. In highway scenes, such small targets often correspond to newly emerging targets, which precisely verifies the improved Lastframe new-born with Detect query self-attention. STFN-ADM-MTHS focuses more on the region where the new target is generated during the decoding process. In (a), (b), (d), (e), and (f), compared with MeMOTR, the bounding boxes on the image plane at the occlusion location of STFN-ADM-MTHS are closer to the real vehicles, which also verifies the improved Repulsion operation.

[0057] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A vehicle tracking method for highway scenarios based on a temporal-spatial fusion network, characterized in that: Includes the following steps: S1: Construct a vehicle tracking dataset for highway scenarios, including factors such as monitoring camera vibration, lighting changes, and weather changes; S2: Based on the characteristics of the vehicle tracking dataset, we construct an STFN-ADM-MTHS multi-vehicle tracking network model for highway scenarios, which is based on spatiotemporal fusion network adaptive decay memory. Step S2 is as follows: S2-1: For the joint decoding structure of detection and tracking under highway conditions, a new-born target detection query with self-attention in the previous frame is constructed, which improves the detection and tracking accuracy by narrowing the detection query range; S2-2: To address the occlusion problem, a Repulsion Loss was added to the tracking model, improving detection and tracking accuracy. The Repulsion Loss, defined as a two-part loss, is incorporated into the model to address occlusion issues in the scene. (1); in, The Ground-Truth exclusion loss term represents the predicted target box being far from other ground-Truth boxes. The prediction rejection loss term represents the distance the target predicted box moves away from the predicted boxes corresponding to the ground truth boxes (Ground-Truth). The weights are... Used to balance the loss function; S2-3: Construct Spatial-Guided Cross-Temporal Feature Aggregation, utilize inter-frame trajectory differences to learn the random motion variation patterns caused by vibration, and consider the occlusion of vehicles on different trajectories at the same time. Regarding the issue of occlusion, for Output By comparing the trajectory output with the previous frame Add them together to aggregate the trajectory features of adjacent frames. To enhance the trajectory representation of the current frame and prevent the loss of trajectory information, while also removing those that have not passed... Output the trajectory difference between the i-th and i-1-th frames Track output from the previous frame Adding them together yields the trajectory features of aggregated adjacent frames. Because the features of the aggregation are directly applied... and Concatenating or averaging features can impair their original representational power; therefore, the gatedfusion module is constructed to adaptively fuse and aggregate features. MLP is represented by the following expression: (3); S2-4: Considering that the occlusion of vehicles on different trajectories at the same time in the tracking sequence is different, the decay rate of long-term memory will also differ. Therefore, an Adaptive Decay Memory is constructed to adaptively update the decay rate of long-term memory. Using trajectory output features as input, the decay rate is obtained through MLP and Sigmoid processing. The long-term memory update formula is as follows: (4); in, This is the trajectory output after aggregating features in the i-th frame; , representing the decay rate of trajectory features in long-term memory; , representing the degree of contribution of the trajectory features in the i-th frame to the long-term memory of the next frame; S3: Train the STFN-ADM-MTHS network model and optimize its parameters, then compare it with other tracking models for vehicle tracking in highway scenarios.

2. The vehicle tracking method in a highway scenario based on a temporal-spatial fusion network according to claim 1, characterized in that: The specific steps of S1 are as follows: S1-1: Construct a vehicle tracking dataset for highway scenarios, including monitoring camera vibration, lighting changes, and weather changes. The dataset consists of 21,558 images across 2 scene segments and 15 time periods, divided into a training set and a test set with a ratio of 2:

1.

3. The vehicle tracking method in a highway scenario based on a temporal-spatial fusion network according to claim 1, characterized in that: Step S2-2 also includes: There is a repulsion loss between the predicted bounding boxes of different ground truths. The repulsion effect becomes less significant with increasing distance. Therefore, the calculation process of the repulsion loss is simplified by defining a threshold. The IOU between Ground and Truth is greater than 1. , Take 0.3 and calculate. and The loss calculation needs to consider the overlapping area, center point distance, and aspect ratio. CIOU is used to calculate the loss term. (2); in, Represents Euclidean distance. and Let represent the center point of the predicted bounding box and the ground truth (GT), respectively, and let c represent the minimum diagonal length containing both the predicted bounding box and the GT. and These represent the width and height of GT, respectively. and These represent the width and height of the predicted bounding box, respectively. Used to quantize aspect ratio, equal to Weight Used for balance .

4. The vehicle tracking method in a highway scenario based on a temporal-spatial fusion network according to claim 3, characterized in that: In step S2-1, the detection query associated with the current frame is performed. Detection output compared to the previous frame By using a self-attention mechanism to focus detection attention on a specific region, the scope of the detection query is reduced, and sufficient semantic information is obtained while retaining the ability to track the relevance of the query.

5. The vehicle tracking method in a highway scenario based on a temporal-spatial fusion network according to claim 3, characterized in that: In step S2-2, to address the occlusion problem in the scene, Repulsion Loss is added to the model. The defined Repulsion Loss consists of two parts. and ,in, The loss term for ground truth box exclusion indicates that the predicted target box is far from other ground truth boxes; The rejection loss term is used to predict the distance between the target predicted box and the predicted boxes corresponding to other ground truth boxes.

6. The vehicle tracking method in a highway scenario based on a temporal-spatial fusion network according to claim 3, characterized in that: In steps S2-3, a Spatial-Guided Cross-Temporal FeatureAggregation module is constructed. Spatially, it calculates the difference between trajectories between frames and finds the correlation between trajectory features and the difference. Temporally, it uses the vehicle position of the previous frame to enhance the current frame.

7. The vehicle tracking method in a highway scenario based on a temporal-spatial fusion network according to claim 3, characterized in that: In steps S2-4, specifically for the Adaptive Decay Memory module, in a stable state, i.e. when the tracked vehicle exhibits continuous and stable features in the video, the decay rate of long-term memory increases accordingly, and the current frame information accurately reflects the current state of the vehicle. Therefore, more weight is given to the observation of the current frame to improve the stability and accuracy of tracking. In an unstable state, when some or all of the information of the tracked vehicle is temporarily unavailable, the decay rate of long-term memory slows down accordingly, and the observation information of the current frame is incomplete or inaccurate. In this case, more reliance is placed on previous long-term memory to maintain the continuity of tracking.

8. The vehicle tracking method in a highway scenario based on a temporal-spatial fusion network according to claim 1, characterized in that: Step S3 is as follows: S3-1: For the evaluation of the tracking model, the CLEAR evaluation metrics are used, including MOTA, MOTP, CLR_Re, CLR_Pr, FP, FN, IDF1, and HOTA; S3-2: Analyze the effectiveness of the improved strategy in the STFN-ADM-MTHS model, compare the metrics of MeMOTR and MOTR models, and comprehensively compare and select the vehicle tracking method for highway scenarios.

Citation Information

Patent Citations

  • Training method and device based on cross-camera multi-target tracking model

    CN115393384A

  • Method for tracking vehicles in highway tunnel scene based on convolutional fusion network

    CN118587247A