Detection enhancement vehicle multi-target tracking method based on feature repair and relationship optimization

Abstract

This invention discloses a detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization. Video frames are input into a backbone network to extract shared features, constructing degraded observation features and reference features. The reference features are then subjected to mask-guided forward diffusion, combined with degraded observation features for reverse denoising to obtain repaired features. After uncertainty evaluation, these repaired features are adaptively fused with the degraded observation features to obtain fused features. The fused features are input into a detection decoder to generate a detection embedding. This embedding is then combined with historical tracking embeddings to model local competition relationships, resulting in an optimized detection embedding. The optimized detection embedding and historical tracking embeddings are input into a joint decoder, and the trajectory is updated by a trajectory management module. The tracking result is output, and the tracking embedding for the next time step is generated. The model is trained using low-light and partially occluded datasets, and the results are visualized. This invention's method exhibits strong detection discriminative power and tracking stability in multi-target vehicle tracking under low-light and partially occluded environments.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent video analysis technology, specifically relating to a detection-enhanced multi-target tracking method for vehicles based on feature repair and relationship optimization. Background Technology

[0002] With the rapid development of the automotive industry and the continuous growth of motor vehicle ownership, China's annual automobile production and sales both exceeded 34 million vehicles by 2025. While the increased vehicle fleet has improved public travel efficiency and promoted regional coordinated development, it has also exacerbated traffic congestion and led to numerous safety accidents, making road safety a pressing public issue. Against this backdrop, the country is vigorously promoting the construction of Intelligent Transportation Systems (ITS). ITS relies on technologies such as video processing and artificial intelligence to achieve real-time traffic condition monitoring, dynamic traffic flow analysis, and rapid early warning of traffic incidents. Vehicle target tracking is a core foundational component of ITS.

[0003] Vehicle multi-object tracking (Vehicle MOT) is a key technology in ITS (Information Technology System). Its core task is to detect multiple vehicle targets in continuous video, combine historical trajectories to complete cross-frame identity association, and update the trajectory, providing data support for upper-layer applications such as traffic flow statistics and abnormal behavior analysis. Compared with single-object tracking, vehicle multi-object tracking needs to handle multiple vehicle detection, identity differentiation, and re-association after occlusion simultaneously, making it more in line with the needs of real traffic scenarios. From a technological development perspective, multi-object tracking has evolved from traditional motion modeling and data association methods to detection-after-association modes based on deep learning, and then to end-to-end modes incorporating Transformers. Existing methods perform well in general scenarios, but there is still room for improvement in low-light and partially occluded scenarios.

[0004] Low-light environments weaken vehicle edges, textures, and local structural features in vehicle images while amplifying noise, leading to degraded detection features and impacting tracking performance. While existing diffusion model-related methods are suitable for handling noise-contaminated degraded features, they suffer from high computational costs. Local occlusion results in incomplete extraction of local vehicle textures, contours, and structures, easily causing interference from nearest-neighbor tracking embeddings and leading to vehicle ID swapping. Existing methods mostly mitigate these problems through occlusion recovery or subsequent association, but this delayed processing can amplify errors. Therefore, addressing these technical pain points necessitates optimizing relevant technical modules to improve the stability and reliability of multi-target vehicle tracking, providing support for efficient ITS operation. Summary of the Invention

[0005] The purpose of this invention is to provide a detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization, which has the characteristics of strong detection discrimination and tracking stability in low-light and partially occluded environments.

[0006] The technical solution adopted in this invention is a detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization, comprising:

[0007] Step 1: Input video frames into the backbone network to extract shared features, and construct degradation observation features and reference features; Step 2: Perform mask-guided forward diffusion on the reference features, combine with degraded observation features for reverse denoising to obtain repaired features, and adaptively fuse with degraded observation features after uncertainty assessment to obtain fused features; Step 3: Input the fused feature into the detection decoder to generate the detection embedding, combine it with the historical tracking embedding to model local competition relationships, and enhance the obtained optimized detection embedding; Step 4: Input the optimized detection embedding and the historical tracking embedding into the joint decoder, update the trajectory through the trajectory management module, output the tracking result and generate the tracking embedding for the next moment; Step 5: Train and evaluate the model using a low-light and partial occlusion dataset and visualize the results.

[0008] The invention is further characterized by: Step 1 specifically includes: Step 1.1: Place the first t Frame video frames Inputting ResNet-50 extracts local convolutional features, then using TransformerEncoder to model the global context, yielding shared features, denoted as... ,in and Spatial dimensions representing features Indicates the number of channels; Step 1.2: Shared features The process is divided into two routes to construct degraded observation features and reference features respectively: Shared features Enter one Convolution performs channel mapping and dimension alignment to obtain degraded observation features. ; At the same time, shared features Input a channel adaptive fusion module, which includes, in sequence: a Convolution, one Convolution, a function with an inflation rate of 2 Dilated convolution, one SE channel attention module, one Convolution, global average pooling, and sigmoid activation; finally, the channel machine fusion weights output by the sigmoid activation are broadcast in the spatial dimension and shared with the features. Perform element-wise multiplication to obtain the reference feature. .

[0009] Step 2 specifically includes: Step 2.1: Perform token-level mask-guided forward diffusion on the reference features, and introduce a degenerate adaptive noise scheduling mechanism to optimize the noise injection process of forward diffusion; Step 2.2: Perform brightness correction processing on the degraded observation features to eliminate local brightness interference and obtain the debiased conditional features; Step 2.3: Using the debiased conditional features as constraints, perform a reverse denoising operation on the noisy reference features after forward diffusion to obtain the repaired features; Step 2.4: Evaluate the trajectory fluctuation uncertainty of the repair features and generate fusion weights for each position and each channel; Step 2.5: Based on the fusion weight, the repair features and the degradation observation features are fused element by element to obtain the fused features.

[0010] Step 2.1 specifically includes: Step 2.1.1: For the reference feature The absolute mean of the channel dimensions is used to obtain the response score for each token. After min-max normalization and fixed threshold binarization, a token-level reliability mask is obtained. ,in, , representing the number of tokens; Step 2.1.2: Broadcast the token across the channel dimension to expand it to , thus obtaining the expanded mask; Step 2.1.3: Calculate the proportion of unreliable tokens in the token-level reliable mask and use it as a scalar for degradation strength, expressed as:

[0011] In the formula, This represents the proportion of unreliable tokens in the current frame, which is also a scalar value indicating the degradation intensity. ; Step 2.1.4: Use a standard linear noise schedule Generate lightly and heavily noisy scheduling sequences, specifically: given the number of diffusion steps. First in the interval Press up by number of steps Generate uniform linear interpolation ,make ;structure

[0012] And obtain the corresponding:

[0013] In the formula, This is a slightly noisy scheduling sequence. This is a heavily noisy scheduling sequence; Step 2.1.5: Utilize the degradation intensity scalar Linear interpolation is performed on the two sets of scheduling sequences to obtain the degenerate adaptive noise scheduling coefficients, expressed as:

[0014] In the formula, This represents the noise scheduling coefficient at step k in the recursive formula of the regular Markov chain after degenerate adaptive scheduling. Represents the degenerate adaptive noise scheduling coefficients; Step 2.1.6: Combining the expanded mask and the degenerate adaptive noise scheduling coefficients, perform mask-guided forward diffusion on the reference features to obtain noisy reference features, as shown below:

[0015] In the formula, This represents element-wise multiplication. As the starting point of diffusion, For the first Noisy reference features after step diffusion It is forward-injected Gaussian noise. For the first Step noise scheduling coefficient; This represents a standard Gaussian distribution with a mean of 0 to a variance of the identity matrix. Represents the identity matrix.

[0016] Step 2.2 specifically involves: Extracted degradation observation features Through a step size of 1 Average pooling (AvgPool) is used to obtain the brightness response features. ; Then reduce the degradation of observation features by using residuals. Local brightness interference in the image is used to obtain the conditional features after polarization removal. :

[0017] In the formula, Indicates global average pooling. This is a channel-level global statistical vector.

[0018] Step 2.3 specifically includes: Step 2.3.1: Place the first Noisy reference features after step diffusion Conditional characteristics after debiasing Common input DiT denoising process, predicting forward injection noise , represented as:

[0019] In the formula, This represents the noise injection process predicted by the DiT network; Step 2.3.2: Reconstruct the mean of the inverse denoising based on the predicted forward injected noise, expressed as:

[0020] In the formula, This represents the mean reconstructing process of reverse denoising based on predicted forward-injected noise; Step 2.3.3: Construct a Gaussian inverse transition distribution using a fixed diagonal covariance, expressed as:

[0021] In the formula, Indicates a Gaussian inverse transition distribution. Indicates the first scalar variance of the backsampling step. It is the identity matrix; Step 2.3.4: Perform backsampling iterations step by step along the expansion step number, where the back transition distribution of each step is in Gaussian form, expressed as:

[0022] Stepwise sampling to Finally, the repair features are obtained. .

[0023] Step 2.4 specifically involves: Step 2.4.1: Select the reverse denoising end L Each denoising step involves the restoration of features, and the restoration features at each step... Flattened into a sequence along spatial dimensions ; Step 2.4.2: Calculate the position-level trajectory fluctuation uncertainty using variance statistics and channel averaging, expressed as:

[0024] In the formula, Indicates the first spatial location Uncertainty response, This represents variance statistics, where D represents the number of channels; After calculation for all spatial locations, the following results were obtained. Reshaping back into the spatial domain yields uncertainty. ; Step 2.4.3: Addressing Uncertainty Considered as a single-channel feature map After a Convolution performs channel mapping, and then the channel is further mapped using the Sigmoid activation function. Intervals are used to obtain position-by-position and channel-by-channel fusion weights. .

[0025] In step 2.5, based on the fusion weights, the following steps are performed: and Perform element-wise weighted fusion to obtain fusion features. , represented as:

[0026] In the formula, .

[0027] Step 3 specifically includes: Step 3.1: Merge features Input the detection decoder to generate the detection embedding set of the current video frame. ,in Indicates the number of embedded elements detected. ; Meanwhile, the current tracking embedding set generated by the Timing Interaction Module (TIM) from the previous video frame is denoted as... , This indicates the number of trajectories that are still being preserved. ; Step 3.2: Perform linear projection on each detection embedding and tracking embedding using a linear layer, as shown below:

[0028] In the formula, , This represents the linear feature representation of the detection embedding and the tracking embedding. and It is a learnable mapping matrix; Step 3.3: For each detection embedding Construct its connection with the first j The competition relationship representation vector of the tracking embedding is denoted as:

[0029] In the formula, This represents element-wise multiplication. Indicates a splicing operation; Step 3.4: Using MLP, represent the competition relationship as a vector. Mapped to relation token ; Step 3.5: Use a shared linear hierarchical mapping to directly map relation tokens to obtain the competition score. , represented as:

[0030] In the formula, , For learnable parameters, for all Relationship sharing; Step 3.6: Embedding the fixed detection Select the top-scoring embeddings with the highest competitive scores from all tracking embeddings. These constitute a set of local competitive neighborhood indexes:

[0031] In the formula, Indicates the relationship with the first The detection embedding has the strongest competitive relationship. A set of tracking embedded indexes; This represents the TopK statistical algorithm; Step 3.7: Place the first Detection embedding Mapped to query vector , represented as:

[0032] In the formula, It is a learnable mapping matrix; Step 3.8: For the set of locally competing neighborhood indexes Each relationship token within Generate key vectors respectively Sum value vector , represented as:

[0033] In the formula, , It is a learnable mapping matrix; Step 3.9: Place the first Each detection embedding contains all key vectors and value vectors within its corresponding neighborhood, stacked together as follows:

[0034] Let query matrix for:

[0035] Step 3.10: Calculate the first... Each detection embedding is a normalized response weight for each relation token in its local competitive neighborhood. ; Step 3.11: Define the first Each detection embeds a corresponding competitive context. , represented as:

[0036] Step 3.12: Embed the original detection With the context of competition By splicing them together, we can obtain Then it is mapped back through a linear layer. Dimension, obtained Finally, the competition-enhanced detection embedding is generated in the form of residuals:

[0037] In the formula, This indicates the enhanced detection embedding. This represents the linear normalization operation; Perform steps 3.1 to 3.12 on all detection embeddings to obtain the optimized detection embedding set. , represented as: .

[0038] Step 4 specifically includes: Step 4.1: Embed the tracking data into the set and the optimized detection embedding set Input to the joint decoder, output the current embedding of all preserved trajectories. ; Step 4.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require The confidence level is obtained from the prediction head of the input baseline. With bounding box ; Step 4.3: The baseline trajectory management module receives the embedding of all currently retained trajectories. Confidence level With bounding box While performing new target initialization, activity trajectory update and short-term lost trajectory maintenance, and outputting tracking results, the trajectory status is sent to the baseline time-series interaction module TIM to generate the tracking embedding for the next moment.

[0039] The beneficial effects of this invention are: This invention presents a detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization. Building upon the baseline model MEMOTR, it constructs a feature diffusion repair module and a local competition relationship adaptation module. Tracking optimization is achieved through feature repair, adaptive fusion, and local competition relationship enhancement. Compared to existing technologies, feature repair and adaptive fusion perform mask-guided forward diffusion and reverse denoising on reference features, completing feature repair and adaptive fusion, effectively improving detection bias and missed detection issues under low-light conditions. Local competition relationship enhancement models the local competition relationships between detection embedding and historical tracking embedding and enhances features, improving anti-interference capabilities in occluded scenes, reducing ID swapping and trajectory misalignment, and significantly improving the accuracy and stability of multi-target vehicle tracking in complex traffic scenarios. This invention effectively improves the accuracy and trajectory stability of multi-target vehicle tracking and is more suitable for applications in complex traffic scenarios. Attached Figure Description

[0040] Figure 1 This is the overall network architecture diagram of the detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization of the present invention; Figure 2 This is a network structure diagram of the feature diffusion repair module proposed in the detection-enhanced multi-target tracking method for vehicles based on feature repair and relationship optimization in this invention; Figure 3 This is a network structure diagram of the local competitive relationship adaptation module proposed in the detection-enhanced multi-target tracking method for vehicles based on feature repair and relationship optimization in this invention; Figure 4 This is a visualization comparison of the tracking results of the detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization of this invention with other methods on the FastTracker dataset; Figure 5 This is a visualization comparison of the tracking results of the detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization of this invention with other methods on the CityFlow dataset; Figure 6 This is a visualization comparison of the tracking results of the detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization of this invention with other methods on the TrafficMOT dataset. Detailed Implementation

[0041] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0042] Example 1 This embodiment provides a detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization. The overall process is as follows: Figure 1 As shown, the specific steps include the following: Step 1: Input video frames into the backbone network to extract shared features, and construct degradation observation features and reference features; Step 2: Perform mask-guided forward diffusion on the reference features, combine with degraded observation features for reverse denoising to obtain repaired features, and adaptively fuse with degraded observation features after uncertainty assessment to obtain fused features; Step 3: Input the fused feature into the detection decoder to generate the detection embedding, combine it with the historical tracking embedding to model local competition relationships, and enhance the obtained optimized detection embedding; Step 4: Input the optimized detection embedding and the historical tracking embedding into the joint decoder, update the trajectory through the trajectory management module, output the tracking result and generate the tracking embedding for the next moment; Step 5: Train and evaluate the model using a low-light and partial occlusion dataset and visualize the results.

[0043] Example 2 Step 1 specifically includes: Step 1.1: Place the first t Frame video frames Inputting ResNet-50 extracts local convolutional features, then using TransformerEncoder to model the global context, yielding shared features, denoted as... ,in and Spatial dimensions representing features Indicates the number of channels; Step 1.2: Shared features The degraded observation features and reference features are constructed separately in two ways, such as Figure 2 As shown: Shared features The input degraded observation feature branch generates degraded observation features that preserve the attributes of the current degraded observations. Specifically, shared features will be used. enter Convolution performs channel mapping and dimension alignment to generate degraded observation features. This branch does not introduce additional enhancement or repair operations. Its main purpose is to preserve the original observational interference caused by imaging degradation as much as possible while maintaining the semantic and structural information of the features, thereby preserving the degraded observational features. It can accurately reflect the degradation state of the input to be repaired.

[0044] Simultaneously, the shared features are input into the reference feature branch to generate a cleaner sample than the degraded observation features, which serves as the reference feature. Although the reference feature is more stable than the original degraded observation, it loses some local details due to the smoothing and fusion process. Therefore, this invention does not directly use it as the input to the final detection branch, but instead uses the reference feature... The clean sample used in the diffusion repair process will serve as the starting point for diffusion. Specifically, shared features will be used. Input a channel adaptive fusion module, the channel adaptive fusion module is as follows Figure 2 As shown, it includes, in sequence: one Convolution, one Convolution, a function with an inflation rate of 2 Dilated convolution, one SE channel attention module, one The algorithm employs convolution, global average pooling, and sigmoid activation. The first two convolutional layers are used for local detail extraction. Dilated convolution expands the receptive field, invents neighborhood context compensation, and enhances the modeling ability of information surrounding degraded regions. The SE module strengthens channels with stronger semantic association with the vehicle target and suppresses channels heavily affected by illumination degradation. Sigmoid activation generates channel-level fusion weight vectors. Finally, the channel-level fusion weights output by sigmoid activation are broadcast spatially and shared with the features. Perform element-wise multiplication to obtain the reference feature. .

[0045] Example 3 Step 2 specifically includes: Step 2.1: Perform token-level mask-guided forward diffusion on the reference features, and introduce a degenerate adaptive noise scheduling mechanism to optimize the noise injection process of forward diffusion.

[0046] In low-light conditions, degradation often exhibits localized clustering. For example, uneven distribution of streetlights or building shadows can cause some areas to experience more detail loss and noise amplification than others. Indiscriminately adding noise to the entire feature map may introduce additional interference to already reliable areas, weakening the targeted repair of truly degraded regions during the diffusion process. Therefore, this invention, based on forward noise addition, introduces token-level mask guidance constraints to provide coarse-grained guidance for the forward diffusion process, making noise injection more focused on suspected degradation areas. Specifically, combined with... Figure 2 Specifically, it includes the following sub-steps: Step 2.1.1: For the reference feature The absolute mean of the channel dimensions is used to obtain the response score for each token. After min-max normalization and fixed threshold binarization (the fixed threshold is set to 0.5 by default in this embodiment), a token-level reliability mask is obtained. ,in, , representing the number of tokens; Step 2.1.2: Broadcast the token across the channel dimension to expand it to This yields the expanded mask.

[0047] Considering that the degradation intensity in low-light scenes fluctuates significantly over time, but its noise scheduling coefficient is usually preset and fixed, consistently using a fixed noise scheduling coefficient for noise addition may result in excessive noise addition for mild degradation and insufficient noise addition for severe degradation. Therefore, this invention introduces a degradation adaptive noise scheduling mechanism to optimize the noise injection process in forward diffusion. Specifically: Step 2.1.3: Calculate the proportion of unreliable tokens in the token-level reliable mask and use it as a scalar for degradation strength, expressed as:

[0048] In the formula, This represents the proportion of unreliable tokens in the current frame, which is also a scalar value indicating the degradation intensity. When the proportion of unreliable tokens is higher, The larger the value, the more severe the degradation.

[0049] Step 2.1.4: Use a standard linear noise schedule Generate lightly and heavily noisy scheduling sequences, specifically: given the number of diffusion steps. First in the interval Press up by number of steps Generate uniform linear interpolation ,make ;structure

[0050] And obtain the corresponding:

[0051] In the formula, This is a slightly noisy scheduling sequence. This is a heavily noisy scheduling sequence; in this embodiment, , , .

[0052] Step 2.1.5: Utilize the degradation intensity scalar Linear interpolation is performed on the two sets of scheduling sequences to obtain the degenerate adaptive noise scheduling coefficients, expressed as:

[0053] In the formula, This represents the noise scheduling coefficient at step k in the recursive formula of the regular Markov chain after degenerate adaptive scheduling. Represents the degenerate adaptive noise scheduling coefficients; Step 2.1.6: Combining the expanded mask and the degenerate adaptive noise scheduling coefficients, perform mask-guided forward diffusion on the reference features to obtain noisy reference features, as shown below:

[0054] In the formula, This represents element-wise multiplication. As the starting point of diffusion, For the first Noisy reference features after step diffusion It is forward-injected Gaussian noise. For the first Step noise scheduling coefficient; This represents a standard Gaussian distribution with a mean of 0 to a variance of the identity matrix. Represents the identity matrix.

[0055] Step 2.2: Perform brightness correction processing on the degraded observation features to eliminate local brightness interference and obtain the debiased conditional features.

[0056] In the conditional diffusion framework, the reverse denoising process relies not only on the noisy features of the current diffusion step but also on the introduction of degraded observation features as conditional constraints. Degraded observation features The approximate vehicle location, target outline, and semantic information of the current scene are still preserved. Therefore, using degraded observation features as a reference can constrain the restoration results, ensuring that the restored results are consistent with the real vehicle structure in the current scene. However, due to the degraded observation features... The scene itself is affected by low illumination and non-uniform lighting, with significant luminance contamination. If these conditional features are not addressed, the denoising network may be interfered with by invalid conditions during the gradual reconstruction process, thus affecting the final reconstruction quality. Therefore, it is necessary to process the degraded observation features. Perform brightness correction processing, specifically: Extracted degradation observation features Through a step size of 1 Average pooling (AvgPool) is used to obtain the brightness response features. ; Then reduce the degradation of observation features by using residuals. Local brightness interference in the image is used to obtain the conditional features after polarization removal. :

[0057] In the formula, Indicates global average pooling. This is a channel-level global statistical vector. It is related to... and Element-wise operations are performed to copy and broadcast the vector back to the original feature size in the spatial dimension. This residual correction suppresses local brightness shifts while preserving global brightness information, thereby improving the structural usability of the conditional features.

[0058] Step 2.3: Using the debiased conditional features as constraints, perform a reverse denoising operation on the noisy reference features after forward diffusion to obtain the repaired features. This includes the following sub-steps: Step 2.3.1: Place the first Noisy reference features after step diffusion Conditional characteristics after debiasing Common input DiT denoising process, predicting forward injection noise , represented as:

[0059] In the formula, This represents the noise injection process predicted by the DiT network; Step 2.3.2: Reconstruct the mean of the inverse denoising based on the predicted forward injected noise, expressed as:

[0060] In the formula, This represents the mean reconstructing process of reverse denoising based on predicted forward-injected noise; Step 2.3.3: Construct a Gaussian inverse transition distribution using a fixed diagonal covariance, expressed as:

[0061] In the formula, Indicates a Gaussian inverse transition distribution. Indicates the first scalar variance of the backsampling step. It is the identity matrix; Step 2.3.4: Perform backsampling iterations step by step along the expansion step number, where the back transition distribution of each step is in Gaussian form, expressed as:

[0062] Stepwise sampling to Finally, the repair features are obtained. .

[0063] Step 2.4: Evaluate the trajectory fluctuation uncertainty of the repair features and generate position-by-position and channel-by-channel fusion weights.

[0064] Although diffusion-based restoration can mitigate the interference of low-light conditions on features in complex traffic scenarios, it is essentially a stepwise sampling and restoration process, and its output may still contain uncertainties in local areas. While the restoration results are generally better than the original degraded observation features, local offsets or over-restoration may still occur. Directly inputting these restored features indiscriminately into subsequent detection branches will also affect detection quality. This invention proposes trajectory fluctuation uncertainty estimation, specifically including the following: Step 2.4.1: In reverse denoising, the predictions of the final repaired features by different denoising steps fluctuate. If the diffusion repair process in a region corresponding to a certain spatial location is relatively stable, then the predictions of the final repair results for that location by each denoising step will be relatively consistent. Based on this phenomenon, the end of the reverse denoising process is selected. L (This embodiment) L =5) denoising steps of repair features, for each step of repair features Flattened into a sequence along spatial dimensions ; Step 2.4.2: Calculate the position-level trajectory fluctuation uncertainty using variance statistics and channel averaging, expressed as:

[0065] In the formula, Indicates the first spatial location Uncertainty response, This represents variance statistics, where D represents the number of channels; After calculation for all spatial locations, the following results were obtained. Reshaping back into the spatial domain yields uncertainty. This quantity reflects the degree of consistency in the estimation of the final recovery result at the same location by different denoising steps during the same reverse repair process.

[0066] Step 2.4.3: Addressing Uncertainty Considered as a single-channel feature map After a Convolution performs channel mapping, and then the channel is further mapped using the Sigmoid activation function. Intervals are used to obtain position-by-position and channel-by-channel fusion weights. The larger the... This indicates that the higher the uncertainty of the current location in restoration, the more the fusion should favor preserving the degraded observation features, and vice versa.

[0067] Step 2.5: Based on the fusion weights, the repair features and degradation observation features are fused element-wise with weights to obtain the fused features; specifically: Based on the fusion weights, and Perform element-wise weighted fusion to obtain fusion features. , represented as:

[0068] In the formula, .

[0069] Example 4 Combination Figure 1 and Figure 3 Step 3 specifically includes: Step 3.1: Merge features Input the detection decoder to generate the detection embedding set of the current video frame. ,in Indicates the number of embedded elements detected. ; Meanwhile, the current tracking embedding set generated by the Timing Interaction Module (TIM) from the previous video frame is denoted as... , This indicates the number of trajectories that are still being preserved. ; Step 3.2: Perform linear projection on each detection embedding and tracking embedding using a linear layer, as shown below:

[0070] In the formula, , This represents the linear feature representation of the detection embedding and the tracking embedding. and It is a learnable mapping matrix; Step 3.3: For each detection embedding Construct its connection with the first j The competition relationship representation vector of the tracking embedding is denoted as:

[0071] In the formula, This represents element-wise multiplication. This indicates a concatenation operation. This relation vector simultaneously encodes the individual representations of the detection embedding and the tracking embedding, the differences between their representations, and their interaction relationships, which can be used to reflect the intensity of their competition under local occlusion.

[0072] Step 3.4: Using MLP, represent the competition relationship as a vector. Mapped to relation token ; Step 3.5: Use a shared linear hierarchical mapping to directly map relation tokens to obtain the competition score. , represented as:

[0073] In the formula, , For learnable parameters, for all Relationship sharing; Step 3.6: Embedding the fixed detection Select the top-scoring embeddings with the highest competitive scores from all tracking embeddings. These constitute a set of local competitive neighborhood indexes:

[0074] In the formula, Indicates the relationship with the first The detection embedding has the strongest competitive relationship. A set of tracking embedded indexes; This represents the TopK statistical algorithm; Step 3.7: Place the first Detection embedding Mapped to query vector , represented as:

[0075] In the formula, It is a learnable mapping matrix; Step 3.8: For the set of locally competing neighborhood indexes Each relationship token within Generate key vectors respectively Sum value vector , represented as:

[0076] In the formula, , It is a learnable mapping matrix; Step 3.9: Place the first Each detection embedding contains all key vectors and value vectors within its corresponding neighborhood, stacked together as follows:

[0077] Let query matrix for:

[0078] Step 3.10: Calculate the first... Each detection embedding is a normalized response weight for each relation token in its local competitive neighborhood. ; Step 3.11: Define the first Each detection embeds a corresponding competitive context. , represented as:

[0079] Competition Context This indicates the competitive context information formed when the current detection is embedded in the corresponding local competitive neighborhood and is jointly influenced by the surrounding tracked targets.

[0080] Step 3.12: Embed the original detection With the context of competition By splicing them together, we can obtain Then it is mapped back through a linear layer. Dimension, obtained Finally, the competition-enhanced detection embedding is generated in the form of residuals:

[0081] In the formula, This indicates the enhanced detection embedding. This represents the linear normalization operation; Perform steps 3.1 to 3.12 on all detection embeddings to obtain the optimized detection embedding set. , represented as: .

[0082] Example 5 Step 4 specifically includes: Step 4.1: Embed the tracking data into the set and the optimized detection embedding set Input to the joint decoder, output the current embedding of all preserved trajectories. ; Step 4.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require The confidence level is obtained from the prediction head of the input baseline. With bounding box ; Step 4.3: The baseline trajectory management module receives the embedding of all currently retained trajectories. Confidence level With bounding box While performing new target initialization, activity trajectory update and short-term lost trajectory maintenance, and outputting tracking results, the trajectory status is sent to the baseline time-series interaction module TIM to generate the tracking embedding for the next moment.

[0083] Example 6 In step 5, the model can be trained and evaluated by referring to the following steps.

[0084] Step 5.1: Initialize the model using the official DAB-Deformable-DETR weights pre-trained on the COCO dataset. During training, referencing the implementation details of the baseline MeMOTR, basic data augmentation techniques such as random scaling and cropping are employed, and the input image's short side is scaled to 800, with the long side capped at 1536. The optimizer used is AdamW, with an initial learning rate set to... The number of training rounds was set to 18. The video clip length was increased gradually from 2 frames to 5 frames.

[0085] Step 5.2: Train and evaluate the model using the low-light and partial occlusion datasets FastTracker, CityFlow, and TrafficMOT, and visualize the results. Compare the results with the baseline models MeMOTR and ByteTrack on the same datasets. The results analysis is as follows: like Figure 4 As shown, in the FastTracker dataset, this invention stably detected a total of 9 vehicles in three consecutive frames, with the detection box positions more closely matching the real vehicles, and the overall results are relatively complete. MeMOTR in the... t Only 4 vehicles were detected in the first frame, resulting in a missed detection of vehicles in the dark area on the right. t The vehicle was only detected in +1 frame, indicating a delayed detection; meanwhile, a vehicle that had just entered the field of view on the right was not detected in time. ByteTrack in the... t Only 3 vehicles were detected in the first frame, missing not only vehicles in the dark area on the right but also vehicles in the middle of the intersection. t The vehicle in the middle of the intersection was only detected in frame +1, while vehicles in the dark area on the right continued to be missed. At the same time, MeMOTR and ByteTrack showed varying degrees of jitter in their detection boxes across three consecutive frames, while the method of this invention maintained more stable detection results.

[0086] like Figure 5 As shown, in the CityFlow dataset, this invention detects more vehicles across three consecutive frames, and the overall results are more stable. MeMOTR in the [missing data]... t The frame missed detecting the black vehicle behind it, which was obscured by the orange vehicle. t +1 frame to the t In frame +2, after the red vehicle on the right rear partially occludes with the adjacent white vehicle, a clear ID swapping phenomenon occurs. ByteTrack in the... tThe same frame also missed detecting the occluded vehicle. In the following two frames, the red vehicle behind failed to maintain an independent and stable target result, and its detection result was significantly suppressed by the adjacent white vehicle, indicating incomplete target detection under partial occlusion. In contrast, the present invention can maintain the target detection result relatively stably before and after occlusion, without obvious expansion of missed detections or identity confusion.

[0087] like Figure 6 As shown, in the TrafficMOT dataset, this invention maintains relatively complete and stable detection results across three consecutive frames. MeMOTR in the [missing data]... t The frame missed detecting vehicles in the low-light area behind, in the first frame t +1 frame to the t In frame +2, after partial occlusion of the two cars on the left, one of the cars exhibits an ID change. ByteTrack in the... t The first frame also missed detecting vehicles behind it, and in the following two frames, another vehicle on the left was missed due to obstruction. In contrast, this invention detected more vehicles in three consecutive frames, and the detection results were more stable.

[0088] The combined visualization results from the three sets show that low lighting weakens vehicle edges, textures, and local structural information, causing degradation of detection-related features and making it easier to miss or delay detection of targets in dark areas. Local occlusion compresses the visible area of ​​the target, making the currently detected target more similar to neighboring tracked targets in spatial location and local appearance, thus reducing the discriminative power of the detection embedding. This ultimately manifests as missed detections, unstable detection results, and identity confusion after local occlusion. This invention uses a feature diffusion repair module to specifically restore features degraded by low lighting, and combines it with a local competition relationship adaptation module to explicitly mitigate the interference of neighboring trajectories on the detection embedding under local occlusion conditions. Therefore, in low-light scenarios like FastTracker, locally occluded scenarios like CityFlow, and scenarios where low lighting and occlusion coexist like TrafficMOT, it can detect more real targets and maintain more stable detection results, improving subsequent tracking performance.

Claims

1. A detection-enhanced multi-target vehicle tracking method based on feature repair and relation optimization, characterized in that, include: Step 1: Input video frames into the backbone network to extract shared features, and construct degradation observation features and reference features; Step 2: Perform mask-guided forward diffusion on the reference features, combine with degraded observation features for reverse denoising to obtain repaired features, and adaptively fuse with degraded observation features after uncertainty assessment to obtain fused features; Step 3: Input the fused feature into the detection decoder to generate the detection embedding, combine it with the historical tracking embedding to model local competition relationships, and enhance the obtained optimized detection embedding; Step 4: Input the optimized detection embedding and the historical tracking embedding into the joint decoder, update the trajectory through the trajectory management module, output the tracking result and generate the tracking embedding for the next moment; Step 5: Train and evaluate the model using a low-light and partial occlusion dataset and visualize the results.

2. The enhanced vehicle multi-target tracking method based on feature repair and relationship optimization according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: Place the first t Frame video frames Inputting ResNet-50 extracts local convolutional features, then using TransformerEncoder to model the global context, yielding shared features, denoted as... ,in and Spatial dimensions representing features Indicates the number of channels; Step 1.2: Shared features The process is divided into two routes to construct degraded observation features and reference features respectively: Shared features Enter one Convolution performs channel mapping and dimension alignment to obtain degraded observation features. ; At the same time, shared features Input a channel adaptive fusion module, the channel adaptive fusion module comprising: a Convolution, one Convolution, a function with an inflation rate of 2 Dilated convolution, one SE channel attention module, one Convolution, global average pooling, and sigmoid activation; finally, the channel machine fusion weights output by the sigmoid activation are broadcast in the spatial dimension and shared with the features. Perform element-wise multiplication to obtain the reference feature. .

3. The enhanced vehicle multi-target tracking method based on feature repair and relationship optimization according to claim 1, characterized in that, Step 2 specifically includes: Step 2.1: Perform token-level mask-guided forward diffusion on the reference features, and introduce a degenerate adaptive noise scheduling mechanism to optimize the noise injection process of forward diffusion; Step 2.2: Perform brightness correction processing on the degraded observation features to eliminate local brightness interference and obtain the debiased conditional features; Step 2.3: Using the debiased conditional features as constraints, perform a reverse denoising operation on the noisy reference features after forward diffusion to obtain the repaired features; Step 2.4: Evaluate the trajectory fluctuation uncertainty of the repair features and generate position-by-position and channel-by-channel fusion weights; Step 2.5: Based on the fusion weights, perform element-wise weighted fusion of the repair features and the degradation observation features to obtain the fused features.

4. The detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization according to claim 3, characterized in that, Step 2.1 specifically includes: Step 2.1.1: For the reference feature The absolute mean of the channel dimensions is used to obtain the response score for each token. After min-max normalization and fixed threshold binarization, a token-level reliability mask is obtained. ,in, , representing the number of tokens; Step 2.1.2: Broadcast the token on the channel dimension to expand it into... , thus obtaining the expanded mask; Step 2.1.3: Calculate the proportion of unreliable tokens in the token-level reliable mask and use it as a scalar for degradation strength, expressed as: In the formula, This represents the proportion of unreliable tokens in the current frame, which is also a scalar value indicating the degradation intensity. ; Step 2.1.4: Use a standard linear noise schedule Generate lightly and heavily noisy scheduling sequences, specifically: given the number of diffusion steps. First in the interval Press up by number of steps Generate uniform linear interpolation ,make ;structure And obtain the corresponding: In the formula, This is a slightly noisy scheduling sequence. This is a heavily noisy scheduling sequence; Step 2.1.5: Utilize the degradation intensity scalar Linear interpolation is performed on the two sets of scheduling sequences to obtain the degenerate adaptive noise scheduling coefficients, expressed as: In the formula, This represents the noise scheduling coefficient at step k in the recursive formula of the regular Markov chain after degenerate adaptive scheduling. Represents the degenerate adaptive noise scheduling coefficient; Step 2.1.6: Combining the expanded mask and the degenerate adaptive noise scheduling coefficients, perform mask-guided forward diffusion on the reference features to obtain noisy reference features, as shown below: In the formula, This represents element-wise multiplication. As the starting point of diffusion, For the first Noisy reference features after step diffusion It is forward-injected Gaussian noise. For the first Step noise scheduling coefficient; This represents a standard Gaussian distribution with a mean of 0 to a variance of the identity matrix. Represents the identity matrix.

5. The detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization according to claim 4, characterized in that, Step 2.2 specifically involves: Extracted degradation observation features Through a step size of 1 Average pooling (AvgPool) is used to obtain the brightness response features. ; Then reduce the degradation of observation features by using residuals. Local brightness interference in the image is used to obtain the conditional features after polarization removal. : In the formula, Indicates global average pooling. This is a channel-level global statistical vector.

6. The detection-enhanced multi-target tracking method for vehicles based on feature repair and relationship optimization according to claim 5, characterized in that, Step 2.3 specifically includes: Step 2.3.1: Place the first Noisy reference features after step diffusion Conditional characteristics after debiasing Common input DiT denoising process, predicting forward injection noise , represented as: In the formula, This represents the noise injection process predicted by the DiT network; Step 2.3.2: Reconstruct the mean of the inverse denoising based on the predicted forward injected noise, expressed as: In the formula, This represents the mean reconstructing process of reverse denoising based on predicted forward-injected noise; Step 2.3.3: Construct a Gaussian inverse transition distribution using a fixed diagonal covariance, expressed as: In the formula, Indicates a Gaussian inverse transition distribution. Indicates the first scalar variance of the backsampling step. It is the identity matrix; Step 2.3.4: Perform backsampling iterations step by step along the expansion step number, where the back transition distribution of each step is in Gaussian form, expressed as: Stepwise sampling to Finally, the repair features are obtained. .

7. The enhanced vehicle multi-target tracking method based on feature repair and relationship optimization according to claim 6, characterized in that, Step 2.4 specifically involves: Step 2.4.1: Select the reverse denoising end L Each denoising step involves the restoration of features, and the restoration features at each step... Flattened into a sequence along spatial dimensions ; Step 2.4.2: Calculate the position-level trajectory fluctuation uncertainty using variance statistics and channel averaging, expressed as: In the formula, Indicates the first spatial location Uncertainty response, This represents variance statistics, where D represents the number of channels; After calculation for all spatial locations, the following results were obtained. Reshaping back to the spatial domain yields uncertainty. ; Step 2.4.3: Addressing Uncertainty Considered as a single-channel feature map After a Convolution performs channel mapping, and then the channel is further mapped using the Sigmoid activation function. Intervals are used to obtain position-by-position and channel-by-channel fusion weights. .

8. The detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization according to claim 7, characterized in that, In step 2.5, based on the fusion weights, the following steps are performed: and Perform element-wise weighted fusion to obtain fusion features. , represented as: In the formula, .

9. The detection-enhanced multi-target tracking method for vehicles based on feature repair and relationship optimization according to claim 1, characterized in that, Step 3 specifically includes: Step 3.1: Merge features Input the detection decoder to generate the detection embedding set of the current video frame. ,in Indicates the number of detected embeddings. ; Meanwhile, the current tracking embedding set generated by the Timing Interaction Module (TIM) from the previous video frame is denoted as... , This indicates the number of trajectories that are still being preserved. ; Step 3.2: Perform linear projection on each detection embedding and tracking embedding using a linear layer, as shown below: In the formula, , This represents the linear feature representation of the detection embedding and the tracking embedding. and It is a learnable mapping matrix; Step 3.3: For each detection embedding Construct its connection with the first j The competition relationship representation vector of the tracking embedding is denoted as: In the formula, This represents element-wise multiplication. Indicates a splicing operation; Step 3.4: Using MLP, represent the competition relationship as a vector. Mapped to relation token ; Step 3.5: Use a shared linear hierarchical mapping to directly map relation tokens to obtain the competition score. , represented as: In the formula, , For learnable parameters, for all Relationship sharing; Step 3.6: Embedding the fixed detection Select the top-scoring embeddings with the highest competitive scores from all tracking embeddings. These constitute a set of local competitive neighborhood indexes: In the formula, Indicates the relationship with the first The detection embedding has the strongest competitive relationship. A set of tracking embedded indexes; This represents the TopK statistical algorithm; Step 3.7: Place the first Detection embedding Mapped to query vector , represented as: In the formula, It is a learnable mapping matrix; Step 3.8: For the set of locally competing neighborhood indexes Each relationship token within Generate key vectors respectively Sum value vector , represented as: In the formula, , It is a learnable mapping matrix; Step 3.9: Place the first Each detection embedding contains all key vectors and value vectors within its corresponding neighborhood, stacked together as follows: Let query matrix for: Step 3.10: Calculate the first... Each detection embedding is a normalized response weight for each relation token in its local competitive neighborhood. ; Step 3.11: Define the first Each detection embedding corresponds to a competitive context. , represented as: Step 3.12: Embed the original detection With the context of competition By splicing them together, we can obtain Then it is mapped back through a linear layer. Dimension, obtained Finally, the competition-enhanced detection embedding is generated in the form of residuals: In the formula, This indicates the enhanced detection embedding. This represents the linear normalization operation; Perform steps 3.1 to 3.12 on all detection embeddings to obtain the optimized detection embedding set. , represented as: 。 10. The detection-enhanced multi-target vehicle tracking method based on feature repair and relationship optimization according to claim 1, characterized in that, Step 4 specifically includes: Step 4.1: Embed the tracking data into the set and the optimized detection embedding set Input to the joint decoder, output the current embedding of all preserved trajectories. ; Step 4.2: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] The confidence level is obtained from the prediction head of the input baseline. With bounding box ; Step 4.3: The baseline trajectory management module receives the embedding of all currently retained trajectories. Confidence level With bounding box While performing new target initialization, activity trajectory update and short-term lost trajectory maintenance, and outputting tracking results, the trajectory status is sent to the baseline time-series interaction module TIM to generate the tracking embedding for the next moment.

Images