Video target tracking method based on multi-layer attention feature enhancement

By employing a multi-layer attention feature enhancement method and utilizing the initial detection guidance map and channel correlation map, the robustness of video target tracking algorithms in complex scenarios is insufficient, achieving high-precision target tracking results.

CN123118699APending Publication Date: 2026-11-17SHAANXI LUHENG ELECTRONIC TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611395908.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-09
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing video target tracking algorithms struggle to effectively utilize the spatiotemporal phase structure of video sequences when processing high-resolution video streams, resulting in insufficient robustness in complex scenes. They are particularly prone to feature drift and positioning errors when faced with sudden changes in illumination, motion blur, and background occlusion.

Method used

By constructing a multi-layer attention feature enhancement method, we utilize the initial detection guidance map and channel correlation map to perform channel-wise weighted summation and complex domain mapping. Combined with dynamic noise suppression threshold and adaptively adjusted weight matrix, we perform dynamic filtering to output temporal synchronous features, thereby improving the robustness and accuracy of the features.

Benefits of technology

It effectively overcomes feature distortion caused by sudden changes in illumination and motion blur, improves the accuracy and robustness of target tracking, and ensures high-precision target localization and scale estimation in complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118699A_ABST
    Figure CN123118699A_ABST
Patent Text Reader

Abstract

This invention discloses a video target tracking method based on multi-layer attention feature enhancement, belonging to the field of computer vision technology. The method includes the following steps: acquiring video frames and extracting features; outputting denoised features through spatiotemporal prediction anti-noise closed-loop calibration; generating a binarized foreground mask by performing material-aware fusion and topology evolution; correcting the output local enhanced features using constraint fusion; generating multi-layer fused features through spatiotemporal domain collaborative modulation fusion; outputting the final enhanced features by performing feature phase-guided enhancement using multi-source contrast parameters; constructing a target template based on temporal adaptive fusion; performing cross-correlation operations to achieve target localization and scale estimation; updating the template through convergence constraint correction; and determining the target bounding box of the current frame. This invention overcomes feature distortion caused by sudden changes in illumination and motion blur, achieving improved target tracking accuracy and robustness in complex scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a video target tracking method based on multi-layer attention feature enhancement. Background Technology

[0002] With the continuous expansion of video surveillance data scale and the constant improvement of acquisition resolution, classical computers face severe computational challenges in real-time target tracking of high-resolution video streams. Existing video target tracking algorithms, such as tracking methods based on correlation filtering or deep Siamese networks, although improving localization efficiency through feature matching, mainly rely on the statistical distribution of pixel values ​​of target appearance features for template updating and searching. This method, which is purely based on appearance features, ignores the complex spatiotemporal phase structure implicit in the video sequence (such as phase distortion caused by motion blur and sudden changes in illumination) and deep signal domain phase consistency correlations, resulting in frequency domain information distortion when constructing feature representations, thus limiting the robustness of long-term target tracking in complex scenes. In addition, classical tracking methods often fail to fully utilize the instantaneous phase statistical characteristics of inter-frame features to dynamically constrain the filtering process, leading to severe feature drift when dealing with fast motion or background occlusion, increasing tracking and localization errors.

[0003] Therefore, how to provide a video target tracking method based on multi-layer attention feature enhancement is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] This invention proposes a video target tracking method based on multi-layer attention feature enhancement. It constructs a channel correlation map based on feature sampling from an initial detection guidance map, and then performs channel-wise weighting using a feature modulation matrix to output weighted corrected features. Complex domain mapping is applied to the weighted corrected features to extract the instantaneous phase of the features, and a dynamic noise suppression threshold is constructed based on multi-scale frequency analysis to perform phase continuity correction. The corrected phase is used to construct an adaptively adjustable weight matrix and temporal aggregation scale, and dynamic convolution is performed to output temporal synchronous features. A spatial adaptive suppression factor is constructed by combining global variance distribution and local gradient structure field, and dynamic filtering is performed to output the final enhanced features. This process, by establishing an enhancement mapping path from "channel-level modulation" to "phase domain correction" and then to "spatial domain adaptive filtering," effectively overcomes feature distortion caused by sudden changes in illumination and motion blur, ensuring that the final enhanced features accurately reflect the essential structure of the target. This achieves the technical effect of improving target tracking accuracy and robustness in complex scenes.

[0005] The video target tracking method based on multi-layer attention feature enhancement according to embodiments of the present invention specifically includes:

[0006] The current video frame is acquired to extract a multi-level initial feature set, a spatiotemporal prediction anti-noise closed loop is constructed, and a closed loop calibration decoupling transformation is performed to output a multi-level denoised feature set.

[0007] Get the target bounding box of the previous frame, construct a foreground guidance map based on motion prediction, perform material-aware fusion and topological dynamic evolution, and output a binarized foreground mask;

[0008] The initial spatial offset is predicted based on a multi-level denoised feature set and a binarized foreground mask. The boundary gradient field constraint and the neighborhood correlation constraint are used for fusion and adaptive correction to output a local enhanced feature set.

[0009] Based on a multi-level denoising feature set and a local enhancement feature set, spatial-frequency domain collaborative modulation fusion is performed using contrast response and frequency domain filtering guidance to output a multi-level fused feature set.

[0010] Foreground mask guidance is generated based on multi-layer fusion feature set, feature phase guidance enhancement is performed using multi-source contrast parameters, and the final enhanced feature is output.

[0011] The target template is constructed by temporal adaptive fusion based on inter-frame features, and cross-correlation operation is performed with the final enhanced features to achieve target localization and scale estimation, and the target bounding box and confidence score of the current frame are output.

[0012] The update coefficients are calculated based on the target bounding box and confidence score of the current frame. Convergence constraint correction is performed on the features of the current frame, and the template features and confidence distribution are updated to determine the target bounding box of the current frame.

[0013] Optionally, the spatiotemporal prediction noise-resistant closed loop specifically includes:

[0014] Perform multi-level convolution calculations and downsampling on the current video frame, extract spatial detail features and semantic features at different scales along the network depth, and output a multi-level initial feature set;

[0015] Based on the current video frame and the previous video frame, a pixel-level dense motion vector field is calculated, and a motion consistency constraint mapping is constructed based on the pixel-level dense motion vector field.

[0016] The motion consistency constraint mapping is used to perform a reverse twisting geometric alignment operation on the deep features in the multi-level initial feature set. The geometrically aligned deep features are then adaptively fused with the shallow features to output a temporally aligned multi-level feature set.

[0017] Based on the temporally aligned multi-level feature set, the feature change trend of adjacent pixels in the temporal dimension is calculated, the theoretical feature value at the current moment is predicted based on the feature change trend, and the predicted residual distribution is calculated based on the theoretical feature value and the actual observed value.

[0018] Based on the predicted residual distribution, self-moving noise suppression weights are generated. The self-moving noise suppression weights are then used to perform feature correction on the temporally aligned multi-level feature set, and a noise suppression feature set is output.

[0019] Based on the noise suppression feature set, the local spatial attention weight and inter-channel correlation matrix of the feature map are calculated. The composite dispersion index is calculated by combining the mean and variance of the channel response distribution. A dynamic channel filtering threshold is generated based on the composite dispersion index. The dynamic channel filtering threshold is used to perform spatially aware channel filtering on the noise suppression feature set, and the filtered multi-level initial feature set is output.

[0020] Based on the selected multi-level initial feature set, the semantic similarity matrix and hierarchical expression integrity index between each feature layer are calculated. The inter-level redundancy coefficient is calculated based on the semantic similarity matrix, and the adaptive compensation weight is generated based on the hierarchical expression integrity index.

[0021] Based on the inter-level redundancy coefficient and the adaptive compensation weight, a closed-loop calibration decoupling transformation is performed on the selected multi-level initial feature set to output a multi-level denoised feature set.

[0022] Optionally, material-aware fusion specifically includes:

[0023] Obtain the target bounding box of the previous frame, predict the center position of the current frame through inter-frame optical flow tracing, and calculate the target scale fluctuation index by combining the geometric dimensions of the target bounding box of the previous frame with the spatiotemporal motion confidence.

[0024] Based on the target scale fluctuation index, a dynamic Gaussian distribution standard deviation is generated. A two-dimensional Gaussian distribution weight map is constructed by combining the predicted value of the current frame center position with the dynamic Gaussian distribution standard deviation, and a foreground mask guidance map is output.

[0025] Based on the foreground mask guide map, an anisotropic diffusion field is constructed using the predicted value of the current frame center position as the signal emission origin. The normal structure change rate and tangential texture complexity of each pixel position in the foreground mask guide map are calculated. The structural impedance response value is calculated based on the normal structure change rate and the tangential texture complexity. The physical material-aware mapping is performed on the foreground mask guide map using the structural impedance response value, and a material-aware enhanced guide map is output.

[0026] Based on the material-aware enhancement guide map, the directional consistency matrix and spatial structure response intensity of the local pixel gradient are calculated. An edge degradation sensitivity factor is generated based on the directional consistency matrix and the spatial structure response intensity. The material-aware enhancement guide map is then adjusted using the edge degradation sensitivity factor to perform nonlinear response adjustment, and an edge enhancement guide map is output.

[0027] Based on the edge enhancement guidance map, the coordinates of high-confidence connected components in the edge enhancement guidance map are extracted as the initial growth point set, and the geometric gradient curvature of the boundary of the initial growth point set and the neighborhood smoothing constraint cost are calculated.

[0028] The total variational energy minimization driving value is calculated based on the geometric gradient curvature and the neighborhood smoothing constraint cost. The total variational energy minimization driving value is used to perform morphological diffusion update and hole filling on the neighborhood pixels, and the topological complete mask map is output.

[0029] Based on the topologically complete mask, the grayscale value distribution of the topologically complete mask is mapped to the core confidence region, the transition edge region and the background noise region, and the peak response intensity of the core confidence region and the gradient fidelity threshold of the transition edge region are calculated.

[0030] A hierarchical truncation constraint is constructed based on the peak response intensity and the gradient fidelity threshold. The hierarchical truncation constraint is used to perform pixel-level classification and truncation processing on the topological complete mask image, and a binary foreground mask is output.

[0031] Optionally, the fusion adaptive correction specifically includes:

[0032] Based on the multi-level denoising feature set and the foreground mask guiding map, the joint distribution of feature response confidence and mask geometric gradient is calculated, and nonlinear feature mapping is performed based on the joint distribution to output the mask feature set;

[0033] A regular sampling grid is constructed based on the mask feature set, and the nodes of the regular sampling grid are determined as initial sampling points;

[0034] Based on the mask feature set, a weighted summation and correlation comparison are performed on the pixels in the feature space neighborhood of the initial sampling point to calculate the initial spatial offset of the initial sampling point.

[0035] The gradient field distribution of the foreground and background boundaries is calculated based on the foreground mask guide map. A correction vector pointing to the high confidence region of the edge is calculated based on the gradient field distribution and the initial spatial offset. The trajectory is iteratively updated using the correction vector on the initial spatial offset, and the effective spatial offset is output.

[0036] Construct the neighborhood topology connection relationship between the initial sampling points, define the elastic constraint coefficient between nodes based on the neighborhood topology connection relationship, calculate the elastic deformation force generated by the relative displacement between nodes based on the effective spatial offset of the neighboring nodes, use the elastic deformation force to smooth and correct the effective spatial offset, and output the physical constraint spatial offset.

[0037] Based on the physical constraint space offset and the mask feature set, the target space geometric state of the initial sampling point is calculated, and local adaptive aggregation is performed on the sampling features based on the target space geometric state to output a local enhanced feature set.

[0038] Optionally, the spatial-frequency domain cooperative modulation fusion specifically includes:

[0039] The first spliced ​​feature is generated by upsampling the deep features of the multi-level denoising feature set and concatenating them with the shallow features of the local enhancement feature set.

[0040] The shallow features of the multi-level denoising feature set are downsampled and the deep features of the local enhancement feature set are concatenated to generate the second concatenated feature;

[0041] Spatial Gaussian difference calculation is performed on the first splicing feature and the second splicing feature respectively to generate a joint local contrast response map. Non-uniform suppression is performed on the first splicing feature using the joint local contrast response map to generate a first suppressed feature, and non-linear gain is performed on the second splicing feature to obtain a second gained feature. An adaptive enhanced feature set is output.

[0042] An orthogonal basis transformation is performed on the adaptive enhancement feature set to generate a spatial frequency distribution map. The frequency response threshold is iteratively solved based on the spatial arrangement structure parameters. The frequency transmission weight matrix of the spatial frequency distribution map is calculated using the frequency response threshold. The frequency component screening and inverse orthogonal basis transformation are performed on the spatial frequency distribution map using the frequency transmission weight matrix, and the filtered enhancement feature set is output.

[0043] A dynamic weighted coefficient matrix is ​​generated by performing channel-level correlation self-aggregation calculation on the first suppression feature and the second gain feature. The dynamic weighted coefficient matrix is ​​then used to perform channel-by-channel weighted modulation on the filter enhancement feature set to output a multi-layer fused feature set.

[0044] Optionally, the characteristic phase-guided enhancement specifically includes:

[0045] Target detection is performed on the first frame image to obtain an initial target bounding box, and an initial detection guidance map is generated based on the initial target bounding box and a mask mapping is performed on the first frame image.

[0046] Based on the initial detection guidance map and the multi-layer fusion feature set, region sampling is performed to extract anchor point features and negative sample features; the first semantic alignment vector between the anchor point features and the first frame target feature map and the second semantic alignment vector between the anchor features and the negative sample features are calculated.

[0047] Calculate a vector-level first similarity based on the first semantic alignment vector, calculate a vector-level second similarity based on the second semantic alignment vector, and fuse the vector-level first similarity and the vector-level second similarity to generate a set of contrast parameters;

[0048] Based on the set of contrast parameters, calculate the inter-channel correlation spectrum and the statistical moment distribution of each channel; construct a channel coupling constraint matrix using the inter-channel correlation spectrum; perform nonlinear mapping to generate a feature modulation matrix by combining the channel coupling constraint matrix and the statistical moment distribution of each channel; perform channel-by-channel weighting on the multi-layer fusion feature set based on the feature modulation matrix, and output weighted correction features;

[0049] Perform complex domain mapping and orthogonal decomposition on the weighted correction features to extract the instantaneous phase of the features; obtain the instantaneous phase of the target corresponding to the target feature map of the first frame, and calculate the phase consistency metric between the instantaneous phase of the features and the instantaneous phase of the target;

[0050] Multi-scale frequency analysis is performed based on the phase consistency metric and the signal strength at each scale is calculated. A dynamic noise suppression threshold is then constructed based on the signal strength at each scale.

[0051] The dynamic noise suppression threshold is used to perform continuity correction on the instantaneous phase of the feature to eliminate boundary abrupt changes. An adaptive adjustment weight matrix is ​​constructed based on the corrected instantaneous phase of the feature. The adaptive adjustment weight matrix is ​​used to perform component-level gain and suppression on the weight-corrected feature. The temporal aggregation scale is calculated according to the statistical distribution of the corrected instantaneous phase of the feature to perform dynamic convolution and output temporal synchronization features.

[0052] Based on the aforementioned temporal synchronization features, the global variance distribution and local gradient structure field within the feature domain are calculated; the global variance distribution and local gradient structure field within the feature domain are fused to construct a spatial adaptive suppression factor;

[0053] The spatial adaptive suppression factor is used to perform dynamic filtering on the temporal synchronization features, and the final enhanced features are output.

[0054] Optionally, the time-adaptive fusion specifically includes:

[0055] Input the initial frame target features and the historical template accumulated features, calculate the inter-frame similarity distribution between the initial frame target features and the historical template accumulated features; calculate the feature adaptive update rate based on the inter-frame similarity distribution to perform weighted cumulative fusion on the final enhanced features to generate target template features;

[0056] Perform channel-dimensional cross-correlation operation on the target template features and the final enhanced features to generate channel response mapping; perform spatial-dimensional cross-correlation operation on the target template features and the final enhanced features to generate spatial response mapping.

[0057] The channel response map and the spatial response map are fused to output a preliminary response matrix. The channel statistical distribution entropy of the preliminary response matrix is ​​calculated to generate an adaptive threshold weight. The adaptive threshold weight is used to perform channel-wise weighted modulation on the preliminary response matrix to output a target sensing response map.

[0058] Based on the target sensing response map, the gradient direction dispersion and local energy distribution statistics in the neighborhood are calculated, and a topology sensing filter kernel is constructed based on the gradient direction dispersion and local energy distribution statistics.

[0059] The topological structure sensing filter kernel is used to perform a convolution operation on the target sensing response map to smooth the response of non-target regions and sharpen the response of target edges, and output a two-dimensional response confidence map;

[0060] Based on the two-dimensional response confidence map, the initial peak position of the target in the current frame is extracted. The target temporal motion velocity vector is calculated by combining the target position information of historical frames. The temporal position statistical distribution of the target position information of historical frames is calculated to generate a dynamic velocity judgment threshold. The temporal motion velocity vector is compared with the dynamic velocity judgment threshold.

[0061] If the temporal motion velocity vector exceeds the dynamic velocity determination threshold, a spatiotemporal metric correction factor is constructed based on the motion parameters, and a geometric transformation is performed on the two-dimensional response confidence map to compensate for motion distortion, and a corrected response map is output.

[0062] Based on the modified response map, non-Euclidean geometric metrics and topological features are calculated, an adaptive sampling matrix is ​​constructed, and nonlinear resampling interpolation is performed to eliminate discrete sampling errors and output a geometrically continuous optimized response map.

[0063] Based on the geometric continuity optimization response map, the moment distribution characteristics of the response values ​​are calculated to solve the sub-pixel-level center coordinates and geometric scale parameters of the target. The sub-pixel-level center coordinates are mapped to the video frame coordinate system and the target bounding box and confidence score are solved in combination with the geometric scale parameters.

[0064] Optionally, the convergence constraint correction specifically includes:

[0065] Obtain the historical confidence level distribution, calculate the distribution statistical mean and distribution statistical variance based on the historical confidence level distribution, and construct the dynamic confidence level boundary based on the distribution statistical mean and distribution statistical variance;

[0066] Obtain the confidence score of the current frame, calculate the confidence deviation based on the current frame confidence score and the distribution statistical mean, and calculate the deviation distance index based on the confidence deviation and the dynamic confidence boundary;

[0067] The inverse deviation weight is calculated based on the deviation distance index, and feature update coefficients are constructed based on the inverse deviation weight. The target bounding box of the current frame and the multi-level denoising feature set are obtained. Based on the target bounding box of the current frame, the multi-level denoising feature set is cropped to obtain the cropped features of the current frame.

[0068] Obtain the accumulated features of historical templates, construct the correlation and coupling matrix between feature dimensions, map the accumulated features of historical templates and the current frame cropping features to the first temporal state parameter and the second temporal state parameter, respectively, calculate the state adjustment amplitude based on the confidence score of the current frame, and calculate the state synchronization difference measure between dimensions based on the first temporal state parameter and the second temporal state parameter.

[0069] Based on the state synchronization difference metric, the correlation coupling matrix and the state adjustment amplitude, the adaptive convergence constraint threshold is calculated, the target feature components that meet the adaptive convergence constraint threshold are selected, and the correlation correction feature vector is output.

[0070] The prior predicted features are calculated based on the historical template cumulative features. The feature prediction error covariance is calculated based on the prior predicted features. The adaptive weight adjustment factor is calculated based on the feature prediction error covariance. The weighted residual correction is performed on the associated modified feature vector using the adaptive weight adjustment factor, and the corrected target feature vector is output.

[0071] The corrected target feature vector is updated by weighted fusion based on the feature update coefficients to obtain the updated template features of the current frame; the confidence score of the current frame is updated by weighted fusion based on the feature update coefficients to obtain the updated confidence distribution of the current frame, and the updated template features, the updated confidence distribution, and the updated target bounding box of the current frame are output.

[0072] This invention provides a video target tracking method based on multi-layer attention feature enhancement, involving computer vision and deep learning technologies, which has the following beneficial effects:

[0073] (1) By constructing a spatiotemporal prediction noise-resistant closed loop and performing closed loop calibration decoupling transformation, this invention can separate effective signals from environmental noise by utilizing temporal correlation characteristics, effectively filter out dynamic interference in video streams, solve the problem of weak noise resistance of existing feature extraction methods in complex backgrounds, and significantly improve the purity and robustness of multi-level initial features.

[0074] (2) By performing material-aware fusion and topological dynamic evolution, this invention can refine the target edge by combining surface material characteristics and structural association information, effectively distinguish similar foreground and background, solve the defect of blurred boundary in traditional segmentation methods under target deformation or occlusion, and greatly improve the generation accuracy of binarized foreground mask.

[0075] (3) By constructing a feature modulation matrix using a set of contrast parameters and constructing a dynamic noise suppression threshold using multi-scale frequency analysis, this invention can perform channel-by-channel weighting based on channel correlation and perform continuity correction on the instantaneous phase of the feature, effectively suppressing phase distortion and boundary abrupt changes, overcoming the limitation of traditional feature expression being sensitive to changes in illumination, and enhancing the structural expression ability of the final enhanced feature.

[0076] (4) By utilizing time-series adaptive fusion to construct a target template and performing cross-correlation calculations, this invention can combine convergence constraints to correct and update template features and confidence distribution, eliminate tracking drift errors in real time, ensure high accuracy of target positioning and scale estimation, and provide reliable technical support for high-precision target tracking in complex scenarios. Attached Figure Description

[0077] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0078] Figure 1 This is an overall flowchart of the video target tracking method based on multi-layer attention feature enhancement proposed in this invention;

[0079] Figure 2 This is a flowchart illustrating the working principle of the feature phase-guided enhancement method for video target tracking based on multi-layer attention feature enhancement proposed in this invention. Detailed Implementation

[0080] The invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0081] refer to Figure 1 and Figure 2 A video target tracking method based on multi-layer attention feature enhancement specifically includes:

[0082] The current video frame is acquired to extract a multi-level initial feature set, a spatiotemporal prediction anti-noise closed loop is constructed, and a closed loop calibration decoupling transformation is performed to output a multi-level denoised feature set.

[0083] Get the target bounding box of the previous frame, construct a foreground guidance map based on motion prediction, perform material-aware fusion and topological dynamic evolution, and output a binarized foreground mask;

[0084] The initial spatial offset is predicted based on a multi-level denoised feature set and a binarized foreground mask. The boundary gradient field constraint and the neighborhood correlation constraint are used for fusion and adaptive correction to output a local enhanced feature set.

[0085] Based on a multi-level denoising feature set and a local enhancement feature set, spatial-frequency domain collaborative modulation fusion is performed using contrast response and frequency domain filtering guidance to output a multi-level fused feature set.

[0086] Foreground mask guidance is generated based on multi-layer fusion feature set, feature phase guidance enhancement is performed using multi-source contrast parameters, and the final enhanced feature is output.

[0087] The target template is constructed by temporal adaptive fusion based on inter-frame features, and cross-correlation operation is performed with the final enhanced features to achieve target localization and scale estimation, and the target bounding box and confidence score of the current frame are output.

[0088] The update coefficients are calculated based on the target bounding box and confidence score of the current frame. Convergence constraint correction is performed on the features of the current frame, and the template features and confidence distribution are updated to determine the target bounding box of the current frame.

[0089] In this embodiment, the spatiotemporal prediction noise reduction closed loop specifically includes:

[0090] The system reads the current video frame image data with a resolution of 1920 pixels by 1080 pixels, inputs the image data into a 3x3 convolutional layer with a stride of 2 pixels to perform convolution calculation, then performs batch normalization and connects it to the ReLU linear rectified activation function to output a first-scale feature map; performs a 3x3 convolution calculation with a stride of 2 pixels and ReLU activation function processing on the first-scale feature map to output a second-scale feature map; performs a 3x3 convolution calculation with a stride of 2 pixels and ReLU activation function processing on the second-scale feature map to output a third-scale feature map; and concatenates the first-scale feature map, the second-scale feature map, and the third-scale feature map to generate a multi-level initial feature set.

[0091] Read the current video frame and the previous video frame with a resolution of 1920 pixels multiplied by 1080 pixels. Calculate the horizontal and vertical displacement of each pixel in the previous video frame in the current video frame. Combine the horizontal and vertical displacements to generate a two-dimensional vector. Arrange the two-dimensional vectors of all pixels according to their spatial positions to construct a pixel-level dense motion vector field. Set a motion consistency threshold and select pixels with a motion vector magnitude of less than 5 pixels and a directional deviation of less than 10 degrees as consistent motion pixels. Mark the consistent motion pixels with a value of 1 and the remaining pixels with a value of 0 to generate a motion consistency constraint mapping.

[0092] The third-scale feature map in the multi-level initial feature set generated in the previous step is read as the deep feature. The pixel-level dense motion vector field generated in the previous step is read, and the motion vector corresponding to each pixel in the deep feature is extracted. The motion vector coordinates are used to perform a reverse distortion geometric alignment operation on the deep feature to generate a geometrically aligned deep feature. The geometrically aligned deep feature is added element-wise to the first-scale feature map and connected to the Sigmoid activation function. Adaptive fusion processing is performed to output a temporally aligned multi-level feature set.

[0093] Read the temporally aligned multi-level feature set generated in the previous step, select the feature value of the same pixel position in the feature map at the current time and the feature value at the previous time, calculate the difference between the feature value at the previous time and the feature value at the current time as the feature change gradient, and use the feature change gradient to predict the theoretical feature value at the next time; calculate the absolute value of the difference between the theoretical feature value and the actual observed feature value at the current time to obtain the prediction residual value, and construct the prediction residual distribution.

[0094] Read the predicted residual distribution generated in the previous step, set the residual threshold, and when the predicted residual value is less than 3, set the weight coefficient to 0.1. When the predicted residual value is greater than 3, set the weight coefficient to 0.9 to generate the self-moving noise suppression weight. Multiply the self-moving noise suppression weight by the corresponding feature value in the time-aligned multi-level feature set generated in the previous step, perform a weighted multiplication operation, and output the noise suppression feature set.

[0095] Read the noise suppression feature set generated in the previous step, calculate the spatial attention weights of the feature maps using a 1x1 convolution kernel, and normalize them using a Softmax activation function to obtain the local spatial attention weights; calculate the global average pooling values ​​of the feature maps of different channels, calculate the correlation values ​​between channels through a fully connected layer, and construct the inter-channel correlation matrix; calculate the mean and variance of the response distribution of each channel, and multiply the mean by the variance to obtain the composite dispersion value; set the dispersion threshold to 0.5. When the composite dispersion value is less than 0.5, set the dynamic channel filtering threshold to 0.2; when the composite dispersion value is greater than 0.5, set the dynamic channel filtering threshold to 0.8; use the dynamic channel filtering threshold to filter the noise suppression feature set generated in the previous step, and output the filtered multi-level initial feature set.

[0096] Read the multi-level initial feature set generated in the previous step after filtering, calculate the cosine similarity values ​​between different feature layers, and construct a semantic similarity matrix; calculate the information entropy value of each feature layer using the information entropy formula, as a hierarchical expression integrity index; calculate 1 minus the values ​​in the semantic similarity matrix to obtain the inter-level redundancy coefficient; normalize the hierarchical expression integrity index and use it as the adaptive compensation weight; multiply the inter-level redundancy coefficient by the multi-level initial feature set generated in the previous step after filtering, subtract the product of the adaptive compensation weight and the inter-level redundancy coefficient, perform closed-loop calibration decoupling transformation, and output a multi-level denoised feature set.

[0097] In the specific implementation process, this implementation method inherits the basic logic of using optical flow method for temporal motion estimation and feature alignment in traditional video target tracking, but has made in-depth modifications to address the multi-level feature distortion and noise coupling problems caused by background clutter, motion blur and occlusion in dynamic scenes.

[0098] Traditional techniques typically rely solely on motion vectors for simple geometric distortion alignment, failing to utilize temporal feature change trends to predict model noise and generate self-motion suppression weights. This results in feature correction exhibiting lag and difficulty in filtering out noise interference that overlaps with the target motion when processing fast-moving or texture-deficient regions.

[0099] This implementation method creatively constructs an anti-noise feedback system from "pixel-level dense motion field construction" to "closed-loop calibration decoupling transformation". Based on the feature change trend of adjacent pixels, it calculates the predicted residual distribution of theoretical feature values ​​and actual observation values, generates self-motion noise suppression weights to correct temporal alignment features, combines local spatial attention and channel correlation matrix to calculate composite dispersion index to generate dynamic channel screening threshold, and uses inter-level redundancy coefficients and adaptive compensation weights to perform closed-loop calibration decoupling transformation.

[0100] This modified system completely solves the problem of impure feature representation caused by rigid alignment in traditional methods that ignore temporal prediction errors and noise redundancy between channels. It realizes the transformation of abstract motion prediction residuals into specific feature channel filtering and weight adjustment signals, effectively eliminating spurious effects and redundant semantic interference caused by background abrupt changes, and significantly improving the temporal stability and spatial discriminability of multi-level features in complex dynamic environments. It provides a highly robust data foundation for subsequent accurate target localization and feature fusion.

[0101] In this embodiment, material perception fusion specifically includes:

[0102] Read the top-left corner x-coordinate, top-left corner y-coordinate, width, and height values ​​of the target bounding box from the previous frame. The width and height of the target bounding box in the previous frame are in pixels. Calculate the horizontal and vertical displacements of the center point of the target bounding box in the current frame using the Lucas-Kanade sparse optical flow method. Add the horizontal displacement to the x-coordinate of the center point in the previous frame and the vertical displacement to the y-coordinate of the center point in the previous frame to obtain the predicted center position value in the current frame. Obtain the width and height of the target bounding box from the previous frame, calculate the product of the width and height as the target area, calculate the magnitude of the motion displacement vector in the previous frame, divide the displacement magnitude by the target area to obtain the spatiotemporal motion confidence score, multiply the confidence score by 0.5 as a weighting factor, and calculate the average rate of change of the width and height values ​​multiplied by the weighting factor to obtain the target scale fluctuation index.

[0103] Read the target scale fluctuation index, multiply the fluctuation index by a constant of 5 pixels to obtain the dynamic Gaussian distribution standard deviation, construct a two-dimensional grid with the horizontal axis ranging from -3 to +3 standard deviations and the vertical axis ranging from -3 to +3 standard deviations. Using the predicted value of the current frame center position as the grid center point, calculate the Gaussian probability density value of each coordinate point in the grid, normalize the value to between 0 and 1, and generate a two-dimensional Gaussian distribution weight map as the foreground mask guide map.

[0104] Read the foreground mask guide map, take the predicted value of the current frame center position as the origin, calculate the normal structure change rate of each pixel in the guide map using the anisotropic diffusion equation, calculate the tangential texture complexity of the pixel using the Sobel operator, divide the normal structure change rate by the tangential texture complexity to obtain the structural impedance response value, multiply the response value by the value of the corresponding pixel in the foreground mask guide map, perform physical material-aware mapping, and output the material-aware enhanced guide map.

[0105] Read the material-aware enhancement guide map, calculate the horizontal and vertical gradient values ​​of each pixel using a 3x3 Sobel operator, calculate the direction cosine similarity matrix of the gradient vector as the direction consistency matrix, calculate the sum of squared gradient magnitudes as the spatial structure response intensity, divide the value of the direction consistency matrix by the value of the spatial structure response intensity to obtain the edge degradation sensitivity factor, perform a nonlinear transformation on the factor using the Tanh hyperbolic tangent activation function, multiply the transformation result by the value of the corresponding pixel in the material-aware enhancement guide map, and output the edge enhancement guide map.

[0106] Read the edge enhancement guide map, set the confidence threshold to 0.7, extract the coordinates of connected components with pixel values ​​greater than 0.7 as the initial growth point set, calculate the gradient direction change rate of the boundary pixels of the growth point set as the geometric gradient curvature, calculate the sum of the absolute values ​​of the differences between the boundary pixels of the growth point set and the neighboring pixel values ​​as the neighborhood smoothing constraint cost, set the cost coefficient to 0.5, and use the weighted sum of the geometric gradient curvature and the neighborhood smoothing constraint cost as the driving value for minimizing the total variational energy. Use the driving value to update the pixel values ​​of the neighboring pixels, perform morphological diffusion update and hole filling, and output the topological complete mask map.

[0107] Read the complete topological mask image, statistically analyze the histogram distribution of pixel grayscale values ​​in the mask image, and define the region with grayscale values ​​greater than 0.8 as the core confidence region, the region with grayscale values ​​between 0.3 and 0.8 as the transition edge region, and the region with grayscale values ​​less than 0.3 as the background noise region. Calculate the average grayscale value of pixels in the core confidence region as the peak response intensity, and calculate the average gradient of pixel grayscale values ​​in the transition edge region as the gradient fidelity threshold. Set the value of pixels with grayscale values ​​greater than 0.9 times the peak response intensity and gradient values ​​greater than 1.1 times the gradient fidelity threshold to 255, and set the values ​​of other pixels to 0. Perform pixel-level classification and truncation processing, and output a binarized foreground mask.

[0108] In this embodiment, the fusion adaptive correction specifically includes:

[0109] The multi-level denoised feature set and the foreground mask guide map are read. The sigmoid activation function is used to normalize the multi-level denoised feature set to obtain the feature response confidence. The Sobel operator is used to calculate the gradient values ​​of the foreground mask guide map in the horizontal and vertical directions, and the square root of the sum of the squares of the gradient values ​​is used to obtain the mask geometric gradient. The feature response confidence is multiplied by the mask geometric gradient to construct a joint distribution matrix. The joint distribution matrix is ​​input into a fully connected layer containing 256 neurons to perform nonlinear feature mapping and output the mask feature set.

[0110] Read the mask feature set and construct a regular sampling grid with a row spacing of 16 pixels and a column spacing of 16 pixels. Determine the coordinates of the grid intersections as the initial sampling points. For each initial sampling point, extract the pixel values ​​in a square neighborhood with a side length of 9 pixels centered on that point from the mask feature set. Calculate the weighted sum of the pixels in the neighborhood using a Gaussian weighted kernel. Divide the feature value at the initial sampling point by the weighted sum to obtain the correlation ratio. Multiply the correlation ratio by the unit distance of 8 pixels to calculate the initial spatial offset of the initial sampling point.

[0111] Read the foreground mask guide image, use the Scharr operator to calculate the horizontal and vertical gradient components of the foreground and background boundaries in the foreground mask guide image, combine the horizontal and vertical gradient components to obtain the gradient field distribution, perform a dot product between the vector direction of the initial spatial offset and the direction of the maximum gradient in the gradient field distribution, calculate the direction similarity, multiply the direction similarity by the step size coefficient of 2 pixels to obtain the correction vector, use the correction vector to perform vector addition operation on the initial spatial offset, perform trajectory iterative update, and output the effective spatial offset.

[0112] Read the positional relationship between the initial sampling points, connect the sampling points of adjacent rows and adjacent columns to construct the neighborhood topology connection relationship, set the elastic constraint coefficient between the connected nodes to 0.5 Newtons per meter, calculate the difference between the effective spatial offsets of the neighborhood nodes as the relative displacement, multiply the relative displacement by the elastic constraint coefficient between the nodes to obtain the elastic deformation force, subtract the displacement corresponding to the elastic deformation force from the effective spatial offset, perform smoothing and correction, and output the physical constraint spatial offset.

[0113] Read the physical constraint space offset and the mask feature set, map the physical constraint space offset to the pixel coordinate system to obtain the target space geometry of the initial sampling point, determine the weight coefficients of bilinear interpolation based on the target space geometry, perform weighted summation on the feature values ​​of the corresponding pixels in the mask feature set using the weight coefficients, perform local adaptive aggregation, and output the local enhanced feature set.

[0114] In this embodiment, spatial-frequency domain collaborative modulation fusion specifically includes:

[0115] Read the deep features from the multi-level denoising feature set, and use a bilinear interpolation algorithm with a stride of 2 pixels to amplify the deep features by a factor of 2 in both the horizontal and vertical directions, outputting the upsampled deep features. Read the shallow features from the local enhancement feature set, and concatenate the channel dimensions of the upsampled deep features with those of the shallow features to generate the first concatenated feature. Read the shallow features from the multi-level denoising feature set, and use 2x2 max pooling with a stride of 2 pixels to perform downsampling on the shallow features, outputting the downsampled shallow features. Read the deep features from the local enhancement feature set, and concatenate the channel dimensions of the downsampled shallow features with those of the deep features to generate the second concatenated feature.

[0116] Read the first stitched feature, construct a Gaussian kernel with a standard deviation of 1.5 pixels, perform Gaussian convolution smoothing operation on the first stitched feature to obtain the first smoothed feature, and calculate the difference between the first stitched feature and the first smoothed feature to obtain the first Gaussian difference feature. Read the second stitched feature, perform Gaussian convolution smoothing operation on the second smoothed feature using a Gaussian kernel with a standard deviation of 1.5 pixels to obtain the second smoothed feature, and calculate the difference between the second stitched feature and the second smoothed feature to obtain the second Gaussian difference feature. Calculate the sum of the absolute values ​​of the first Gaussian difference feature and the second Gaussian difference feature to generate a joint local contrast response map. Read the joint local contrast response map, set the suppression coefficient to 0.1, multiply the joint local contrast response map value by the suppression coefficient and add it to the first stitched feature value to output the first suppressed feature. Read the joint local contrast response map, set the gain slope to 1.2, perform nonlinear mapping on the joint local contrast response map using the Sigmoid activation function and multiply the result by the gain slope to generate the second gained feature. Combine the first suppressed feature and the second gained feature as an adaptive enhancement feature set.

[0117] Read the adaptive enhancement feature set, perform discrete cosine transform to generate a spatial frequency distribution map, set the initial value of the spatial arrangement structure parameter to 0.1, and use an iterative update strategy with a step size of 0.05 to solve for the frequency response threshold; calculate the difference between the magnitude of each frequency component in the spatial frequency distribution map and the frequency response threshold. If the magnitude is greater than the threshold, set the frequency transmission weight to 1.0; otherwise, set it to 0.2, generating a frequency transmission weight matrix; multiply the frequency component values ​​of the spatial frequency distribution map with the values ​​of the frequency transmission weight matrix, perform frequency component filtering, and perform inverse discrete cosine transform on the filtered results to output the filtered enhancement feature set.

[0118] Read the first suppression feature and the second gain feature, calculate the Pearson correlation coefficient of the two features in the channel dimension, and construct a dynamic weighting coefficient matrix; read the filter enhancement feature set, use the dynamic weighting coefficient matrix to perform channel-by-channel weighted modulation on the filter enhancement feature set, and output a multi-layer fusion feature set.

[0119] In this embodiment, the characteristic phase-guided enhancement specifically includes:

[0120] The first frame image is read, and the target detection confidence threshold is set to 0.85. Regions with confidence greater than 0.85 are filtered out, and an initial target bounding box containing the target center coordinates and width and height is obtained. The first frame image is masked based on the pixel coordinates of the initial target bounding box, with pixel values ​​inside the box set to 1 and pixel values ​​outside the box set to 0, generating an initial detection guidance map. The multi-layer fusion feature set is read, and 256 feature vectors within the target region are extracted as anchor features based on the initial detection guidance map. 128 feature vectors from regions with a distance greater than 100 pixels from the target center are extracted as negative sample features. The first frame target feature map is read, and the cosine similarity between the anchor features and the first frame target feature map is calculated to obtain the first semantic alignment vector. The cosine similarity between the anchor features and the negative sample features is calculated to obtain the second semantic alignment vector. The magnitude of the first semantic alignment vector is calculated and its mean is taken to obtain the vector-level first similarity. The magnitude of the second semantic alignment vector is calculated and its mean is taken to obtain the vector-level second similarity. The vector-level first similarity is multiplied by 0.6, and the vector-level second similarity is multiplied by 0.4 to generate a contrast parameter set.

[0121] The contrast parameter set is read, and the inter-channel correlation spectrum is calculated using the Pearson correlation coefficient. The mean and variance of each channel's eigenvalues ​​are calculated as the statistical moment distribution. A correlation threshold of 0.3 is set, and elements in the correlation spectrum with values ​​greater than 0.3 are set to 1, while the rest are set to 0, constructing a channel coupling constraint matrix. The channel coupling constraint matrix is ​​multiplied by the variance values ​​in the statistical moment distribution and then fed into a Tanh activation function to generate a feature modulation matrix. The feature modulation matrix is ​​used to perform a channel-wise weighted multiplication on the multi-layer fused feature set, outputting weighted corrected features.

[0122] The system reads the weighted correction features, treats the feature values ​​as the real part of a complex number and the value 0 as the imaginary part, performs complex domain mapping and Hilbert transform, and extracts the complex phase angle as the instantaneous phase of the features. It then reads the first frame of the target feature map and extracts its instantaneous phase. The absolute value of the difference between the instantaneous phase of the features and the instantaneous phase of the target is calculated, and the absolute value of the difference is divided by pi to obtain the phase consistency metric. Three frequency scales are set: 2 Hz, 4 Hz, and 8 Hz. Based on the phase consistency metric, the signal mean is calculated at each scale as the signal strength at each scale. The minimum signal strength at each scale is multiplied by a coefficient of 0.8 to construct a dynamic noise suppression threshold. The system reads the instantaneous phase of the features. When the phase consistency metric is less than the dynamic noise suppression threshold, the phase value of the current pixel is replaced with the phase value of the previous pixel, a continuity correction is performed, and the corrected instantaneous phase of the features is output.

[0123] The corrected instantaneous phase of the feature is read, and the sine value of the phase is calculated as the adaptive adjustment weight matrix. Component multiplication is performed on the weighted corrected feature using the adaptive adjustment weight matrix. The standard deviation of the corrected instantaneous phase of the feature within the time window is calculated, and the unit of standard deviation is set to radians. The standard deviation is multiplied by a constant 5 as the temporal aggregation scale. A dynamic convolution kernel is constructed based on the temporal aggregation scale, and convolution operation is performed on the weighted feature to output the temporal synchronized feature.

[0124] The temporal synchronization features are read, and the variance of all feature values ​​is calculated to obtain the global variance distribution within the feature domain. The Sobel operator is used to calculate the horizontal and vertical gradients of the features, and the summation yields the local gradient structure field. The global variance distribution and the local gradient structure field are added together, and then normalized using the Sigmoid activation function to generate a spatially adaptive suppression factor. The spatially adaptive suppression factor is used to perform element-wise multiplication filtering on the temporal synchronization features, outputting the final enhanced features.

[0125] In the specific implementation process, this implementation method inherits the basic logic of generating initial bounding boxes and comparing feature similarity in traditional video target tracking, but it has been deeply modified to address the feature phase distortion and structural distortion problems caused by sudden changes in illumination, motion blur and background clutter in complex scenes.

[0126] Traditional techniques typically rely solely on feature amplitude responses for simple similarity measurement or weighted fusion, neglecting the analysis and utilization of instantaneous phase information of features. They cannot distinguish noise from effective signals from a frequency domain perspective, resulting in feature representation lacking robustness and being easily affected by phase abrupt changes when processing weak textures or low-contrast regions.

[0127] This implementation method creatively constructs a feature enhancement system from "semantic contrast parameter generation" to "phase domain dynamic filtering". It calculates the contrast parameter set based on the semantic alignment vector of anchor point features and negative sample features, uses the channel coupling constraint matrix to perform channel-by-channel weighted modulation to output weight correction features, and performs complex domain mapping on the weight correction features to extract instantaneous phase and phase consistency metric values. It combines multi-scale frequency analysis to construct a dynamic noise suppression threshold to correct phase continuity, and finally uses a spatial adaptive suppression factor to perform dynamic filtering to output the final enhanced features.

[0128] This modified system completely solves the problems of insufficient anti-interference ability and blurred structural boundaries caused by the lack of phase information in traditional methods. It realizes the transformation of abstract phase consistency measurement into concrete feature gain and suppression weight, effectively eliminates boundary abrupt changes and feature aliasing caused by non-stationary noise, and significantly improves the structural integrity and temporal synchronization of features in dynamic environments, providing a highly reliable feature foundation for subsequent high-precision target localization and template updating.

[0129] In this embodiment, time-adaptive fusion specifically includes:

[0130] Read the target features of the initial frame and the accumulated features of the historical template, calculate the cosine similarity between the target features of the initial frame and the accumulated features of the historical template, and normalize the similarity value to between 0 and 1 to obtain the inter-frame similarity distribution; set the baseline update rate to 0.1, multiply the inter-frame similarity distribution value by 0.1 to obtain the feature adaptive update rate; read the final enhanced features, multiply the final enhanced feature value by the feature adaptive update rate and add it to the accumulated feature value of the historical template, perform weighted cumulative fusion, and generate the target template features.

[0131] Read the target template features and the final enhanced features. In the channel dimension, perform a dot product operation on the target template features and the final enhanced features and then connect them to the Softmax activation function to generate a channel response map. In the spatial dimension, flatten the target template features and the final enhanced features into vectors and calculate the cross-correlation coefficient to construct a spatial response map. Add the channel response map values ​​and the spatial response map values ​​to obtain the preliminary response matrix. Calculate the proportion of each channel value in the preliminary response matrix to the total value and take the logarithm to base 2. Summate the values ​​to obtain the channel statistical distribution entropy. Set the entropy threshold to 2.5 bits. When the channel statistical distribution entropy is less than 2.5 bits, set the adaptive threshold weight to 1.2; otherwise, set it to 0.8. Use the adaptive threshold weight to perform channel-wise weighted modulation on the preliminary response matrix and output the target perception response map.

[0132] Read the target-aware response map, calculate the variance of the gradient direction within a 3x3 pixel window in the neighborhood to obtain the gradient direction dispersion, and calculate the sum of squares of the pixel values ​​in the neighborhood as the local energy distribution statistic; add the gradient direction dispersion value and the local energy distribution statistic value, and normalize them by using the Sigmoid activation function to generate the topology-aware filter kernel; use the topology-aware filter kernel to perform convolution operation on the target-aware response map to output a two-dimensional response confidence map.

[0133] Read the two-dimensional response confidence map and extract the pixel coordinates with the largest response value as the initial peak position of the target in the current frame; read the target position information of historical frames, calculate the Euclidean distance between the initial peak position of the target in the current frame and the target position in the historical frames, divide the Euclidean distance by the time interval of 33 milliseconds between the two frames to obtain the target temporal motion velocity vector; calculate the standard deviation of the target position information in historical frames as the temporal position statistical distribution, multiply the standard deviation by a coefficient of 3 to obtain the dynamic velocity judgment threshold; read the target temporal motion velocity vector and compare the magnitude of the target temporal motion velocity vector with the dynamic velocity judgment threshold.

[0134] If the magnitude of the target temporal velocity vector exceeds the dynamic velocity determination threshold, then motion parameters including rotation angle and scaling ratio are constructed as spatiotemporal metric correction factors; the spatiotemporal metric correction factors are used to perform an affine transformation on the two-dimensional response confidence map to compensate for motion distortion, and the corrected response map is output; if it does not exceed the threshold, the two-dimensional response confidence map is directly output as the corrected response map.

[0135] Read the modified response map, calculate the surface curvature of the response map as a non-Euclidean geometric measure, and extract the extreme points of the response as topological features. Based on the non-Euclidean geometric measure and topological features, construct an adaptive sampling matrix, and use a bicubic interpolation algorithm to perform nonlinear resampling interpolation calculation on the modified response map to eliminate discrete sampling errors and output a geometrically continuous optimized response map.

[0136] Read the geometric continuity optimization response map, calculate the zeroth and first moments of the response values, and use the first moment divided by the zeroth moment to obtain the sub-pixel-level center coordinates of the target; calculate the second central moment of the response values ​​and take the square root to obtain the geometric scale parameter; map the sub-pixel-level center coordinates to the video frame coordinate system, construct the bounding rectangle in combination with the geometric scale parameter, calculate the mean of the response values ​​within the bounding box as the confidence score, and output the target bounding box and the confidence score.

[0137] Read the historical confidence distribution of the most recent 30 frames, calculate the arithmetic mean of the confidence values ​​of all historical frames as the statistical mean of the distribution, and calculate the average of the sum of squares of the differences between the historical confidence values ​​and the statistical mean of the distribution as the statistical variance of the distribution. Set the confidence coefficient to 1.96, calculate the root mean square of the distribution, which is the square root of the statistical variance of the distribution. Use the statistical mean of the distribution plus the confidence coefficient multiplied by the statistical mean of the distribution to obtain the upper bound of the confidence, and use the statistical mean of the distribution minus the confidence coefficient multiplied by the statistical mean of the distribution to obtain the lower bound of the confidence. Combine the upper bound of the confidence and the lower bound of the confidence to construct the dynamic confidence boundary.

[0138] Read the confidence score of the current frame, calculate the absolute value of the difference between the confidence score of the current frame and the distribution statistical mean as the confidence deviation, read the dynamic confidence boundary, calculate the ratio of the confidence deviation to the width of the dynamic confidence boundary. If the confidence deviation is less than the width of the dynamic confidence boundary, set the ratio to 0.1; otherwise, set the ratio to 1.0 to obtain the deviation distance index.

[0139] Read the deviation distance index, set the constant value to 1.0, and divide the constant value by the sum of the deviation distance index and the constant value to obtain the inverse deviation weight; multiply the inverse deviation weight by a coefficient of 0.5 to construct the feature update coefficient; read the target bounding box and the multi-level denoising feature set of the current frame, perform a region clipping operation on the multi-level denoising feature set based on the coordinate parameters of the target bounding box of the current frame, extract the feature vectors within the bounding box, and obtain the clipped features of the current frame.

[0140] Read the historical template cumulative features, calculate the correlation between each dimension of the historical template cumulative features using the Pearson correlation coefficient, and construct the correlation coupling matrix between feature dimensions; map the historical template cumulative features to the first temporal state parameter, and map the current frame cropping features to the second temporal state parameter; calculate the state adjustment amplitude by multiplying the current frame confidence score by 0.2; calculate the Euclidean distance between the first temporal state parameter and the second temporal state parameter as a measure of state synchronization difference between dimensions.

[0141] Read the state synchronization difference measure, the correlation coupling matrix, and the state adjustment amplitude. Multiply the state synchronization difference measure and the correlation coupling matrix values ​​and add the state adjustment amplitude. Calculate the sum as the adaptive convergence constraint threshold. Read each feature component in the first time-series state parameter, calculate the difference between the feature component value and the adaptive convergence constraint threshold, select the feature components whose absolute difference value is less than the threshold as target feature components, and combine them to output the correlation correction feature vector.

[0142] The algorithm reads the accumulated features of the historical template and uses the Kalman filter algorithm to calculate the prior predicted features for the next time step based on these features. It calculates the difference between the prior predicted features and the associated corrected feature vector as the feature prediction error, and calculates the covariance matrix of the feature prediction error as the feature prediction error covariance. It reads the diagonal element values ​​of the feature prediction error covariance, divides the value by 1 by the sum of the diagonal element values ​​and the value 1, and processes this value using the Sigmoid activation function to obtain the adaptive weight adjustment factor. It then uses the adaptive weight adjustment factor to perform weighted residual correction on the associated corrected feature vector, that is, multiplies the adaptive weight adjustment factor by the residual between the prior predicted features and the associated corrected feature vector and adds it to the associated corrected feature vector, outputting the corrected target feature vector.

[0143] Read the corrected target feature vector and feature update coefficients, calculate the product of the corrected target feature vector and feature update coefficients, and perform weighted fusion update with the historical template accumulated features to output the current frame updated template features; read the current frame confidence score, perform weighted fusion update on the current frame confidence score using the feature update coefficients to generate the current frame updated confidence distribution; output the current frame updated template features, the current frame updated confidence distribution, and the updated current frame target bounding box.

[0144] Example 1: To verify the feasibility of this invention for high-precision continuous tracking of dynamic targets in complex video surveillance scenarios, the method of this invention was applied to the urban road violation monitoring system of a smart traffic management platform in a certain city (hereinafter referred to as "Company T"). In traditional traffic monitoring systems, tracking algorithms based on simple correlation filtering or optical flow methods are usually used. These methods are prone to target drift and loss in complex environments such as rapid vehicle movement, severe occlusion, or drastic changes in lighting. Furthermore, it is difficult to accurately recapture and continuously track targets that have been occluded, which seriously affects the accuracy of violation evidence collection and behavior analysis. To solve the above problems, Company T decided to adopt the video target tracking method based on spatiotemporal prediction anti-noise closed loop and feature phase guidance enhancement proposed in this invention.

[0145] During implementation, Company T first used high-definition network cameras deployed on main urban roads and intersections to acquire real-time traffic video streams. After preprocessing operations such as image denoising, temporal jitter correction, and optical distortion correction, a standardized temporal input image sequence was constructed. Simultaneously, Company T's algorithm experts performed precise bounding box annotations and behavioral attribute labeling on key targets such as vehicles and pedestrians in the collected video data, serving as the benchmark for model training and testing.

[0146] Company T constructed a multi-level convolutional network, extracting deep features rich in semantic information and shallow features containing subtle spatial details along the network depth, forming a multi-level initial feature set. Next, the system used pixel-level dense motion vector fields between two frames to construct motion consistency constraint mapping, performing a reverse twisting geometric alignment operation on the deep features to achieve feature alignment in the temporal dimension. Based on this, theoretical feature values ​​were predicted based on feature change trends and compared with actual observations to generate a prediction residual distribution. Then, self-motion noise suppression weights were used to correct the features, effectively filtering out dynamic background noise. Subsequently, a composite dispersion index was calculated by combining local spatial attention weights and inter-channel correlation matrices, performing spatially aware channel filtering and closed-loop calibration decoupling transformation, outputting a high-purity multi-level denoised feature set, laying a solid foundation for subsequent feature enhancement.

[0147] In the feature enhancement and target localization stage, this invention performs target detection on the first frame image to obtain an initial target bounding box and a guiding map. Based on this, it calculates the semantic alignment vector and contrast parameter set of anchor point features and negative sample features, and generates a feature modulation matrix to correct the feature weights. Simultaneously, it performs complex domain mapping on the corrected features to extract the instantaneous phase, constructs a dynamic noise suppression threshold based on the phase consistency metric, and performs phase continuity correction. Combined with temporal aggregated scale dynamic convolution and spatial adaptive suppression factor filtering, it outputs the final enhanced features with extremely strong robustness.

[0148] In the core tracking and response generation stage, this invention performs cross-correlation calculations based on target template features and final enhanced features to generate a preliminary response matrix containing channel and spatial information, and outputs a target perception response map by combining adaptive threshold weights. The response map is then smoothed and sharpened using a topological structure perception filter to obtain a two-dimensional response confidence map. Subsequently, the target's temporal motion velocity vector and dynamic velocity judgment threshold are calculated based on historical location information. When the motion exceeds the limit, a spatiotemporal metric correction factor is used to perform a geometric transformation on the response map, and subpixel-level precision target bounding boxes and confidence scores are obtained through nonlinear resampling interpolation and moment distribution calculation.

[0149] In the final convergence constraint correction stage, this invention constructs a dynamic confidence boundary and deviation distance index based on historical confidence distribution to generate feature update coefficients. By constructing an association coupling matrix and calculating the adaptive convergence constraint threshold using temporal state parameters, association correction feature vectors are selected. Then, using the Kalman filter principle combined with the feature prediction error covariance, an adaptive weight adjustment factor is calculated to perform weighted residual correction on the target feature vector. Finally, the updated template features, confidence distribution, and target bounding box are output, achieving adaptive convergence and accurate correction of the tracking model.

[0150] During implementation, the technical team at Company T discovered that, compared to traditional methods, the method of this invention significantly improves the accuracy and robustness of target tracking in complex traffic scenarios. Traditional methods are prone to losing targets when vehicles change lanes at high speeds, when light changes abruptly upon entering or exiting tunnels, or when the target is obstructed by large vehicles. In contrast, the method of this invention effectively achieves stable tracking and high-precision positioning across all scenarios through spatiotemporal prediction noise-resistant closed loops, phase guidance enhancement, and convergence constraint correction mechanisms.

[0151] Specifically addressing the issues of feature overexposure caused by headlight glare at night and image blurring in rainy or foggy weather, the dynamic noise suppression threshold and phase consistency metric employed in this invention can adaptively eliminate environmental noise interference, ensuring the semantic integrity of target features. Furthermore, when dealing with frequent vehicle occlusion and truncation phenomena in high-density traffic, the system successfully predicts and recovers the target's trajectory during occlusion through topology-based perceptual filtering and adaptive convergence constraint correction, avoiding identity switching and tracking interruptions. These improvements not only significantly reduce the cost of manual review but also provide a solid technical guarantee for the automated violation evidence collection and emergency response of intelligent transportation systems in all weather and road conditions, propelling urban traffic management towards greater refinement and intelligence.

[0152] To further verify the actual performance of the method of the present invention, Company T conducted a detailed comparative test between the method of the present invention and the traditional method. The specific performance data is shown in Table 1:

[0153] Table 1. Comparison of Target Tracking Performance of Company T's City Road Violation Monitoring System

[0154] index Traditional methods Method of the present invention Increase Average accuracy of target tracking (%) 84.3 97.5 +13.2% Track the number of failed restarts (times / hour) 5.6 0.3 -94.6% Average target localization error (pixels) 12.5 2.8 -77.6% Retention rate under severe shading (%) 65.0 92.0 +27.0% Center drift of a fast-moving target (pixels) 15.8 3.5 -77.8% False positive rate for similar target interference (%) 14.2 1.5 -89.4% Light-induced mutation fitness score (1-10) 6.5 9.2 +41.5% Time taken to process a single frame of data (milliseconds) 35 42 +20.0% Long-term tracking robustness index 0.72 0.95 +31.9% Overall system operation and maintenance satisfaction (%) 88.0 98.0 +10.0%

[0155] As shown in Table 1, the performance of the urban road violation monitoring system was comprehensively improved after applying the method of this invention. The average target tracking accuracy increased from 84.3% of the traditional method to 97.5%, and the number of tracking failure restarts decreased significantly from 5.6 times / hour to 0.3 times / hour, significantly enhancing the system's continuous working capability. The average target positioning error decreased from 12.5 pixels to 2.8 pixels, and the tracking retention rate under severe occlusion increased from 65.0% to 92.0%, effectively solving the problem of target loss in complex environments. Although the processing time for a single frame of data increased slightly, it was within an acceptable range, resulting in a significant improvement in robustness and accuracy. Through the method of this invention, Company T successfully achieved precise locking and full-process trajectory recording of vehicles violating traffic regulations, providing reliable data support for traffic law enforcement and significantly improving the intelligence level of smart traffic management.

[0156] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A video object tracking method based on multi-layer attention feature enhancement, characterized in that, Includes the following steps: The current video frame is acquired to extract a multi-level initial feature set, a spatiotemporal prediction anti-noise closed loop is constructed, and a closed loop calibration decoupling transformation is performed to output a multi-level denoised feature set. Get the target bounding box of the previous frame, construct a foreground guidance map based on motion prediction, perform material-aware fusion and topological dynamic evolution, and output a binarized foreground mask; The initial spatial offset is predicted based on a multi-level denoised feature set and a binarized foreground mask. The boundary gradient field constraint and the neighborhood correlation constraint are used for fusion and adaptive correction to output a local enhanced feature set. Based on a multi-level denoising feature set and a local enhancement feature set, spatial-frequency domain collaborative modulation fusion is performed using contrast response and frequency domain filtering guidance to output a multi-level fused feature set. Foreground mask guidance is generated based on multi-layer fusion feature set, feature phase guidance enhancement is performed using multi-source contrast parameters, and the final enhanced feature is output. The target template is constructed by temporal adaptive fusion based on inter-frame features, and cross-correlation operation is performed with the final enhanced features to achieve target localization and scale estimation, and the target bounding box and confidence score of the current frame are output. The update coefficients are calculated based on the target bounding box and confidence score of the current frame. Convergence constraint correction is performed on the features of the current frame, and the template features and confidence distribution are updated to determine the target bounding box of the current frame. 2.The method of claim 1, wherein, The spatiotemporal prediction noise reduction closed loop specifically includes: Perform multi-level convolution calculation and downsampling processing on the current video frame, extract spatial detail features and semantic features at different scales along the feature hierarchy of the multi-level initial feature set, and output the multi-level initial feature set; Based on the current video frame and the previous video frame, a pixel-level dense motion vector field is calculated, and a motion consistency constraint mapping is constructed based on the pixel-level dense motion vector field. The motion consistency constraint mapping is used to perform a reverse twisted geometric alignment operation on the deep features in the multi-level initial feature set. The geometrically aligned deep features are then adaptively fused with the shallow features to output a temporally aligned multi-level feature set. Based on the temporally aligned multi-level feature set, the feature change trend of adjacent pixels in the temporal dimension is calculated, the theoretical feature value at the current moment is predicted based on the feature change trend, and the predicted residual distribution is calculated based on the theoretical feature value and the actual observed value. Based on the predicted residual distribution, self-moving noise suppression weights are generated. The self-moving noise suppression weights are then used to perform feature correction on the temporally aligned multi-level feature set, and a noise suppression feature set is output. Based on the noise suppression feature set, the local spatial attention weight and inter-channel correlation matrix of the feature map are calculated. The composite dispersion index is calculated by combining the mean and variance of the channel response distribution. A dynamic channel filtering threshold is generated based on the composite dispersion index. The dynamic channel filtering threshold is used to perform spatially aware channel filtering on the noise suppression feature set, and the filtered multi-level initial feature set is output. Based on the selected multi-level initial feature set, the semantic similarity matrix and hierarchical expression integrity index between each feature layer are calculated. The inter-level redundancy coefficient is calculated based on the semantic similarity matrix, and the adaptive compensation weight is generated based on the hierarchical expression integrity index. Based on the inter-level redundancy coefficient and the adaptive compensation weight, a closed-loop calibration decoupling transformation is performed on the selected multi-level initial feature set to output a multi-level denoised feature set. 3.The method of claim 2, wherein, The reverse twisted geometric alignment specifically includes: Based on the mapping between deep features in the multi-level initial feature set and the motion consistency constraint, a rule sampling grid coordinate is constructed. Calculate the pixel-level displacement offset vector of the grid coordinate points based on the motion consistency constraint mapping, and use the pixel-level displacement offset vector to perform coordinate transformation on the regular sampling grid coordinates to generate a distorted sampling grid; Bilinear interpolation sampling is performed on the deep features in the multi-level initial feature set using the distorted sampling grid to output geometrically aligned deep features; Calculate the mean channel response distribution of the geometrically aligned deep features, and calculate the mean channel response distribution of the shallow features in the multi-level initial feature set; The channel-level adaptive fusion coefficient is calculated based on the mean channel response distribution of the geometrically aligned deep features and the mean channel response distribution of the shallow features. The channel-level adaptive fusion coefficients are used to perform channel-wise weighted summation on the geometrically aligned deep features and the shallow features, outputting a temporally aligned multi-level feature set.

4. The method of claim 3, wherein, The closed-loop calibration decoupling transformation specifically includes: A hierarchical sparse constraint matrix is ​​constructed based on the inter-level redundancy coefficients, a hierarchical compensation weight matrix is ​​constructed based on the adaptive compensation weights, and a matrix dot product operation is performed between the hierarchical sparse constraint matrix and the hierarchical compensation weight matrix to output the feature hierarchical correlation matrix. Based on the feature hierarchy correlation matrix, calculate the orthogonal projection components between feature subspaces, construct a feature decoupling operator based on the orthogonal projection components, and use the feature decoupling operator to perform a linear transformation on the filtered multi-level initial feature set to output a decoupled feature set. Calculate the reconstruction error distribution of the decoupled feature set, calculate the feature reconstruction confidence based on the reconstruction error distribution, generate closed-loop correction coefficients based on the feature reconstruction confidence, perform residual compensation on the decoupled feature set using the closed-loop correction coefficients, and output a multi-level denoised feature set.

5. The method of claim 4, wherein, The specific features of the phase-guided enhancement include: Target detection is performed on the first frame image to obtain an initial target bounding box, and an initial detection guidance map is generated based on the initial target bounding box and a mask mapping is performed on the first frame image. Based on the initial detection guidance map and the multi-layer fusion feature set, region sampling is performed to extract anchor point features and negative sample features; the first semantic alignment vector between the anchor point features and the first frame target feature map and the second semantic alignment vector between the anchor features and the negative sample features are calculated. Calculate a vector-level first similarity based on the first semantic alignment vector, calculate a vector-level second similarity based on the second semantic alignment vector, and fuse the vector-level first similarity and the vector-level second similarity to generate a set of contrast parameters; Based on the set of contrast parameters, calculate the inter-channel correlation spectrum and the statistical moment distribution of each channel; construct a channel coupling constraint matrix using the inter-channel correlation spectrum; perform nonlinear mapping to generate a feature modulation matrix by combining the channel coupling constraint matrix and the statistical moment distribution of each channel; perform channel-by-channel weighting on the multi-layer fusion feature set based on the feature modulation matrix, and output weighted correction features; Perform complex domain mapping and orthogonal decomposition on the weighted correction features to extract the instantaneous phase of the features; obtain the instantaneous phase of the target corresponding to the target feature map of the first frame, and calculate the phase consistency metric between the instantaneous phase of the features and the instantaneous phase of the target; Multi-scale frequency analysis is performed based on the phase consistency metric and the signal strength at each scale is calculated. A dynamic noise suppression threshold is then constructed based on the signal strength at each scale. The dynamic noise suppression threshold is used to perform continuity correction on the instantaneous phase of the feature to eliminate boundary abrupt changes. An adaptive adjustment weight matrix is ​​constructed based on the corrected instantaneous phase of the feature. The adaptive adjustment weight matrix is ​​used to perform component-level gain and suppression on the weight-corrected feature. The temporal aggregation scale is calculated according to the statistical distribution of the corrected instantaneous phase of the feature to perform dynamic convolution and output temporal synchronization features. Based on the aforementioned temporal synchronization features, the global variance distribution and local gradient structure field within the feature domain are calculated; the global variance distribution and local gradient structure field within the feature domain are fused to construct a spatial adaptive suppression factor; The spatial adaptive suppression factor is used to perform dynamic filtering on the temporal synchronization features, and the final enhanced features are output.

6. The method of claim 5, wherein, The dynamic noise suppression threshold specifically includes: Based on the phase consistency metric, a scale variation parameter is set, a set of spatial convolution kernels is constructed using the scale variation parameter, and the set of spatial convolution kernels is used to perform a convolution operation on the phase consistency metric to extract multi-scale frequency response; The mean energy is calculated based on the multi-scale frequency response as the signal intensity at each scale, and the dynamic noise suppression threshold is calculated based on the normalized distribution of the signal intensity at each scale. The dynamic noise suppression threshold is used to perform soft threshold truncation on the feature instantaneous phase to generate a corrected phase. The cosine similarity of the corrected phase is calculated as the amplitude response index, and an adaptive adjustment weight matrix is ​​generated based on the amplitude response index. The element-wise multiplication operation of the weight correction feature is performed on the adaptively adjusted weight matrix to output the component-level gain suppression feature, and the standard deviation of the correction phase is calculated as the time-domain aggregation scale. The dynamic convolution kernel size is determined using the temporal aggregation scale, and a convolution operation is performed on the component-level gain suppression feature based on the dynamic convolution kernel size to output the temporal synchronization feature.

7. The method of claim 6, wherein, The time-adaptive fusion specifically includes: Input the initial frame target features and the historical template accumulated features, calculate the inter-frame similarity distribution between the initial frame target features and the historical template accumulated features; calculate the feature adaptive update rate based on the inter-frame similarity distribution to perform weighted cumulative fusion on the final enhanced features to generate target template features; Perform channel-dimensional cross-correlation operation on the target template features and the final enhanced features to generate channel response mapping; perform spatial-dimensional cross-correlation operation on the target template features and the final enhanced features to generate spatial response mapping. The channel response map and the spatial response map are fused to output a preliminary response matrix. The channel statistical distribution entropy of the preliminary response matrix is ​​calculated to generate an adaptive threshold weight. The adaptive threshold weight is used to perform channel-wise weighted modulation on the preliminary response matrix to output a target sensing response map. Based on the target sensing response map, the gradient direction dispersion and local energy distribution statistics in the neighborhood are calculated, and a topology sensing filter kernel is constructed based on the gradient direction dispersion and local energy distribution statistics. The topological structure sensing filter kernel is used to perform a convolution operation on the target sensing response map to smooth the response of non-target regions and sharpen the response of target edges, and output a two-dimensional response confidence map; Based on the two-dimensional response confidence map, the initial peak position of the target in the current frame is extracted. The target temporal motion velocity vector is calculated by combining the target position information of historical frames. The temporal position statistical distribution of the target position information of historical frames is calculated to generate a dynamic velocity judgment threshold. The temporal motion velocity vector is compared with the dynamic velocity judgment threshold. If the temporal motion velocity vector exceeds the dynamic velocity determination threshold, a spatiotemporal metric correction factor is constructed based on the motion parameters, and a geometric transformation is performed on the two-dimensional response confidence map to compensate for motion distortion, and a corrected response map is output. Based on the modified response map, non-Euclidean geometric metrics and topological features are calculated, an adaptive sampling matrix is ​​constructed, and nonlinear resampling interpolation is performed to eliminate discrete sampling errors and output a geometrically continuous optimized response map. Based on the geometric continuity optimization response map, the moment distribution characteristics of the response values ​​are calculated to solve the sub-pixel-level center coordinates and geometric scale parameters of the target. The sub-pixel-level center coordinates are mapped to the video frame coordinate system and the target bounding box and confidence score are solved in combination with the geometric scale parameters.