Multi-target tracking method combining dynamic feature optimization and nonlinear modeling

Abstract

The invention belongs to the field of computer vision, and particularly relates to a multi-target tracking method combining dynamic feature optimization and nonlinear modeling. The method comprises the following steps: optimizing a YOLOv11 backbone network by using a gated biaxial fusion unit and an edge sensing convolution operator, and extracting a multi-scale feature map; feature fusion and detection head regression are carried out to obtain a detection frame set; then adaptively extending a Kalman filter to predict a state and carrying out correlation matching; and finally, updating through a filter to realize a complete multi-target tracking task. According to the method, an enhanced multi-scale feature map set is generated for the input image, so that the feature expression capability in a dense shielding and rapid motion scene is remarkably improved; the prediction precision of the nonlinear motion state of the target is optimized, so that the prediction state becomes more reliable; through the synergistic effect of optimized dynamic feature extraction and nonlinear modeling, the robustness and precision of a multi-target tracking task in a dynamic complex scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to a multi-object tracking method combining dynamic feature optimization and nonlinear modeling. BACKGROUND

[0002] Multi-Object Tracking (MOT) is a core task in the field of computer vision, aiming to detect and track multiple objects in a scene in real time while maintaining their identity consistency. In recent years, multi-object tracking has made significant progress in the fields of autonomous driving, video surveillance, and behavior analysis, and is widely used in pedestrian tracking, vehicle monitoring, and other scenarios. At the same time, the research of deep learning and efficient detection algorithms has significantly improved the accuracy and efficiency of multi-object tracking methods.

[0003] However, the existing technology in the multi-object tracking task, especially in dynamic complex scenes such as sports events and dance performances, faces challenges such as dense occlusion, cross-scale deformation, fast motion blur, and complex nonlinear motion. There are generally deficiencies: In the "FairMOT" scheme disclosed in the document "Y. Zhang, C. Wang, X. Wang, et al., "FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking," International Journal of Computer Vision, vol. 129, pp. 3069-3087, 2021. https: / / doi.org / 10.1007 / s11263-021-01513-4", a single lightweight backbone network (DLA-34) is used to construct a detection branch and a Re-ID feature branch on a shared multi-scale feature map, and real-time multi-object tracking is achieved through online bidirectional association (detection-to-track and track-to-detection). The scheme has the following problems: 1. Since a fixed-scale feature extractor is used, it is difficult to adapt to the cross-scale deformation of targets under dynamic camera perspectives; 2. Since the temporal motion information is not fully utilized for association optimization, the tracking robustness is insufficient in the case of target violent motion or severe occlusion.

[0004] In the scheme disclosed in the document "P. Sun, J. Cao, Y. Jiang, et al., "TransTrack: Multiple Object Tracking with Transformer," in Proc. IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 5696-5705", the target embedding of the previous frame is taken as the query, which interacts with the current frame features through multi-head self-attention and cross-attention mechanisms to realize end-to-end joint optimization of detection and tracking. The scheme has the following problems: 1. Since it relies on global attention mechanism to process target features, in dense occlusion scenarios, target overlap causes attention allocation imbalance, which in turn causes false detection rate and missed detection rate to rise; 2. Since the Transformer structure has large computational load and memory overhead, in high frame rate or resource-limited real-time applications, the inference speed is difficult to meet the actual demand.

[0005] In the scheme disclosed in the document "X. Li, Z. Wang, Y. Zhang, et al., "MambaMOT: Multi-Object Tracking with Selective State Space Models," arXiv preprint arXiv: 2403.10826, 2024. https: / / arxiv.org / abs / 2403.10826", by constructing multiple motion and appearance state space models, and dynamically selecting the optimal state according to reliability and matching degree in each frame to filter and update the target, the joint modeling of target motion pattern and appearance change is realized. The scheme has the following problems: 1. Since it relies on pre-defined multiple state space models and switching strategies, it is difficult to adapt quickly when the motion type exceeds the pre-set range or sudden jump motion occurs, which can easily cause tracking drift; 2. Since multiple candidate models need to be evaluated and state switched in parallel in each frame, the computational overhead increases significantly, which is difficult to meet the real-time processing demand in high-density or high-frame-rate scenarios.

[0006] In the scheme disclosed in the document "X. Zhou, Y. Lu, J. Gao, et al., "Graph-based Relational Tracking for Multi-Object Tracking," arXiv preprint arXiv:2303.12345, 2023. https: / / arxiv.org / abs / 2303.12345", the target detection results in each frame are first constructed as graph nodes, the mutual relationship between targets is encoded through edges, then the node features are iteratively updated using a graph neural network, and the target association and track management are completed based on the updated relationship graph. The scheme has the following problems: 1. Since the relationship between targets is modeled by relying on a complex graph structure, multiple rounds of graph convolution and message passing are required during the inference stage, resulting in a significant increase in computational overhead and latency; 2. Since the graph neural network focuses on relationship modeling and does not perform specialized dynamic feature extraction for fast-moving targets, in dynamic complex scenarios such as high speed or sudden motion, tracking accuracy and identity consistency are affected. SUMMARY

[0007] The present application provides a multi-target tracking method combining dynamic feature optimization and nonlinear modeling to overcome the problems in the prior art that in dense occlusion scenarios, target overlap leads to unbalanced attention allocation, false detection rate and missed detection rate increase; the tracker has difficulty maintaining identity consistency; it is difficult to cope with fast motion scenarios, and the prediction error significantly increases in dynamic complex scenarios.

[0008] To achieve the above purpose, the technical scheme of the present application is: a multi-target tracking method combining dynamic feature optimization and nonlinear modeling, comprising the following steps:

[0009] Step 1, feature extraction: using a gate-bi-axial fusion unit (G-BiF) and an edge-aware convolutional operator (EAC) to optimize the YOLOv11 backbone network and extract multi-scale feature maps, the specific steps are as follows:

[0010] Step 1.1, constructing an optimized YOLOv11 backbone network;

[0011] Step 1.2, incorporating the edge-aware convolutional operator into the YOLOv11 backbone network;

[0012] Step 1.3, extracting multi-scale feature maps of the input image through the YOLOv11 backbone network optimized by the gate-bi-axial fusion unit and the edge-aware convolutional operator;

[0013] Step two, feature fusion and detection head regression, get the detection frame set;

[0014] Step three, adaptive extended Kalman filter prediction state and association matching, including the following steps:

[0015] Step 3.1, the detection frame set is divided into low confidence frame and high confidence frame according to the set confidence threshold 0.6;

[0016] Step 3.2, based on constant turning rate and acceleration model, an adaptive extended Kalman filter combining adaptive noise adjustment and dynamic Jacobian matrix update strategy is constructed;

[0017] Step 3.3, associate detection frame with predicted state;

[0018] Step four, through filter update, realize complete multi-target tracking task.

[0019] Further, in the above step 1.1, the optimized YOLOv11 backbone network is embedded in the C3k2 module of YOLOv11 backbone network with the gate dual-axis fusion unit designed by the application, and the gate dual-axis fusion unit contains dual-flow feature gating and dual-axis enhancement module.

[0020] Further, in the above step 1.1, the construction process of the dual-flow feature gating is:

[0021] First, a standard 1x1 convolution is used as:

[0022]

[0023] Wherein, is the convolution kernel parameter, is the convolution operator, is the bias term;

[0024] The bias term is used to absorb the input feature An additional channel fixed at 1 is added, and the bias term b is combined as an additional weight column into the original weight matrix:

[0025]

[0026] At this time, the convolution operation can be represented as matrix multiplication Input feature Linear projection is realized by depth separable convolution and dual-branch 1x1 convolution to expand the channel, and an activation function ReLU6 is introduced in the first branch to constrain the feature range:

[0027]

[0028] where ReLU6(x) = min(max(x, 0), 6);

[0029] Subsequently, element-level multiplication interaction is performed on the channel-by-channel position:

[0030]

[0031] where represents element-wise multiplication, and the interaction term of each channel k is expanded as:

[0032]

[0033] where is the cross-term coefficient;

[0034] Among all the nonlinear terms (excluding linear terms and constant terms), the symmetric terms X (i) X (j) = X (j) X (i) , the effective interaction dimension of the independent channel number is 4C:

[0035]

[0036] Further, in step 1.1 above, the construction process of the two-axis enhancement module is: adjusting the receptive field through two one-dimensional dynamic convolution kernels in series to improve the adaptability of the detector to target scaling under dynamic camera perspective;

[0037] The features obtained after the double-flow feature gating fusion are The global context information is extracted using large kernel average pooling, and the feature map resolution is maintained by setting padding = 3 and stride = 1; then a 1x1 convolution is used to compress the channel dimension, reducing the computational load of subsequent axial convolution while preserving key information:

[0038]

[0039] The one-dimensional dynamic depth separable convolution in series in the horizontal / vertical direction is set with a convolution kernel size K:

[0040]

[0041] where N is a hyperparameter, which is the number of submodules of the C3k2 module, and the default is 1; asymmetric padding is set respectively:

[0042] X horizonta = DWConv 1×K (X AvgPool ), padding = (0, K / / 2) (10)

[0043] X vertical = DWConvK×1 (X horizontal ),padding=(K / / 2,0) (11)

[0044] By 1x1 convolution and Sigmoid activation function, a higher weight is given to the key region of the target:

[0045] X spatial =σ(Conv 1×1 (X vertical )) (12)

[0046] Where σ is the Sigmoid function;

[0047] Again, the spatial information is fused by 1x1 convolution and depth separable convolution, and the original feature details are reserved by combining residual connection to obtain the feature output as follows:

[0048]

[0049] Further, the specific steps of the above step 1.2, using edge-aware convolution operator to integrate YOLOv11 backbone network are:

[0050] First, the feature of the input EAC is executed Direction bias encoding is performed, which shifts the target feature to different directions left, right, up and down before convolution:

[0051]

[0052] Where ZeroPad represents zero padding operation, (k, 0, 1, 0) is the pixel padding number in the left, right, up and down four directions, and k is the convolution kernel size (default is 3);

[0053] Then the horizontal / vertical convolution kernel is constructed to decompose the feature:

[0054]

[0055] After the above feature decomposition, the channel dimension C' and spatial dimension H'xW' of the output feature map are as follows:

[0056]

[0057] Where s represents the step (default is 2);

[0058] Then four groups of features are spliced along the channel dimension, and normalized by 2x2 convolution kernel without padding:

[0059]

[0060] Where Concat(·) represents the splicing operation along the channel dimension, and 4Cin Intermediate features of ×H′×Wv;

[0061] Finally, output features The dimensions are calculated as follows:

[0062]

[0063] Furthermore, in step 3.2 above, the state vector of the constructed adaptive extended Kalman filter is defined as:

[0064] x = [x, y, a, h, v, yaw, yaw] rate ,acc] T (twenty two)

[0065] Where x and y represent the target position, a and h are the target aspect ratio and height, v is the velocity, yaw is the heading angle, yaw_rate is the yaw rate, and acc is the acceleration;

[0066] Furthermore, in step 3.2 above, the adaptive noise adjustment process is as follows:

[0067] In the design of the filter, the target scale h is first used as the reference scale, and the initial standard deviation of the position and size is defined as follows:

[0068] σ x =σ y =ω p h,σ a =0.1,σ h =ω p h (23)

[0069] Where, ω p Scale factor representing the standard deviation of position and size;

[0070] For velocity v, heading angle yaw, yaw rate yaw_rate, and acceleration acc, adjustment factors are introduced based on the magnitudes of acceleration and yaw rate in the current state to obtain the corresponding noise components:

[0071]

[0072] Where, ω v The scaling factor represents the speed uncertainty;

[0073] Finally, multiply all noise by the process noise factor γ. p The process noise matrix is ​​obtained as follows:

[0074]

[0075] The confidence c of the detector output target bounding box is introduced into the adjustment of the measurement noise R, and the basic measurement noise is set as:

[0076] σ x = σ y = ω p h,σ a = 0.15,σ h = ω p h (26)

[0077] At the same time, the effective adjustment coefficient γ e is defined, which adjusts the size of the coefficient depending on the confidence c of the detector output:

[0078] γ e = γ m (1+(1-c)) (27)

[0079] where γ m represents the measurement noise adjustment factor;

[0080] When the confidence c is low, the effective coefficient increases, and finally the measurement noise matrix is obtained:

[0081] R = diag(γ e σ x , γ e σ y , γ e σ a , γ e σ h ) (28).

[0082] Further, in the above step 3.2, the confidence c is in the range [0, 1], and when it is lower than 0.1, it is taken as 0.1.

[0083] Further, in the above step 3.2, the dynamic Jacobian matrix update strategy is:

[0084] First, the state transition function is defined by a constant turning rate and acceleration model:

[0085]

[0086] s represents the distance traveled by the target in the current time step, represents the "effective" heading angle in the current time step;

[0087] The position update is as follows:

[0088]

[0089] The analytical Jacobian matrix F analytical is obtained by analytically deriving the nonlinear mapping f(x, dt).

[0090] Then the numerical Jacobian matrix F is calculated by finite difference method numeric :

[0091]

[0092] where ∈ i Adaptive to the size of state components;

[0093] Then the relative error is calculated by Frobenius norm:

[0094]

[0095] Where, when the system is in the low nonlinear region, w is close to 0, and the analytical Jacobian dominates; while in the state with high nonlinearity, w increases, and the numerical Jacobian will get more weight;

[0096] Finally, the weighted average is used to obtain the dynamic Jacobian matrix:

[0097] F dynamic =(1-w)F analytical +wF numeric (33)。

[0098] Compared with the prior art, the present application has the following advantages:

[0099] 1. In the step of constructing the optimized YOLOv11 backbone network, the feature extraction network is optimized by using a gated dual-axis fusion unit and an edge-aware convolution operator: by embedding the gated dual-axis fusion unit in the C3k2 module of the YOLOv11 backbone network, the feature extraction capability is significantly enhanced. The gated dual-axis fusion unit includes a dual-flow feature gate and a dual-axis enhancement module. The dual-flow feature gate improves the feature space dimension by element-wise multiplication between channels after linear projection by double branches, so that the model can more accurately perceive the target decision boundary and optimize the detection robustness in dense occlusion scenes. The dual-axis enhancement module improves the adaptability of the detector to target scaling under dynamic camera perspective by adjusting the receptive field through two series of one-dimensional dynamic convolution kernels. At the same time, the edge-aware convolution operator is used in the YOLOv11 backbone network, which focuses the convolution kernel on the specific edge region of the target by zero padding in the left, right, top and bottom directions, enhances the perception of the gradient change of the pixels in the edge region, and reduces the boundary loss caused by blur.

[0100] 2. In step three, an adaptive extended Kalman filter is constructed by using a constant turn rate and acceleration model, combined with adaptive noise adjustment and dynamic Jacobian matrix update strategy, which optimizes the modeling ability of nonlinear motion trajectory. Among them, the adaptive noise adjustment part dynamically adjusts the covariance matrix of process noise and measurement noise according to the detection frame confidence, prioritizes high confidence detection results, and reduces the error influence of low confidence detection; the dynamic Jacobian matrix update calculates the Jacobian matrix of the target nonlinear motion in real time, which makes up for the limitations of the analytical Jacobian in complex motion scenes.

[0101] 3. The application embeds the gated dual-axis fusion unit and the edge-aware convolution operator into the YOLOv11 backbone network in step one to generate an enhanced multi-scale feature map set for the input image, which significantly improves the feature expression ability in dense occlusion and fast motion scenes. These high-quality feature maps are further integrated through the neck network feature fusion layer in step two, and the detection frame set is generated through detection head regression, classification and decoding, providing reliable input for subsequent tracking. The detection frame set is then associated with the state predicted by the adaptive extended Kalman filter constructed in step three using the Hungarian algorithm; among them, the adaptive Kalman filter further optimizes the prediction accuracy of the target nonlinear motion state through its adaptive noise adjustment and dynamic Jacobian matrix update strategy, making the predicted state more reliable. After association is completed, the final target trajectory set is output after filter update in step four. In summary, the application significantly improves the robustness and accuracy of multi-target tracking tasks in dynamic complex scenes through the synergistic effect of optimized dynamic feature extraction and nonlinear modeling. BRIEF DESCRIPTION OF DRAWINGS

[0102] Figure 1 The method flowchart of the application is shown in the figure;

[0103] Figure 2 The gated dual-axis fusion unit in the C3K2 module is shown in the figure;

[0104] Figure 3 The edge-aware convolution operator is shown in the figure;

[0105] Figure 4 The tracking process integrated with the adaptive extended Kalman filter is shown in the figure. DETAILED DESCRIPTION

[0106] In order to make the purpose, technical scheme and advantages of the application more clear and explicit, the specific embodiments of the application are further described in detail below with reference to the accompanying drawings.

[0107] A multi-target tracking method combining dynamic feature optimization and nonlinear modeling is described as follows:

[0108] Step one, feature extraction: using a gate-bi-axial fusion unit (G-BiF) and an edge-aware convolutional operator (EAC) to optimize the YOLOv11 backbone network, and extract multi-scale feature maps, the specific steps are as follows:

[0109] Step 1.1 constructs an optimized YOLOv11 backbone network, wherein the optimized YOLOv11 backbone network is a gate-bi-axial fusion unit designed by the application embedded in the C3k2 module of the YOLOv11 backbone network, and the gate-bi-axial fusion unit comprises a double-flow feature gate and a double-axial enhancement module,

[0110] The gate-bi-axial fusion unit designed by the application determines the calling mode according to whether the C3k parameter is true, Figure 2 Respectively show the way of calling the gate-bi-axial fusion unit (G-BiF) in the C3k2 module when the C3k parameter is true and false, and the module details of the gate-bi-axial fusion unit. The specific construction process includes the following two processes carried out simultaneously:

[0111] Among them: the double-flow feature gate improves the spatial dimension of the input feature through the element-by-element multiplication between the channels after the double-branch linear projection, so that the model can more accurately perceive the target decision boundary and optimize the detection robustness of dense occlusion scenes.

[0112] Firstly, the standard 1x1 convolution is:

[0113]

[0114] Among them, is the convolution kernel parameter, is the convolution operator, is the bias term.

[0115] Using the bias term to absorb the input feature An additional channel fixed at 1 is added, and the bias term b is combined as an additional weight column into the original weight matrix:

[0116]

[0117] At this time, the convolution operation can be represented as matrix multiplication

[0118] Input feature Linear projection is realized by expanding the channel through depth separable convolution and double-branch 1x1 convolution, and an activation function ReLU6 is introduced in the first branch to constrain the feature range:

[0119]

[0120] where ReLU6(x) = min(max(x, 0), 6).

[0121] Then element-wise multiplication interaction is performed channel by channel:

[0122]

[0123] where denotes element-wise multiplication, and the interaction term of each channel k is expanded as:

[0124]

[0125] where is the cross-term coefficient.

[0126] Among all the nonlinear terms (excluding linear terms and constant terms), the symmetric terms X (i) X (j) = X (j) X (i) , the effective interaction dimension of the independent channel number 4C is:

[0127]

[0128] Among them: the dual-axis enhancement module adjusts the receptive field through two one-dimensional dynamic convolution kernels in series to improve the adaptability of the detector to target scaling under dynamic camera perspective.

[0129] The features obtained after dual-flow feature gating fusion are The global context information is extracted using large kernel average pooling, and the feature map resolution is maintained by setting padding = 3, stride = 1. Then a 1x1 convolution is used to compress the channel dimension, reducing the computational load of subsequent axial convolution while preserving key information:

[0130]

[0131] One-dimensional dynamic depth separable convolution in series in horizontal / vertical direction is used to dynamically control the receptive field, and the convolution kernel size K is set as:

[0132]

[0133] where N is a hyperparameter, which is the number of submodules of the C3k2 module, and the default is 1. Asymmetric padding is set respectively to maintain the spatial size unchanged to support the residual connection at the end of the module:

[0134] X horizonta = DWConv 1×K (X AvgPool ), padding = (0, K / / 2) (10)

[0135] X vertical = DWConv K×1 (X horizontal ), padding = (K / / 2, 0) (11)

[0136] By 1x1 convolution and Sigmoid activation function, the key area of the target is given higher weight:

[0137] X spatial = sigma(Conv 1×1 (X vertical )) (12)

[0138] Where sigma is the Sigmoid function.

[0139] Again, through the fusion of spatial information by 1x1 convolution and depth separable convolution, combined with residual connection to retain the original feature details, the feature output is as follows:

[0140]

[0141] Step 1.2, edge-aware convolution operator is used to integrate YOLOv11 backbone network, replacing the ordinary convolution in it:

[0142] Through left and right up and down direction zero padding, edge-aware convolution operator makes the convolution kernel focus on the specific edge area of the target, enhances the perception of the gradient change of the edge area pixels, and reduces the boundary loss caused by blur. See Figure 3 After introducing the edge-aware convolution operator to the YOLOv11 backbone network, the process of forward propagation of the original input image is shown, the internal details of the edge-aware convolution operator are also shown, and the growth process of the downsampling multiple of the backbone network from P1 / 2 to P5 / 32 is marked.

[0143] The feature of the input EAC Perform directional bias encoding, shift the target feature to the left, right, up and down in different directions before convolution:

[0144]

[0145] Where ZeroPad represents zero padding operation, (k, 0, 1, 0) is the pixel padding number in the left, right, up and down four directions, and k is the convolution kernel size (default is 3).

[0146] Construct horizontal / vertical convolution kernel decomposition features:

[0147]

[0148] After the above feature decomposition, the channel dimension C' of the output feature map, the spatial dimension H' x W' are as follows:

[0149]

[0150] where s represents the step size (default is 2).

[0151] The four groups of features are spliced along the channel dimension and normalized by a 2x2 convolution kernel without padding:

[0152]

[0153] where Concat(·) represents a splicing operation along the channel dimension, and 4C in x H' x W' intermediate features are generated by splicing.

[0154] Finally, the output feature The size is calculated as follows:

[0155]

[0156] Step 1.3, the YOLOv11 backbone network optimized by the gated dual-axis fusion unit and the edge-aware convolution operator extracts the multi-scale feature map of the input image, and the specific steps in this embodiment are:

[0157] (1) Preprocess and encode the image

[0158] First, perform normalization on the original input image to standardize the pixel value to the [0, 1] interval, eliminating the effects of light and color differences. Adjust the image to a resolution of 640x640 and encode it into a three-dimensional tensor of height, width, and channel number. Through the mapping of pixel values to feature vectors, an initial feature representation is generated for the backbone network to process.

[0159] (2) Extract features and generate multi-scale feature maps

[0160] Input the encoded image tensor into the optimized YOLOv11 backbone network to extract features. The backbone network integrates the gated dual-axis fusion unit and the edge-aware convolution operator, composed of multiple layers of convolution, C3k2, pooling, and attention modules, as shown in Figure 3 . Extract the spatial features of the image layer by layer, output feature maps F1, F2, F3 containing edge, texture, and semantic information at different resolutions at different depth layers (shallow, middle, deep), forming a multi-scale feature map set {F1, F2, F3}.

[0161] Step two, feature fusion and detection head regression to obtain a set of detection boxes, the specific steps include:

[0162] The neck network receives a multi-scale feature map set {F1, F2, F3} and generates a final fusion feature through a multi-scale fusion function Φ:

[0163] F neck =Φ(F1,F2,F3) (20)

[0164] Wherein, Φ is composed of several up-sampling, down-sampling, splicing and C3K2 modules.

[0165] The fusion feature F neck After detection head regression and classification, the detection frame set D t is decoded.

[0166]

[0167] Wherein, Ψ includes convolution prediction, anchor frame decoding and non-maximum suppression steps, (x i ,y i ) is the frame center, a i ,h i is the aspect ratio and height, c i ∈[0,1] is the confidence.

[0168] Step three, the adaptive extended Kalman filter predicts the state and associates the match, specifically including the following steps:

[0169] Step 3.1, the detection frame set is divided into low confidence frame and high confidence frame according to the set confidence threshold 0.6.

[0170] Step 3.2, based on the constant turning rate and acceleration model, an adaptive extended Kalman filter combining adaptive noise adjustment and dynamic Jacobian matrix update strategy is constructed to predict the target state of the current frame according to the historical trajectory state. The state vector is defined as:

[0171] x=[x,y,a,h,v,yaw,yaw rate ,acc] T (22)

[0172] Wherein, x, y represent the target position, a and h are the target aspect ratio and height, v is the speed, yaw represents the heading angle, yaw_rate is the yaw rate, and acc is the acceleration.

[0173] The adaptive extended Kalman filter is constructed, including the following two processes which are carried out synchronously:

[0174] (1) Adaptive noise adjustment

[0175] In the design of filter, the dynamic setting of process noise Q and measurement noise R plays a key role in tracking performance: process noise Q affects the state uncertainty of the target, and measurement noise R determines whether to trust the measurement or rely more on the prediction in association. Generally, the filter often uses fixed noise, and this section combines the motion state and the confidence information of the detector to adaptively adjust the two types of noise.

[0176] To construct the adaptive process noise matrix Q, first take the target scale h as the reference scale, and define the initial standard deviation of position and size as:

[0177] σ x = σ y = ω p h, σ a = 0.1, σ h = ω p h (23)

[0178] where ω p represents the proportional factor of the position and size standard deviation, which is used to calculate the initial standard deviation of the position and related scale in the state, and combined with the target scale h to determine the uncertainty of the initial position.

[0179] For the subsequent motion state, including velocity v, yaw angle yaw, yaw rate yaw_rate and acceleration acc, the adjustment factor is introduced according to the size of the acceleration and yaw rate in the current state, so as to obtain the corresponding noise component:

[0180]

[0181] where ω v represents the proportional factor of the velocity uncertainty.

[0182] Finally, multiply all the noise by the process noise factor γ p , and get the process noise matrix:

[0183]

[0184] In addition to the scale factor of the state itself, this method further introduces the confidence c of the detector output target bounding box into the adjustment of the measurement noise R, where the confidence c ranges from [0, 1], and below 0.1 is taken as 0.1 to prevent the noise from being too large. Set the basic measurement noise as:

[0185] σ x = σ y = ω p h, σ a = 0.15, σ h = ω p h (26)

[0186] At the same time, the definition of the effective adjustment coefficient γ e is proposed, which depends on the confidence c of the detector output:

[0187] γ e = γ m (1+(1-c)) (27)

[0188] where γ m represents the measurement noise adjustment factor.

[0189] When the confidence c is high, the effective coefficient is small; when c is low, the effective coefficient increases, so that the tolerance of the measurement uncertainty during updating is improved. Then the final measurement noise matrix is obtained:

[0190] R = diag(γ e σ x , γ e σ y , γ e σ a , γ e σ h ) (28)

[0191] For the detection confidence fluctuation caused by target occlusion or blur, this strategy can reduce the adverse effects of false measurements on filter updating.

[0192] (2) Dynamic Jacobian matrix updating strategy

[0193] For the constant turn rate and acceleration model with strong nonlinearity, the Jacobian matrix F analytical derived by analytical derivation alone often cannot accurately describe the nonlinear characteristics in state transition. Therefore, a dynamic Jacobian matrix updating strategy is proposed to reduce the error in filter prediction state.

[0194] First, the state transition function is defined by the constant turn rate and acceleration model:

[0195]

[0196] s represents the distance traveled by the target in the current time step, represents the "effective" heading angle in the current time step, which is obtained by adding the angle change in half the time step to the current angle, and it approximately describes the average heading of the target in this time period, which is used to more accurately update the position.

[0197] The position update is as follows:

[0198]

[0199] The analytical Jacobian matrix F is obtained by analytically deriving the nonlinear mapping f(x, dt).analytical .

[0200] Since the analytical method may produce large errors in the high nonlinear region, the method calculates the numerical Jacobian matrix F numeric :

[0201]

[0202] Where ∈ i Adaptively with the size of the state component.

[0203] To judge the difference between the analytical and numerical Jacobian matrices, the relative error is calculated using the Frobenius norm:

[0204]

[0205] Where, when the system is in the low nonlinear region, w is close to 0, and the analytical Jacobian dominates; while in the state with high nonlinearity w increases, and the numerical Jacobian will get more weight. The weighted average is used to obtain the dynamic Jacobian matrix:

[0206] F dynamic =(1-w)F analytical +wF numeric (33)

[0207] This strategy ensures that in the scene where the target nonlinear motion is relatively severe, the filter can use the accuracy of the numerical Jacobian to make up for the limitations of the analytical Jacobian; while in the motion is stable, the system can still maintain efficient calculation.

[0208] Step 3.3, association detection box and predicted state

[0209] As shown in Figure 4 , the adaptive extended Kalman filter first predicts the expected position and velocity state of the target according to the historical trajectory, and then calculates the matching cost of the predicted state and the high confidence box, the low confidence box based on the similarity measure of the intersection over union, and uses the Hungarian algorithm for association matching; Since the association process of high / low confidence boxes is similar, only the association details of high confidence boxes are shown in the figure. If the IoU is less than the preset threshold 0.2, it is not matched; if it is matched, the filter is updated; for the unmatched historical trajectory, it is marked as "lost" state, and 30 frames are kept to enhance the robustness to the occluded target; for the unmatched detection box, if the confidence is higher than the preset threshold 0.6, a new track is created.

[0210] The association matching process generates the correspondence between the detection box and the tracking state, forming a matching pair set.

[0211] Step four, through the filter update, realize the complete multi-target tracking task, the specific steps include:

[0212] Step 4.1, for each matched pair, extract the target center coordinates, aspect ratio, etc. observation information from the corresponding detection box to form a measurement vector For filter update state:

[0213]

[0214] Step 4.2, the filter calculates the Kalman gain based on the difference between the measurement vector and the predicted state, and updates the target's position, speed, etc. state information Generate the updated target state set:

[0215]

[0216] Where, is the Kalman gain matrix, H is the measurement matrix.

[0217] At the same time, update the covariance matrix of the state estimate Optimize the prediction accuracy of the nonlinear motion trajectory:

[0218]

[0219] Where, I is the identity matrix, which keeps the matrix dimensions consistent before and after.

[0220] Step 4.3, according to the updated target state set, connect the continuous state information (including position, speed, ID, etc.) of each target to form a complete tracking trajectory, and output a target trajectory set containing all target historical motion paths:

[0221]

[0222] Where, M represents the total number of valid trajectories at time t+1, and k is the index of a single trajectory.

[0223] The above describes the specific embodiments of the present application in conjunction with the drawings, but is not a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art without creative labor on the basis of the technical solutions of the present application are still within the scope of protection of the present application.

Claims

1. A multi-target tracking method combining dynamic feature optimization and nonlinear modeling, characterized in that, Includes the following steps: Step 1: Feature Extraction: The YOLOv11 backbone network is optimized using a Gate-BiAxial Fusion Unit (G-BiF) and an Edge-Aware Convolutional Operator (EAC), and multi-scale feature maps are extracted. The specific steps are as follows: Step 1.1 Construct an optimized YOLOv11 backbone network; Step 1.2: Integrate edge-aware convolution operators into the YOLOv11 backbone network; Step 1.3: Extract multi-scale feature maps from the input image using the YOLOv11 backbone network optimized by gated dual-axis fusion units and edge-aware convolution operators; Step 2: Feature fusion and detection head regression to obtain a set of detection boxes; Step 3: Adaptive extended Kalman filter predicts states and performs association matching, including the following steps: Step 3.1: Divide the detection box set into low-confidence boxes and high-confidence boxes according to the set confidence threshold of 0.6; Step 3.2: Based on the constant turning rate and acceleration model, construct an adaptive extended Kalman filter that combines adaptive noise adjustment and dynamic Jacobian matrix update strategies; Step 3.3: Associate the detection box with the predicted state; Step 4: Update the filter to achieve the complete multi-target tracking task.

2. The multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 1, characterized in that, In step 1.1, the optimized YOLOv11 backbone network is to embed the gated dual-axis fusion unit designed in this invention into the C3k2 module of the YOLOv11 backbone network. The gated dual-axis fusion unit includes two parts: dual-stream feature gating and dual-axis enhancement module.

3. The multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 2, characterized in that, In step 1.1, the construction process of the dual-stream feature gating is as follows: First, using a standard 1×1 convolution: in, For convolution kernel parameters, For convolution operators, For bias terms; Use bias terms to absorb input features Add a channel fixed at 1, and incorporate the bias term b as an additional weight column into the original weight matrix: At this point, the convolution operation can be represented as matrix multiplication. Input features Linear projection is achieved by expanding the channels using depthwise separable convolutions and dual-branch 1×1 convolutions. The feature range is constrained by introducing the ReLU6 activation function in the first branch. Where ReLU6(x) = min(max(x,0),6); Then, element-wise multiplication is performed on a channel-by-channel basis: Where ⊙ denotes element-wise multiplication, and the interaction term for each channel k expands to: in The coefficient of the cross term; Among all nonlinear terms (excluding linear and constant terms), the symmetric term X is combined. (i) X (j) =X (j) X (i) Then, the effective interaction dimension with 4C independent channels is:

4. The multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 2, characterized in that, In step 1.1, the construction process of the biaxial enhancement module is as follows: the receptive field is adjusted by two cascaded one-dimensional dynamic convolution kernels to improve the detector's adaptability to target scaling under dynamic camera view. The features obtained after dual-stream feature gating fusion are Global context information is extracted using large-kernel average pooling, while maintaining feature map resolution by setting padding=3 and stride=1. This is followed by a 1×1 convolution to compress the channel dimensions, reducing the computational cost of subsequent axial convolutions while preserving key information. One-dimensional dynamic depthwise separable convolutions are concatenated in the horizontal / vertical directions, with the kernel size K set as follows: Where N is a hyperparameter, representing the number of submodules in the C3k2 module, with a default value of 1; asymmetric padding is set as follows: X horizontal =DWConv 1×K (X AvgPool ),padding=(0,K / / 2) (10) X vertical =DWConv K×1 (X horizonta ),padding=(K / / 20) (11) By using 1×1 convolution and the Sigmoid activation function, higher weights are assigned to the key regions of the target: X spatial =σ(Conv 1×1 (X vertical )) (12) Where σ is the Sigmoid function; Spatial information is then fused again using 1×1 convolution and depthwise separable convolution, and residual connections are combined to preserve the original feature details, resulting in the following feature output:

5. A multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 3 or 4, characterized in that, The specific steps of step 1.2, which involves integrating edge-aware convolution operators into the YOLOv11 backbone network, are as follows: First, the features of the input EAC are analyzed. Perform directional bias encoding, shifting the target features to the left, right, up, and down directions before convolution: ZeroPad represents zero-padding operation, (k,0,1,0) is the number of pixels padded in the four directions (left, right, top, and bottom), and k is the kernel size (default is 3). Then construct horizontal / vertical convolution kernels to decompose the features: After the above feature decomposition, the channel dimension C of the output feature map is... ′ Spatial dimension H ′ ×W ′ as follows: Where s represents the step size (default is 2); The four sets of features are then concatenated along the channel dimension and normalized using a 2×2 convolution kernel without padding. Concat(·) represents a concatenation operation along the channel dimension, generating a 4C array through concatenation. in ×H ′ ×W ′ Intermediate features; Finally, output features The dimensions are calculated as follows:

6. The multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 5, characterized in that, In step 3.2, the state vector of the constructed adaptive extended Kalman filter is defined as: x=[x,y,a,h,v,yaw,yaw rate ,acc] T (22) Where x and y represent the target position, a and h are the target aspect ratio and height, v is the velocity, yaw is the heading angle, yaw_rate is the yaw rate, and acc is the acceleration.

7. A multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 6, characterized in that, In step 3.2, the adaptive noise adjustment process is as follows: In the design of the filter, the target scale h is first used as the reference scale, and the initial standard deviation of the position and size is defined as follows: σ x =σ y =ω p h,σ a =0.1,σ h =ω p h (23) where ω p Scale factor representing the standard deviation of position and size; For velocity v, heading angle yaw, yaw rate yaw_rate, and acceleration acc, adjustment factors are introduced based on the magnitudes of acceleration and yaw rate in the current state to obtain the corresponding noise components: Where, ω v The scaling factor represents the speed uncertainty; Finally, multiply all noise by the process noise factor γ. p The process noise matrix is ​​obtained as follows: The confidence level c of the target bounding box output by the detector is incorporated into the adjustment of the measurement noise R. The base measurement noise is set as follows: s x =s y =ω p h,s a =0.15,σ h =ω p h (26) At the same time, the definition of the effective adjustment coefficient γ is proposed. e The magnitude of the confidence level c depends on the detector output. γ e =γ m (1+(1-c)) (27) where, γ m Indicates the measurement noise adjustment factor; When the confidence level c is low, the effective coefficients increase, ultimately yielding the measurement noise matrix: R=diag(γ e s x ,c e s y ,c e s a ,c e s h ) (28).

8. A multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 7, characterized in that, In step 3.2, the confidence level c ranges from [0,1], and is set to 0.1 when it is lower than 0.

1.

9. A multi-target tracking method combining dynamic feature optimization and nonlinear modeling according to claim 8, characterized in that, In step 3.2, the dynamic Jacobian matrix update strategy is: First, the state transition function is defined using a constant turning rate and acceleration model: s represents the distance traveled by the target in the current time step. Indicates the "valid" heading angle within the current time step; The position is updated as follows: The analytic Jacobian matrix F can be obtained by analytically differentiating this nonlinear mapping through f(x,dt). analytical ; Then, the numerical Jacobian matrix F is calculated using the finite difference method. numeric : Where ∈ i Adapts to the magnitude of the state components; The relative error was then calculated using the Frobenius norm: When the system is in the low nonlinear region, w is close to 0, and the analytical Jacobian dominates; while when the nonlinearity is high, w increases, and the numerical Jacobian will gain more weight. Finally, a weighted average is used to obtain the dynamic Jacobian matrix: F dynamic =(1-w)F analytical +wF numeric (33)。

Images