A visual target tracking method based on flow matching

Abstract

This invention discloses a visual target tracking method based on flow matching, belonging to the field of computer vision and intelligent information processing technology. The method first generates normalized real target bounding box states through data preprocessing, and then generates noisy bounding box states by sampling according to a prior distribution in a continuous state space. Subsequently, a conditional linear probability path and a conditional boundary vector field connecting the two are constructed, and a velocity field prediction network is used to approximate the real conditional velocity field. The flow matching tracking model is trained by constructing training samples using time variables uniformly sampled within (0,1), completing regression constraint optimization. In the inference stage, the continuous evolution and high-precision approximation of the target state are achieved through numerically solving ordinary differential equations. This invention explicitly models the evolution law of the tracking state, balancing fitting accuracy and operational efficiency.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent information processing technology, specifically to a visual target tracking method based on stream matching. Background Technology

[0002] Visual object tracking (VOT) is a fundamental problem in computer vision. Its goal is to continuously predict the spatial location of an object in subsequent frames, given the object's location in the initial frame of a video sequence.

[0003] Existing tracking methods can be mainly divided into three categories: matching methods based on Siamese networks, integrated modeling methods based on Transformers, and tracking methods based on generative models. Among them, Siamese network methods usually achieve target localization through feature matching between templates and search regions, but their adaptability to appearance changes in complex scenes is limited; Transformer methods enhance the modeling ability between the target and the context through a global attention mechanism, but most methods still use a single forward propagation for bounding box regression, lacking explicit modeling of the dynamic evolution process of the target state; Diffusion-based methods that have emerged in recent years model target localization as a random generation process, which improves robustness to some extent, but they rely on multi-step random sampling processes, have low inference efficiency, and lack deterministic modeling mechanisms.

[0004] In summary, existing technologies generally treat target tracking as a discrete prediction problem, lacking the ability to model the continuous evolution of the target state from prior estimate to its true position. This makes it difficult to accurately characterize the uncertainty and dynamic changes in target motion. Therefore, there is an urgent need for a tracking method that can explicitly model the continuous evolution of the target state while balancing computational efficiency and prediction accuracy. Summary of the Invention

[0005] The purpose of this invention is to address the common problems in existing target tracking methods, such as single-step regression prediction, lack of continuous state modeling, and insufficient adaptability to complex scenarios, by providing a tracking method based on a flow matching mechanism. This method constructs a conditional velocity field and introduces ordinary differential equations to continuously model the evolution of the target state, achieving a gradual approximation from the initial prior state to the target's true position. This effectively improves target positioning accuracy and tracking robustness while also considering computational efficiency.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A visual target tracking method based on flow matching, comprising the following steps:

[0008] S1. Obtain the original dataset containing consecutive video frames and corresponding target bounding box annotations. For any video sequence in the dataset, define its starting frame as the template frame, and use the center of the target bounding box in that frame as the anchor point. Local region cropping of pixels yields an image called the template image. In subsequent temporal tracking, the current video frame is defined as the search frame, and the tracking and localization result of the previous frame is used as the center for further processing. The image obtained by cropping the context region of a pixel is called the search image;

[0009] S2. Perform data preprocessing on the template image and search image generated in step S1 to obtain standardized template image and search image, and obtain normalized real target bounding box state by mapping the original data of the target bounding box annotation.

[0010] S3. Combine the standardized template image generated in step S2 with the search image through cross-image joint serialization, visual Transformer global self-attention interaction, target region feature alignment, and spatial multi-scale pooling operations to obtain a feature space representation that integrates target appearance representation, local region information, and global context information.

[0011] S4. Based on the normalized true target bounding box state obtained in step S2, random sampling is performed in the state space to obtain the noisy bounding box state. By constructing a conditional linear probability path, explicitly defining the conditional boundary vector field, and using a velocity field prediction network to perform conditional velocity field approximation, a flow matching tracking prediction model that satisfies continuous state evolution is obtained.

[0012] S5. Based on the feature space representation generated in step S3, combine the current time step t and the current target state. A velocity field prediction network is constructed through high-dimensional temporal embedding, linear mapping of state features, deep cross-modal interaction via multi-layer self-attention mechanism, and residual prediction operation of multi-layer perceptron. The conditional velocity field predicted by the velocity field prediction network at the current evolution time step is obtained. ;

[0013] S6. Based on the predicted conditional velocity field output in real time by the velocity field prediction network described in step S5, and the normalized real target bounding box state obtained in step S2, the flow matching tracking model parameters that achieve optimal conditional velocity field approximation in the full time step and full sample space are obtained by calculating the loss, performing backpropagation, and using the optimizer to update the network weight parameters.

[0014] Beneficial effects

[0015] Compared with the prior art, the present invention adopts the above technical solution and has the following effects:

[0016] 1. By explicitly modeling the target state evolution process, this invention effectively overcomes the trajectory drift problem caused by single-step discrete regression in traditional algorithms. This invention introduces a flow matching mechanism to transform the image target tracking problem into a deterministic conditional velocity field regression problem. During the inference phase, numerical solutions to ordinary differential equations allow the target bounding box state to continuously evolve and gradually approximate along a defined conditional velocity field. This continuous-time evolution mechanism endows the model with strong self-correction capabilities, enabling dynamic correction of prediction biases even when the target undergoes appearance deformation, severe occlusion, or is subject to background semantic interference.

[0017] 2. Multi-vector fully interactive network architecture enhances the collaborative discrimination capability of multi-source information. This invention designs a velocity field prediction network that aligns with the updated graph, uniformly and normally mapping bounding box state embedding, time series encoding, instance region features, and global contextual information into a vector sequence, and achieves deep fusion of multi-source information through a multi-layer self-attention mechanism. This design enables the velocity field prediction network to dynamically and adaptively adjust the attention focus area according to the specific time process of the current evolution and the immediate state feedback, ensuring the accuracy and robustness of the continuous time evolution trajectory.

[0018] 3. Standardized training and inference enable high-precision deterministic supervision constraints. During model training, the real labeled data in the dataset is designated as the final state x1, and the ideal conditional velocity field (x1-x0) determined by it and the starting state x0 is directly used as the supervision label to constrain the predicted conditional velocity field output by the velocity field prediction network. This deterministic supervision mechanism avoids the uncertainty caused by random noise in traditional generative tracking models, making it more suitable for visual target tracking tasks.

[0019] 4. Balancing Fitting Accuracy and Online Tracking Efficiency. This invention employs a deterministic ordinary differential equation integration solution scheme, which significantly reduces the number of iterations in the inference phase compared to traditional diffusion model tracking algorithms that rely on multi-step random sampling. Experimental data show that this invention achieves an average overlap rate of 72.4% and an SR of 81.8% on the GOT-10k dataset. 0.5 The success rate surpasses mainstream comparison methods; at the same time, the online inference speed of this invention reaches 56 FPS, ensuring excellent target bounding box fitting accuracy while taking into account real-time running efficiency, meeting the deployment requirements of industrial-grade online application scenarios. Attached Figure Description

[0020] Figure 1 This is a diagram of the FMTrack model architecture used in this application;

[0021] Figure 2 A diagram of the velocity field network structure in the invention;

[0022] Figure 3 To develop an overall flowchart. Detailed Implementation

[0023] The technical solution of the present invention will be described in detail below.

[0024] The following is only one embodiment of the present invention. The present invention has many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention. All such corresponding changes and modifications should fall within the protection scope of the appended claims.

[0025] The present invention provides a visual target tracking method based on flow matching, the method comprising the following steps:

[0026] S1. Data collection and organization.

[0027] This step inputs a publicly available visual target tracking dataset. Through filtering, cleaning, and other operations, it obtains raw data containing continuous video frames and corresponding high-precision target bounding box annotations.

[0028] First, obtain publicly available single-object tracking datasets for model training and testing. These datasets specifically include GOT-10k, LaSOT, and LaSOT. ext Large-scale datasets. These datasets cover a massive number of video sequences, each consisting of temporally continuous frame images and containing corresponding frame-by-frame bounding box annotations.

[0029] Specifically, the GOT-10k dataset contains diverse motion patterns and over 560 general object classes, and the training and test sets have strictly mutually exclusive categories, making it suitable for evaluating the generalization ability of models; the LaSOT dataset is a long video tracking dataset with an average sequence length exceeding 2500 frames, containing rich scenes of target occlusion, out-of-bounds movement, and complex background interference; LaSOT ext The dataset is an extended version of LaSOT, introducing more long-period unknown object categories and more challenging motion deformations to further test the model's tracking robustness in extreme scenarios. After collecting the dataset, any abnormal annotations, damaged images, or invalid frames were detected and cleaned. Samples with missing annotations due to video editing or targets completely outside the image range were corrected or removed to ensure high fidelity and continuity of the data input to the network.

[0030] For any video sequence in the dataset, its starting frame is defined as the template frame, and the center of the target bounding box in that frame is used as the anchor point for... Local region cropping of pixels yields an image called the template image. In subsequent temporal tracking, the current video frame is defined as the search frame, and the tracking and localization result of the previous frame is used as the center for further processing. The image obtained by cropping the context region of pixels is called the search image.

[0031] S2. Data preprocessing and dataset partitioning.

[0032] This step performs data preprocessing on the template image and search image generated in the previous step to obtain standardized template image and search image. It also obtains normalized real target bounding box states by mapping the original data of the target bounding box annotation and completes the division of the dataset.

[0033] Specifically, the template image and search image obtained from the previous cropping step undergo standardization preprocessing to map the image pixel values ​​to a uniform numerical range. Using bilinear interpolation and other scaling algorithms, the original cropped images at different resolutions are scaled to the specified pixel range. The preprocessed template image is used to highlight the core appearance area of ​​the target, while the search image retains a wider range of contextual information.

[0034] Secondly, the true bounding box of the target in the search image is obtained, and it is converted from the original absolute pixel coordinates of the top left corner in the dataset into a mathematical representation of the center point coordinates and width and height, i.e., [c x , c y [, w, h]. Subsequently, based on the overall resolution size of the search image, the four-dimensional state vector is normalized to map it to the standard continuous real number space of [0, 1], thereby generating the normalized true target bounding box state.

[0035] In this embodiment, all sample data are divided into training set, validation set and test set in a ratio of 7:2:1. The training set is used for model parameter learning, the validation set is used to evaluate model performance and adjust parameters during training, and the test set is used to verify the effect after model training is completed.

[0036] S3. Image feature extraction.

[0037] This step uses the standardized template image and search image generated in the previous step as input. Through cross-image joint serialization, visual Transformer global self-attention interaction, target region feature alignment, and spatial multi-scale pooling operations, a feature space representation that integrates target appearance representation, local region information, and global context information is obtained.

[0038] To fully extract the target's appearance information and its contextual relationship with the background, a feature encoding method based on visual Transformer is used to uniformly model the template image and the search image. Specifically, the preprocessed template image and search image are first divided into fixed-size image blocks, and template vector sequences and search vector sequences are obtained respectively through a linear mapping layer. Then, the template vector sequence and search vector sequence are concatenated, and positional encoding is introduced to preserve spatial location information. in, This represents the vector sequence obtained after concatenation. Represents a sequence of template vectors. Represents the search vector sequence. and These represent the number of image blocks after the template image and the search image are divided, respectively. This represents positional encoding. The concatenated vector sequence is input into a parameter-shared visual Transformer encoder for feature extraction. A multi-layer self-attention mechanism enables global information interaction between the target and background regions, yielding template features and search features. in and These represent template features and search features, respectively. This represents the dimension of the feature vector for each image patch. This represents the visual Transformer computation. Further, the corresponding search features are extracted from the encoder output. And rearrange them into feature maps, represented as: in and These represent the spatial dimensions of the feature map, This represents the feature map obtained after dimensionality reconstruction. Indicates dimensional reconstruction. Based on the current target state. The coordinates of the original image are mapped to the coordinate system of the feature map to obtain the corresponding region. Within this region, the aligned local feature representation is obtained through an alignment operation. in This represents the aligned local feature representation. This indicates the Region of Interest Align operation. This represents the target state at time step t, i.e., the current target state. Further, the aligned local features are spatially aggregated to obtain instance features. : These represent the fixed spatial height resolution and spatial width resolution of the local feature representation after region feature alignment, respectively. These represent the spatial arrangement indices of the aligned local features in the height and width dimensions, respectively. This represents the local feature vector of the target region alignment feature map at coordinates (i, j), which contains the depth semantics and target geometric information at that local spatial location.

[0039] At the same time, the feature map after dimensional reconstruction Aggregation operations are performed to generate global features, which are used to provide overall scene information.

[0040] in This represents the local feature vector of the reconstructed feature map at coordinates (i, j). This represents global features. Based on this, instance features and global features are mapped to a unified feature space through linear projection or feature transformation. in and Let these represent the learnable weight matrix and bias term, respectively. Denotes the uniform spatial features obtained after linear projection. Representing instance features or global features .

[0041] Furthermore, a gating fusion mechanism is employed to analyze instance features. With global features The feature extraction and fusion process is performed. The instance features and global features are concatenated along the channel dimension, and dynamic gating weights are calculated using a linear mapping and a Sigmoid activation function. in, This indicates a concatenation operation along the channel dimension. and Let represent the learnable mapping matrix and bias term of the gating weights, respectively. This represents the Sigmoid activation function. Indicates dynamic gating weights;

[0042] Subsequently, the dynamic gating weights are used The instance features and global features are weighted and fused to calculate the fused visual features. : in, This represents the element-wise multiplication operation of the feature vector; through the adaptive gating fusion mechanism, an adaptive dynamic balance between instance features and global features is achieved.

[0043] Through the above feature encoding and representation learning process, the effective fusion of target appearance features, local region information, and global context information is achieved.

[0044] S4. Flow matching modeling.

[0045] This step is based on the normalized real target bounding box state. Random sampling is performed in the state space to obtain the noisy bounding box state. By constructing a conditional linear probability path, explicitly defining the conditional boundary vector field, and using a velocity field prediction network to perform conditional velocity field approximation operations, a flow matching tracking prediction model that satisfies continuous state evolution is obtained.

[0046] In this embodiment, to model the continuous change process of the target state from the initial estimate to the true position, flow matching theory is introduced to construct a probabilistic path. Let the target's final state in the current frame be... The starting state is , where the state vector This is expressed as the center coordinates and width and height parameters of the target bounding box, i.e. in and represents the center coordinates of the target bounding box, and w and h represent the width and height parameters of the target bounding box, respectively.

[0047] Define a time variable t∈[0,1] to characterize the continuous evolution of the target state, and construct intermediate state samples from the starting state to the ending state using linear interpolation, denoted as: This represents the intermediate sample obtained through linear interpolation, i.e., the state of the randomly sampled noisy bounding box. Let represent the conditional probability path function determined by the starting state x0 and the ending state x1. This expression describes the continuous change path of the target state along the time dimension. During the construction of model training samples, to ensure the numerical stability of boundary calculations and avoid endpoint gradient singularities, t is uniformly randomly sampled within the open interval (0,1) defined by the starting and ending states. Based on this, the corresponding ideal conditional velocity field can be expressed as: This represents the difference between the endpoint state and the starting state, i.e., the ideal speed that the model needs to predict. The direction and magnitude of the change in the target state along this path remain constant.

[0048] Furthermore, to achieve the learning of this velocity field, a parameterized neural network is introduced. As a velocity field prediction network, in which This represents the fusion of visual features. The model is optimized by minimizing the difference between the predicted and actual speeds; its training objective function is... To optimize the model by minimizing the difference between the predicted velocity and the actual velocity, it is represented as follows: .

[0049] During the model training phase, in order to introduce known supervision information to impose deterministic constraints on the model, the real labeled data in the dataset is designated as the endpoint state x1, and the ideal conditional velocity field determined by the endpoint state x1 and the starting state x0 is defined. As a true supervisory label, the calculated loss error is backpropagated by measuring the regression residual between the network's predicted velocity field and the true supervisory label to optimize the network parameters. In this way, the tracking problem is transformed into a conditional velocity field regression problem, enabling the modeling of the continuous evolution of the target state.

[0050] During the online inference phase, x0 and x1 represent the starting and ending states of the inference process, respectively. The change process of the target state is described by the following ordinary differential equation: By numerically integrating this equation, we can start from the initial state. The final state is derived. This allows for the completion of target positioning.

[0051] S5. Construction of the velocity field prediction network.

[0052] This step is based on the feature space representation generated in step 3, combined with the current time step t and the current transition target bounding box state x. tA velocity field prediction network is constructed through high-dimensional temporal embedding, linear mapping of state features, deep cross-modal interaction via multi-layer self-attention mechanism, and residual prediction operation of multi-layer perceptron. The conditional velocity field predicted by the velocity field prediction network at the current evolution time step is obtained. .

[0053] To accurately predict the velocity field under the aforementioned conditions, a velocity field prediction network based on a multi-vector interaction mechanism needs to be constructed. This network takes the current target state as its basis. Time step coding Using visual features as input, a self-attention mechanism is used to achieve multi-source information fusion modeling.

[0054] Specifically, firstly, regarding the time variable... Encode the target state to obtain a temporal embedding vector and then encode the target state. Mapped to state feature representation: , in and For learnable parameters, This indicates the time step encoding. This represents the time embedding vector encoded from the time variable t. This represents the state feature representation encoded from the current target state. Further, based on the current target state, the search feature map is used to... Extract the corresponding instance features And it is combined with global features through gating fusion. Fusion as a fusion visual feature Construct a multi-source interaction vector set : The aforementioned vector set is then input into a multi-layer self-attention network for feature interaction, and the attention calculation process is represented as follows: in These are the query, key, and value matrices obtained by linear transformation of the input vector, respectively. The feature dimension is used. After stacking multiple layers, the fused feature representation is obtained. .

[0055] Finally, the velocity field prediction results are obtained through linear mapping. in and These are the parameters of the multilayer perceptron. Output. Indicates the current state and time step The direction and magnitude of the change in the target state.

[0056] Through the above structure, the velocity field prediction network can fully integrate target state information, temporal dynamic information, and visual context information to achieve accurate modeling of the continuous motion process of the target, thereby improving the stability and accuracy of tracking.

[0057] S6. Model Evaluation.

[0058] In this embodiment, in order to comprehensively evaluate the performance of the proposed method, a variety of standard evaluation indicators are used to quantitatively analyze the model, mainly including indicators such as average overlap rate, success rate, area under the success curve, and accuracy.

[0059] First, the average overlap (AO) measures the overall overlap between the predicted bounding box and the ground truth bounding box, and it is defined based on the intersection over union (IoU) ratio: in, Indicates the predicted bounding box. Represents the true bounding box. The first metric represents the total number of test frames. AO reflects the model's average localization accuracy across the entire sequence. Secondly, the success rate (SR) is used to evaluate the proportion of successful tracking under different overlap thresholds. It is defined as the proportion of frames satisfying IoU > τ out of the total number of frames, where τ is the set IoU threshold, such as 0.5 or 0.75. This metric is usually denoted as SR. 0.5 and SR 0.75 .

[0060] To more comprehensively reflect the performance changes of the model under different threshold conditions, the Area Under the Curve (AUC) is introduced. Specifically, under a series of IoU thresholds τ∈ The success rate is calculated, and a curve showing the success rate as a function of a threshold is plotted. The AUC is defined as follows: AUC can comprehensively reflect the overall performance of the model under different overlap requirements.

[0061] Furthermore, precision (P) measures the accuracy of predicting the center position of a target, and is typically calculated based on the center error, which is defined as: Given a distance threshold Below, precision is defined as satisfying The proportion of frames in the total number of frames is used, with a common threshold being 20 pixels. Additionally, to eliminate the impact of different resolutions, normalized precision (Pnorm) can be used, which involves normalizing the center error before statistical analysis.

[0062] The above multi-dimensional evaluation indicators can be used to comprehensively evaluate the model performance from multiple aspects such as overlap accuracy, success rate distribution, and center positioning error, thereby objectively reflecting the effectiveness and superiority of the present invention in tracking tasks.

[0063] S7. Model tuning.

[0064] This step takes as input the predicted conditional velocity field output in real time by the velocity field prediction network described in step 5, and the normalized real target bounding box state. By calculating the loss, performing backpropagation, and using the optimizer to update the network weight parameters, the flow matching tracking model parameters that achieve optimal conditional velocity field approximation across the entire time step and the entire sample space are obtained.

[0065] During model optimization, gradient descent-based optimizers such as Adam or AdamW are used to optimize network parameters. The model is updated iteratively by calculating the gradient of the loss function with respect to the parameters through backpropagation. During training, hyperparameters such as batch size, learning rate, and number of training epochs are set, and a learning rate decay strategy is used to improve the model's convergence performance.

[0066] Furthermore, to prevent overfitting, a validation set is introduced during training to monitor model performance. Training can be stopped early when the validation error no longer decreases. Regularization methods such as weight decay and dropout can be used to improve the model's generalization ability. After each training round, the model is evaluated using a test set, and performance metrics are calculated to measure its effectiveness. Based on this, the model structure or training parameters are adjusted and optimized.

[0067] Through the above training process, the velocity field prediction network can accurately depict the continuous evolution of the target state from the initial estimate to the true position, thereby ensuring the accurate updating of the target state during the inference stage.

[0068] S8. Effect Verification.

[0069] To verify the actual technical effect, industrial feasibility, and technical progress of the base method described in this invention compared with the prior art, this embodiment constructs a standard deep learning experimental testing system and conducts comprehensive quantitative performance evaluation and core mechanism ablation effect experiments on multiple international authoritative public benchmark datasets.

[0070] The specific hardware and software computing environment and basic hyperparameters of the tracking model described in this embodiment of the invention were standardized, resulting in a highly repeatable experimental benchmark system. The specific configuration and training parameters are specified as follows:

[0071] The experimental computing cluster is equipped with two NVIDIA RTX 3090 graphics processors, each with 24 GB of video memory. The algorithm is implemented based on the deep learning framework PyTorch. The backbone network of the feature encoder adopts a visual Transformer base model (ViT-Base) pre-trained on a masked autoencoder (MAE), and the channel dimension of the latent features within the network is uniformly aligned to 256.

[0072] In the decoding network module parameters, the conditional velocity field decoder consists of four stacked bidirectional self-attention layers, configured with eight attention heads, and the hidden feature dimension of its internal forward propagation network is configured to 1024. When numerically solving the ordinary differential equation to update the state, the lower bound constant used for bounding box scale normalization truncation is configured to 0.05 to prevent bounding box size degradation.

[0073] The model ran for a total of 250 iterations, with each iteration containing 60,000 randomly sampled temporal training image pairs and a batch size of 32. A stepping scheduler was used to reduce the learning rate to 1 / 10 of its original value in the 200th iteration.

[0074] By introducing the internationally recognized authoritative target tracking test sets GOT-10K and LASOT, matrix-cascaded quantization comparison was performed on the core indicators of the method of this invention and several existing mainstream tracking algorithms, and the comparison results that objectively demonstrate significant technological progress are shown below.

[0075] Table 1 Comparison of experimental results:

[0076] A comprehensive comparative analysis of the experimental data in Table 1 shows that on the highly challenging GOT-10k dataset, the method of this invention achieved an average overlap rate of 72.4% and an SR0.5 success rate of 81.8%, surpassing all mainstream comparison methods in the table. In the more demanding LaSOT long-term tracking dataset, this invention also achieved superior performance in AUC and accuracy P. From a global technical perspective, this invention alleviates the technical shortcomings of traditional single-stage regression algorithms in occlusion and deformation scenarios by transforming the tracking task into a deterministic continuous evolution process of the conditional velocity field. Furthermore, while maintaining the aforementioned leading localization accuracy, the online inference speed of this invention reached 56 FPS. The experimental results objectively demonstrate that this invention achieves significant technical progress by ensuring both bounding box fitting accuracy and online operating efficiency.

Claims

1. A method for visual object tracking based on flow matching, the method comprising: The method includes the following steps: S1. Obtain the original dataset containing consecutive video frames and corresponding target bounding box annotations. For any video sequence in the dataset, define its starting frame as the template frame, and use the center of the target bounding box in that frame as the anchor point. Local region cropping of pixels yields an image called the template image. In subsequent temporal tracking, the current video frame is defined as the search frame, and the tracking and localization result of the previous frame is used as the center for further processing. The image obtained by cropping the context region of a pixel is called the search image; S2. Perform data preprocessing on the template image and search image generated in step S1 to obtain standardized template image and search image, and obtain normalized real target bounding box state by mapping the original data of the target bounding box annotation. S3. Combine the standardized template image generated in step S2 with the search image through cross-image joint serialization, visual Transformer global self-attention interaction, target region feature alignment, and spatial multi-scale pooling operations to obtain a feature space representation that integrates target appearance representation, local region information, and global context information. S4. Based on the normalized true target bounding box state obtained in step S2, random sampling is performed in the state space to obtain the noisy bounding box state. By constructing a conditional linear probability path, explicitly defining the conditional boundary vector field, and using a velocity field prediction network to perform conditional velocity field approximation, a flow matching tracking prediction model that satisfies continuous state evolution is obtained. S5. Based on the feature space representation generated in step S3, combine the current time step t and the current target state. A velocity field prediction network is constructed through high-dimensional temporal embedding, linear mapping of state features, deep cross-modal interaction via multi-layer self-attention mechanism, and residual prediction operation of multi-layer perceptron. The conditional velocity field predicted by the velocity field prediction network at the current evolution time step is obtained. ; S6. Based on the predicted conditional velocity field output in real time by the velocity field prediction network described in step S5, and the normalized real target bounding box state obtained in step S2, the flow matching tracking model parameters that achieve optimal conditional velocity field approximation in the full time step and full sample space are obtained by calculating the loss, performing backpropagation, and using the optimizer to update the network weight parameters.

2. The visual target tracking method based on flow matching according to claim 1, characterized in that, The specific method for step S2 is as follows: First, the template image and the search image obtained from cropping in step S1 are standardized and preprocessed to map the image pixel values ​​to a uniform numerical range; then, the original cropped images of different resolutions are scaled to the specified pixel range respectively. Secondly, the true bounding box of the target in the search image is obtained and converted from the original absolute pixel coordinates of the top left corner in the dataset into a four-dimensional state vector containing the center point coordinates and the width and height, i.e., [c x , c y [, w, h], where (c x , c y ) represents the center point coordinates of the search image, w represents the width of the search image, and h represents the height of the search image; then, according to the overall resolution size of the search image, the four-dimensional state vector is normalized to map it to the standard continuous real number space of [0, 1], thereby generating the normalized real target bounding box state.

3. The method of claim 2, wherein, Step S3 specifically includes the following sub-steps: S3-1. Divide the standardized template image and search image generated in step S2 into fixed-size image blocks, and obtain template vector sequences and search vector sequences respectively through a linear mapping layer; then concatenate the template vector sequences and search vector sequences, and introduce position encoding to preserve spatial location information: in, This represents the vector sequence obtained after concatenation. Represents a sequence of template vectors. Represents the search vector sequence. and These represent the number of image blocks after the template image and the search image are divided, respectively. Indicates positional encoding; S3-2. Input the concatenated vector sequence into the parameter-shared visual Transformer encoder for feature extraction. A multi-layer self-attention mechanism is used to achieve global information interaction between the target region and the background region, resulting in template features and search features. in and These represent template features and search features, respectively. This represents the dimension of the feature vector for each image patch. This represents visual Transformer computation; S3-3. Separate search features from the output of the parameter-shared visual Transformer encoder and rearrange them into a feature map for instance region feature extraction, denoted as: wherein with respectively represent the spatial dimensions of the feature map obtained after dimension reconstruction, represents the feature map obtained after dimension reconstruction, represents dimension reconstruction; S3-4., the current target state From the original map coordinates to the feature map coordinate system, the corresponding area range is obtained, and the aligned local feature representation is obtained after the alignment operation in the area: in This represents the aligned local feature representation. This indicates the Region of Interest Align operation. This represents the target state at time step t, i.e., the current target state. S3-5. Spatial aggregation of the aligned local feature representations to obtain instance features. : respectively represent the fixed spatial height resolution and spatial width resolution of the local feature representation after the alignment of the region feature, respectively represent the spatial arrangement index of the aligned local feature representation in the height dimension and the width dimension; represent the local feature vector of the aligned local feature representation at coordinates , which contains the depth semantics and target geometry information at the corresponding local spatial location; S3-6. Feature map after dimension reconstruction Global average pooling is performed to generate global features for providing overall scene information: in This represents the local feature vector of the reconstructed feature map at coordinates (i, j). Represents global features; S3-7. Map instance features and global features to a unified feature space through linear projection or feature transformation: in and Let these represent the learnable weight matrix and bias term, respectively. Denotes the uniform spatial features obtained after linear projection. Representing instance features or global features ; S3-8. Adaptive gating fusion mechanism is used for instance features. With global features To achieve fusion, firstly, the instance features and global features are concatenated along the channel dimension, and then a dynamic gating weight vector is calculated using a linear mapping and a Sigmoid activation function. wherein, denotes a concatenation operation in the channel dimension, and denote a learnable mapping matrix and a bias term for the gating weight, respectively, denotes a Sigmoid activation function, denotes a dynamic gating weight; Subsequently, the dynamic gating weight is utilized The instance features and global features are weighted and fused to obtain a fused visual feature : wherein, represents an element-wise multiplication operation of the feature vector; through the adaptive gating fusion mechanism, adaptive dynamic balance of the instance feature and the global feature is realized; Through the above feature encoding and representation learning process, the effective fusion of target appearance features, local region information, and global context information is achieved.

4. The visual target tracking method based on flow matching according to claim 3, characterized in that, Step S4 specifically includes the following sub-steps: S4-1. Let the goal be to have the end state of the current frame be and the start state be where the state vector is represented as the center coordinates and width-height parameters of the target bounding box, i.e. wherein and respectively represent the center coordinates of the target bounding box, and w and h respectively represent the width and height parameters of the target bounding box; S4-2. Define a time variable t∈[0,1] to characterize the continuous evolution of the target state, and construct intermediate state samples from the starting state to the ending state through linear interpolation, represented as: This represents the intermediate sample obtained through linear interpolation, i.e., the state of the randomly sampled noisy bounding box. The conditional probability path function is determined by the starting state x0 and the ending state x1. This expression describes the continuous change path of the target state along the time dimension. In the process of constructing model training samples, in order to ensure the numerical stability of boundary calculation and avoid endpoint gradient singularity, t is uniformly randomly sampled in the open interval (0,1) defined by the starting state and the ending state. S4-3. The ideal conditional velocity field corresponding to the conditional probability path described in step S4-2 is expressed as follows: This represents the difference between the endpoint state and the starting state, i.e., the ideal speed that the model needs to predict. The direction and magnitude of the change in the target state along this path remain constant. S4-4. Introducing Parametric Neural Networks As a velocity field prediction network, in which Indicates the fusion of visual features; The model is optimized by minimizing the difference between the predicted speed and the true speed, whose training objective function is To optimize the model by minimizing the difference between the predicted speed and the true speed, denoted as: During the model training phase, in order to introduce known supervision information to impose deterministic constraints on the model, the real labeled data in the dataset is designated as the endpoint state x1, and the ideal conditional velocity field determined by the endpoint state x1 and the starting state x0 is defined. As a true supervisory label; by measuring the regression residual between the network's predicted velocity field and the true supervisory label, the calculated loss error is backpropagated to optimize the network parameters; S4-5. Online Reasoning: x0 and x1 represent the starting state and ending state of the reasoning, respectively; the change process of the target state is described by the following ordinary differential equation: By numerically integrating this equation, the final state is derived from the initial state and the target localization is completed.​ 5. The method of claim 4, wherein, The step S5 specifically comprises the following sub-steps: S5-1. Construct a velocity field prediction network based on a multi-vector interaction mechanism. This network uses the current target state as the basis for prediction. Time step coding And visual features are used as input to achieve multi-source information fusion modeling through a self-attention mechanism; Specifically, firstly, regarding the time variable... Encode the target state to obtain a temporal embedding vector and then encode the target state. Mapped to state feature representation: ， wherein and are learnable parameters, denotes the time step encoding; denotes the time embedding vector after encoding the time variable t, denotes the state feature representation after encoding the current target state; Further, from the search feature map based on the current target state extract the corresponding instance features and fuse them with the global features into the fusion visual features , construct a multi-source interaction vector set : And the above vector set is input into a multi-layer self-attention network for feature interaction. The attention calculation process is represented as: wherein are the query, key and value matrices, respectively, resulting from a linear transformation of the input vector, is the feature dimension; After the multi-layer stacking, the fused feature representation is obtained ; Finally, the velocity field prediction is obtained by linear mapping : in and For multilayer perceptron parameters, output Indicates the current state and time step The direction and magnitude of the change in the target state.

Images