An Intelligent Tracking Method for Surveillance Videos Based on Collaborative Optimization of Dynamic Features

Abstract

The present invention discloses an intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features, which relates to the fields of computer vision and pattern recognition, and surveillance video tracking, and includes the following steps: (1) Collect and label a scenario dataset for surveillance video tracking, and classify the scenario dataset into a training set and a test set; (2) Define an intelligent tracking model for collaborative optimization of dynamic features. (3) Learn the intelligent tracking model for collaborative optimization of dynamic features: including a training tracker stage and a test tracker stage. The present invention solves the problem that complex tracking and matching algorithms in the prior art cannot meet the requirement of real-time response in a low-computing-resource environment. The present invention cooperates with real-world surveillance cameras to analyze video stream images through collaborative optimization of dynamic features, achieving the technical effect of synchronous tracking.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and pattern recognition, and surveillance video tracking, and particularly to an intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features. Background Art

[0002] In multi-faceted real-world application scenarios, surveillance videos need to have the ability of intelligent tracking and matching with collaborative optimization of dynamic features, which can effectively identify and track targets. Therefore, the method of collaborative optimization of the dynamic features of targets is a key factor for surveillance videos to accurately perform tasks. Although with the development of vision technology, single-object tracking technology has become more mature and the tracking accuracy has become higher and higher. However, the challenges it faces are also complex and diverse, especially problems such as occlusion in complex scenarios, changes in target morphology, and the inability to distinguish between targets and backgrounds. In traditional feature matching methods, it is often difficult to completely separate the appearance features of the target from the background features, especially when the target and background features are similar, this interference phenomenon is more obvious. The existing methods have limitations and cannot meet the requirements of high accuracy and speed for the recognition and tracking of surveillance video targets.

[0003] The current tracking and matching methods for surveillance videos have many deficiencies, which are mainly manifested in the following aspects: First, the occlusion problem. Although single-object tracking technology has a certain anti-occlusion ability, in complex environments, when the target is occluded for a long time or completely, the tracking effect will still be greatly affected. For example, when the target is completely occluded by a crowd or an object, the tracking algorithm may lose the positioning of the target, resulting in the loss of the target. Second, in complex surveillance scenarios, the background may contain a large number of dynamic objects (such as leaves, vehicles, pedestrians, etc.), and these elements will interfere with the single-object tracking algorithm. The dynamic changes in the background may cause the tracking algorithm to misjudge and mistake the dynamic objects in the background for targets, especially when the background is similar to the target. Third, the ability to handle changes in target appearance is limited. In practical applications, the appearance of the target may change significantly, such as due to changes in lighting, changes in target posture, rapid movement of the target, or changes in the appearance of the target itself (such as dressing style, carrying items, etc.). Although some tracking and matching methods can partially solve this problem, in the case of drastic or rapid appearance changes, it is still difficult for the tracking algorithm to accurately track the target. Fourth, the real-time performance is limited. The tracking and matching methods of single-object tracking have a relatively high computational complexity. Especially when processing high-resolution videos or when multiple targets need to be monitored simultaneously, the real-time performance of the system is easily limited. Complex tracking and matching algorithms may not be able to meet the requirements of real-time response in low-computing-resource environments. Summary of the Invention

[0004] The object of the present invention is to provide an intelligent tracking method for surveillance videos based on dynamic feature collaborative optimization in view of the deficiencies of the prior art, aiming to achieve the adaptive matching of surveillance videos. This method combines an advanced Transformer model and single-object tracking technology to provide high-precision recognition and tracking capabilities under complex conditions such as similar object interference, occlusion, and rapid target movement. By using the dynamic feature collaborative optimization intelligent tracking method to match the target object and accurately locate its position, the method of the present invention aims to improve the adaptability of surveillance video tracking in various dynamic scenarios and provide reliable basic support for its accurate execution of tasks.

[0005] To achieve the above object, the present invention provides the following technical solutions: An intelligent tracking method for surveillance videos based on dynamic feature collaborative optimization, comprising the following steps:

[0006] (1) Collect and label the dataset of the surveillance video tracking usage scenario, and classify the scenario dataset into a training set and a test set;

[0007] (2) Define the dynamic feature collaborative optimization intelligent tracking model: Specify the position and category of the target, use the predicted bounding box to locate the position of the target in subsequent video frames, and indicate the foreground and background category information to which the target object enclosed by the bounding box belongs. The predicted bounding box of the dynamic feature collaborative optimization intelligent tracking model is set as an axis-aligned rectangular box. The dynamic feature collaborative optimization intelligent tracking model consists of an encoder that simply extracts target features, a decoder that generates a tracking structure, and a dynamic feature collaborative optimization fusion module;

[0008] (3) Learn the dynamic feature collaborative optimization intelligent tracking model: including the training tracker stage and the test tracker stage, where,

[0009] (3-1) Training tracker stage: The surveillance video dynamic feature collaborative optimization intelligent tracker uses SeqTrack as the benchmark model. After reading the picture, it crops the target based on the center position of the target box, and then performs jittering and data augmentation on all batch pictures, that is, transforms the color space, randomly translates, and normalizes. The pictures after data augmentation are used as the input data of the dynamic feature collaborative optimization intelligent tracking model. First, feature extraction is performed through the encoder, and then the relationship between the template frame and the search frame is associated and modeled through the dynamic feature collaborative optimization fusion method and the decoder to obtain the tracking result;

[0010] (3-2) Test tracker stage: First, read pictures different from the training dataset scenarios from the test set part of the surveillance video tracking usage scenario dataset. After cropping the read pictures, the processed pictures are used as the input data of the dynamic feature collaborative optimization intelligent tracking model. After passing through the dynamic feature collaborative optimization intelligent tracking model, the target tracking prediction result of the current frame is output.

[0011] Preferably, step (1) specifically includes the following steps:

[0012] (1-1) Collect a monitoring video tracking usage scenario dataset composed of various types of targets and the scenarios where different targets are located. Each scenario in the monitoring video tracking usage scenario dataset contains video frames composed of multiple consecutive pictures, and each picture in the video frame contains the same target to be tracked;

[0013] (1-2) Perform picture annotation on the pictures in the monitoring video tracking usage scenario dataset. The picture annotation information of the monitoring video tracking usage scenario dataset includes classification labels and positioning labels. The classification labels have two types of information: foreground target and background; The positioning label includes the coordinates of the target annotation box, and the coordinates are represented as the center point coordinates, width, and height of the annotated target box;

[0014] (1-3) Then divide the monitoring video tracking usage scenario dataset into two parts: a training set and a test set. The target objects in the pictures of the training set do not include the target objects in the pictures of the test set.

[0015] Preferably, when the input of the encoder in step 2) is the template frame picture of and the search frame picture of the template frame picture and the search frame picture pass through linear projection and then add learnable position embeddings. Learnable position embeddings are a technique representing the position information of elements in a sequence. Position embeddings are used to retain the position information of image patches and elements in the sequence because the Transformer itself does not have the ability to process position information. The combined embedding of the two is input into the encoder, including multiple first attention blocks with output vectors of The input of the dynamic feature collaborative optimization fusion module is the feature vector of a feature sequence variable At is added (At is to initialize a blank sequence marker, and then use adaptive max pooling operation to learn the feature marker sequence of the template and the search area to obtain a feature marker sequence suitable for interacting with the template), so that it learns the features suitable for interacting with the template frame, including dynamic feature attention block a, dynamic feature attention block b, and dynamic feature attention block c with output vectors of

[0016] Preferably, when the input of the decoder in step (2) is the template frame picture and the search frame picture of the feature vectors obtained by connecting the template frame and the search frame after feature extraction, including the first multi-head attention block a and the first multi-head attention block b with output vectors of

[0017] Preferably, the step (3-1) of training the tracker includes the following steps:

[0018] (3-1-1) After the video frame pictures in the video pass through the dynamic feature collaborative optimization intelligent tracking model, a prediction vector is obtained, and the prediction vector includes the predicted target object category, the coordinates of the center point of the predicted bounding box, width, and height;

[0019] (3-1-2) Calculate the loss of the result predicted by the dynamic feature collaborative optimization intelligent tracking model through the loss function, and the calculated loss value will backpropagate the gradient back to the monitoring video dynamic feature collaborative optimization intelligent tracker to update the parameters of the monitoring video dynamic feature collaborative optimization intelligent tracker. Among them, the loss function of the dynamic feature collaborative optimization intelligent tracking model uses cross-entropy loss to maximize the log-likelihood of the target sequence based on the previous subsequence and the input video frame. The formula of the objective function is shown in formula (2):

[0020] (2)

[0021] where is the softmax probability, s is the search image, t is the template, z is the target sequence, j is the position of the sequence, L is the length of the target sequence, represents the previous subsequence used to predict the current token .

[0022] Preferably, the step (3-2) of testing the tracker includes the following steps: In the tracker testing stage, the encoder perceives the template image and the search area in the subsequent video frames. The initial input of the decoder is the start sequence (the start sequence is a special sequence designed specifically for the model to tell the model that the target sequence starts to be generated), telling the model to start generating. The model reads out the target sequence sequence by sequence. For each sequence, the model selects the appropriate sequence according to the maximum likelihood. The maximum likelihood selection is shown in formula (3):

[0023] (3)

[0024] where is the softmax probability, s is the search image, t is the template, z is the target sequence, j is the position of the sequence, L is the length of the target sequence, represents the previous subsequence used to predict the current token .

[0025] Preferably, the output vectors connected in sequence in step (2) are all Each of the multiple first attention blocks is provided with an attention layer, a linear layer, and a normalization layer. The attention operation is as shown in formula (4) below: (When interacting with each feature token sequence, the attention operation is the self-attention layer of the first attention layer, that is, the following formula)

[0026] (4)

[0027] Where Q represents the position or target that needs to be focused on currently, which represents the position in the search area that needs to be matched here, and comes from the linear transformation of the search area features, encoding the features of each position in the search area for matching with the template features. K represents the candidate information to be queried, which represents the features of the target template (the target information in the initial frame) here and serves as the reference for matching. V contains the information actually used for weighted aggregation, which carries the detailed feature information of the target template here and is used for weighted aggregation to generate the final tracking result. Both K and V come from the linear transformation of the template features, retaining the semantic and spatial information of the target. T represents the transpose of the matrix for convenient calculation; d k represents the dimension of the K vector.

[0028] Calculate the similarity between the Query of the search area and the Key of the template to obtain the attention weights. The positions with high weights indicate that a certain position in the search area is highly correlated with the template target. Finally, use the attention weights to perform weighted summation on the Value of the template to generate the final tracking response map.

[0029] Preferably, the multi-head attention block 1 and the multi-head attention block 2 described in step (2) are both provided with a masked multi-head attention layer, a multi-head attention layer, a feed-forward neural network layer, and a normalization layer; among them, the masked multi-head attention layer uses a causal mask to ensure that the output of each sequence element only depends on the sequence elements in front of it. The input vector of the multi-head attention layer , contains weight matrices , where , and the final output result is as shown in formula group (5):

[0030] (5)

[0031] Where ; where ; head i represents the subspace corresponding to the i-th attention head, W i is the projection matrix for obtaining the corresponding Q, K, and V, W is the output matrix, Concat() is to splice the outputs of multiple heads in the feature dimension, and the Mutihead() function is to complete the projection, attention calculation, and splicing of multiple heads to obtain the final result.

[0032] Preferably, the dynamic feature attention blocks a, b, and c described in step (2) are all provided with an adaptive attention layer, a feed-forward neural network layer, and a normalization layer.

[0033] The present invention has the following beneficial effects: The present invention utilizes computer vision analysis technology and cooperates with monitoring cameras in reality to dynamically and synergistically optimize the analysis of video stream images, dynamically and synergistically identify and track the target position, alleviates the adjustment of the existence of similar objects and target occlusion in the monitored video, enables the dynamic feature collaborative optimization intelligent tracking model of the monitored video to adapt to the target position conditions in different actual scenarios, and improves the tracking accuracy. The method provided by the present invention can more accurately and stably identify and track the target, thereby providing better basic support for the monitored video tracking task. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0035] Figure 1 It is a schematic diagram of the training and testing process of the present invention.

[0036] Figure 2 It is a schematic diagram of the annotation of the monitored video tracking usage scenario dataset created by the present invention.

[0037] Figure 3 It is a schematic diagram of the target tracker model of the present invention.

[0038] Figure 4 It is a schematic diagram of the structure of the first attention block and the first multi-head attention block of the target tracker model of the present invention.

[0039] Figure 5 It is a schematic diagram of the dynamic feature collaborative optimization intelligent tracking module of the target tracker model of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. Therefore, the detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] Embodiment

[0042] The following are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the following embodiments. All technical solutions falling within the concept of the present invention belong to the protection scope of the present invention.

[0043] Referring to the attached drawings of the specification Figure 1 , an intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features includes the following steps:

[0044] An intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features includes the following steps:

[0045] (1) Collect a dataset of surveillance video tracking usage scenarios: In real-world scenarios of surveillance video tracking applications such as police tracking criminal suspects, crowd counting, pedestrian tracking, and vehicle monitoring, collect various different types of targets and the scenarios where the different targets are located to form a dataset of surveillance video tracking usage scenarios. Each scenario in the dataset of surveillance video tracking usage scenarios contains video frames composed of multiple consecutive pictures, and each picture in the video frame contains the same target to be tracked, as Figure 2 shown. Label the pictures in the dataset of surveillance video tracking usage scenarios. The picture annotation information in the dataset of surveillance video tracking usage scenarios includes classification labels and localization labels. The classification labels have two types of information: foreground targets and background; the localization labels include the coordinates of the target annotation box, and the coordinates are represented as the center point coordinates, width, and height of the annotated target box. Then divide the dataset of surveillance video tracking usage scenarios into two parts: a training set and a test set. The target objects in the pictures of the training set do not include the target objects in the pictures of the test set;

[0046] (2) Define the intelligent tracking model for dynamic feature collaborative optimization: Consider object tracking as an autoregressive learning process, which is equivalent to using the model to learn what the object looks like and locate the desired object. That is, it is necessary to detect both the category information and the location information of the object at the same time. The tracking content of the intelligent tracking model for dynamic feature collaborative optimization includes: the location and category of the specified object, using a predicted bounding box to locate the object's location in subsequent video frames, and indicating the foreground and background category information to which the object enclosed by the bounding box belongs. The predicted bounding box of the intelligent tracking model for dynamic feature collaborative optimization is set as an axis-aligned rectangle box. The intelligent tracking model for dynamic feature collaborative optimization consists of an encoder that simply extracts object features, a decoder that generates a tracking structure, and a dynamic feature collaborative optimization fusion module. Among them,

[0047] As Figure 2 shown, the input of the encoder is the template frame image and the search frame image. After the template frame and the search frame are linearly projected, and then added with a learnable position embedding, the combined embedding is input into the encoder. It includes the first attention blocks a, b, c, d, e, f, g, h, i, j, k, l whose output vectors are all sequentially connected.

[0048] As Figure 5 shown, the input of the dynamic feature collaborative optimization fusion module is the feature vector, and a feature sequence variable At is added to let it learn the features suitable for interacting with the template frame. It includes the output vectors that are all Dynamic Feature Attention Blocks 1, 2, and 3. To achieve this goal, we designed a learning module that uses adaptive max pooling operations to combine the features of the template frame and the search frame. Among them, Adaptive Max Pooling is a pooling operation, as shown in formula (1): Adaptive Max Pooling can dynamically optimize and adjust feature collaboration according to the size of the input feature map and the size of the required output feature map to retain more or less information and adapt to input images or feature maps of different sizes. The obtained At performs cross-attention operation with the feature sequence of the template frame for feature fusion, while the search frame only performs self-attention operation. This ensures the purity of the template frame information and fully utilizes the background information of the search frame to distinguish interfering objects. To adapt to the addition of At, first, At is an empty feature sequence, which is sent into the encoder together with the feature sequences of the template frame and the search frame. This requires improving the encoder. For this reason, we designed a dynamic collaborative optimization encoder. Specifically, the input of the adaptive attention consists of three parts: one is Vt suitable for interacting with the template, one is the feature sequence of the template frame, and the other is the feature sequence of the search frame. Specifically, the features of the template frame and the search frame are enhanced through self-attention operation, and the interference from the search area is eliminated. At the same time, cross-attention operation is used to learn the Vt feature and the search area feature to obtain more discriminative target features.

[0049] (1)

[0050] where is the search frame, is the template frame, is the learned feature. The obtained performs cross-attention with the tokens in the template frame for feature fusion.

[0051] The input of the decoder is the feature vector obtained by connecting the template frame and the search frame after feature extraction, including the output vectors connected in sequence, all of which are the first multi-head attention block a and the first multi-head attention block b;

[0052] The pictures of the scene dataset used for monitoring video tracking are input into the encoder-decoder for feature extraction and feature fusion, aiming to fully extract the discriminative features of the target object to distinguish it from other interfering objects. When the encoder extracts the picture features, more discriminative features are obtained through the dynamic feature collaborative optimization fusion module, and then the learned features are sent into the decoder, and finally the tracking result is obtained. The object tracker trained by this network structure is a single-stage end-to-end tracking model, that is, the position information of the target can be obtained simultaneously in one stage.

[0053] (3) Learning dynamic feature collaborative optimization intelligent tracking model: including the training tracker stage and the testing tracker stage. Among them,

[0054] (3-1) Training tracker stage: The monitoring video dynamic feature collaborative optimization intelligent tracker uses SeqTrack as the benchmark model. The training samples are batch-selected from the training set of the monitoring video tracking usage scenario dataset created in step (1). Randomly read n pictures in each batch, and the value of n can be arbitrarily selected. The common range is: . After reading the pictures, crop the target based on the center position of the target box to size. Then, perform jittering and data augmentation on all batch pictures, that is, transform the color space, randomly translate, and normalize. The pictures after data augmentation are used as the input data of the dynamic feature collaborative optimization intelligent tracking model. First, perform feature extraction through the encoder, and then use the dynamic feature collaborative optimization fusion method and the decoder to relate and model the relationship between the template frame and the search frame to obtain the tracking result. After the video frame pictures in the video pass through the dynamic feature collaborative optimization intelligent tracking model, a prediction vector is obtained. The prediction vector includes the predicted target object category, the coordinates of the center point of the predicted bounding box, width, and height. Then, calculate the loss of the result predicted by the dynamic feature collaborative optimization intelligent tracking model through the loss function. The calculated loss value will backpropagate the gradient back to the monitoring video dynamic feature collaborative optimization intelligent tracker through the backpropagation algorithm to update the parameters of the monitoring video dynamic feature collaborative optimization intelligent tracker. Through such a learning mode, the dynamic feature collaborative optimization intelligent tracking model iteratively learns to identify and track the target object, continuously fitting the distribution of the real data, and training to obtain an optimal monitoring video dynamic feature collaborative optimization intelligent tracker tracking model. Among them, the loss function of the dynamic feature collaborative optimization intelligent tracking model uses cross-entropy loss to maximize the log-likelihood of the target sequence based on the previous subsequence and the input video frame. The formula of the objective function is shown in formula (2):

[0055] (2)

[0056] Where is the softmax probability, s is the search image, t is the template, z is the target sequence, j is the position of the sequence, and L is the length of the target sequence. represents the previous subsequence used to predict the current token .

[0057] (3-2) Testing tracker stage: First, read pictures different from the training dataset scenario from the test set part of the monitoring video tracking usage scenario dataset, and uniformly crop the read pictures to the size of A pixel image, and then the processed image is used as the input data for collaborative optimization of the intelligent tracking model. After passing through the dynamic feature collaborative optimization intelligent tracking model, the target tracking prediction result of the current frame is output. The position of the predicted target box in the current frame is retained, and the next frame of the image is cropped based on the center point of the target box. In the tracker test phase, the encoder perceives the template image and the search area in subsequent video frames. The initial input of the decoder is the start sequence, which tells the model to start generating, and the model "reads out" the target sequence sequence by sequence. For each sequence, the model selects the appropriate sequence according to the maximum likelihood, as shown in Equation (3):

[0058] (3)

[0059] As Figure 4 shown, the first attention blocks a, b, c, d, e, f, g, h, i, j, k, and l described in step 2) are all provided with an attention layer, a linear layer, and a normalization layer. The attention operation is as shown in Equation (4):

[0060] (4)

[0061] where Q represents the position or target that needs to be focused on currently, here it represents the position to be matched in the search area, which comes from the linear transformation of the search area features, encodes the features of each position in the search area, and is used to match the template features; K represents the candidate information to be queried, here it represents the features of the target template, which is used as the reference for matching; V contains the information actually used for weighted aggregation, here it carries the detailed feature information of the target template, and is used for weighted aggregation to generate the final tracking result; both K and V come from the linear transformation of the template features, retaining the semantic and spatial information of the target; T represents the transpose of the matrix; d k represents the dimension of the K vector.

[0062] As Figure 4 shown, the first multi-head attention block 1 and the first multi-head attention block 2 described in step (2) are both provided with a multi-head attention layer, a linear layer, and a normalization layer. Among them, the masked multi-head attention layer uses a causal mask to ensure that the output of each sequence element only depends on the sequence elements before it and does not pay attention to the subsequent outputs. The input vector of the multi-head attention layer , contains weight matrices , where , and the final output result is as shown in Equation (5):

[0063] (5)

[0064] Among them .head i represents the subspace corresponding to the i-th attention head. W i is the projection matrix to obtain the corresponding QKV, and W is the output matrix, both of which are initialized through model parameters. Concat() is to splice the outputs of multiple heads in the feature dimension. The Mutihead() function is to complete the projection, attention calculation and splicing of multiple heads to obtain the final result.

[0065] As shown in the attached drawings of the specification Figure 4 As shown, the first attention blocks a, b, c, d, e, f, g, h, i, j, k, l described in step (2) are all composed of sequentially connected first attention blocks.

[0066] As shown in the attached drawings of the specification Figure 4 As shown, the first multi-head attention blocks a and b described in step (2) are both composed of first multi-head attention blocks.

[0067] As shown in the attached drawings of the specification Figure 5 As shown, the dynamic feature attention blocks a, b, and c described in step (2) are all composed of sequentially connected dynamic feature collaborative optimization and fusion modules.

[0068] The above embodiments are only for illustrating the technical concept and characteristics of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be covered within the protection scope of the present invention.

Claims

1. An intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features, characterized in that It includes the following steps: (1) Collect and label the monitoring video tracking usage scenario dataset, and classify the scenario dataset into a training set and a test set; (2) Define the dynamic feature collaborative optimization intelligent tracking model: Specify the position and category of the target, use the predicted bounding box to locate the position of the target in subsequent video frames, and indicate the foreground and background category information to which the target object enclosed by the bounding box belongs. The predicted bounding box of the dynamic feature collaborative optimization intelligent tracking model is set as an axis-aligned rectangular box. The dynamic feature collaborative optimization intelligent tracking model consists of an encoder that simply extracts target features, a decoder that generates the tracking structure, and a dynamic feature collaborative optimization fusion module; (3) Learn the dynamic feature collaborative optimization intelligent tracking model: It includes the training tracker stage and the test tracker stage, where, (3-1) Training tracker stage: The monitoring video dynamic feature collaborative optimization intelligent tracker uses SeqTrack as the benchmark model. After reading the picture, it crops the target based on the center position of the target box, and then performs jittering and data augmentation on all batch pictures, that is, transforms the color space, randomly translates, and normalizes. The pictures after data augmentation are used as the input data of the dynamic feature collaborative optimization intelligent tracking model. First, feature extraction is performed through the encoder, and then the relationship between the template frame and the search frame is associated and modeled through the dynamic feature collaborative optimization fusion method and the decoder to obtain the tracking result; (3-2) Test tracker stage: First, read pictures different from the training dataset scenarios from the test set part of the monitoring video tracking usage scenario dataset. After cropping the read pictures, the processed pictures are used as the input data of the dynamic feature collaborative optimization intelligent tracking model. After passing through the dynamic feature collaborative optimization intelligent tracking model, the target tracking prediction result of the current frame is output.

2. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 1, wherein The step (1) specifically includes the following steps: (1-1) Collect the monitoring video tracking usage scenario dataset composed of various different types of targets and the scenarios where the different targets are located. Each scenario in the monitoring video tracking usage scenario dataset contains video frames composed of multiple consecutive pictures, and each picture in the video frame contains the same target to be tracked; (1-2) Perform picture annotation on the pictures of the monitoring video tracking usage scenario dataset. The picture annotation information of the monitoring video tracking usage scenario dataset includes classification labels and positioning labels. The classification labels have two types of information: foreground target and background; The positioning label includes the target annotation box coordinates, and the coordinates are represented as the center point coordinates, width, and height of the annotated target box; (1-3) Then divide the monitoring video tracking usage scenario dataset into two parts: a training set and a test set. The target objects in the pictures of the training set do not include the target objects in the pictures of the test set.

3. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 1, characterized in that, When the input of the encoder in the step (2) is of the template frame picture and of the search frame picture, the template frame picture and the search frame picture are linearly projected and then added with learnable position embeddings, and the combined embeddings of the two are input into the encoder, including a plurality of first attention blocks with output vectors that are all ; the input of the dynamic feature collaborative optimization and fusion module is of the feature vector, and a feature sequence variable of At is added to let it learn the features suitable for interacting with the template frame, including the dynamic feature attention block a, the dynamic feature attention block b, and the dynamic feature attention block c with output vectors that are all .

4. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 1, wherein When the input of the decoder in the step (2) is the template frame picture and the search frame picture, and the feature vectors obtained by connecting the template frame and the search frame after feature extraction, including the output vectors connected in sequence, are all the first multi-head attention block a and the first multi-head attention block b.

5. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 1, characterized in that, The step (3-1) training tracker stage includes the following steps: (3-1-1) After the video frame pictures in the video pass through the dynamic feature collaborative optimization intelligent tracking model, a prediction vector is obtained. The prediction vector includes the predicted target object category, the predicted bounding box center point coordinates, width, and height; (3-1-2) Calculate the loss of the results predicted by the dynamic feature collaborative optimization intelligent tracking model through the loss function. The calculated loss value will backpropagate the gradient back to the monitoring video dynamic feature collaborative optimization intelligent tracker through the backpropagation algorithm to update the parameters of the monitoring video dynamic feature collaborative optimization intelligent tracker. Among them, the loss function of the dynamic feature collaborative optimization intelligent tracking model uses cross-entropy loss to maximize the log-likelihood of the target sequence based on the previous subsequence and the input video frame. The formula of the objective function is shown in formula (2): (2) Among them is the softmax probability, s is the search image, t is the template, z is the target sequence, j is the position of the sequence, L is the length of the target sequence, represents the previous subsequence used to predict the current token of the target sequence.

6. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 1, wherein, The (3-2) test tracker stage includes the following steps: In the tracker test stage, the encoder perceives the template image and the search area in the subsequent video frames. The initial input of the decoder is the start sequence, which tells the model to start generating. The model reads out the target sequence sequence by sequence. For each sequence, the model selects the appropriate sequence according to the maximum likelihood. The maximum likelihood selection is shown in formula (3): (3) where is the softmax probability, s is the search image, t is the template, z is the target sequence, j is the position of the sequence, L is the length of the target sequence, represents the previous subsequence used to predict the current token .

7. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 3, wherein The output vectors connected in sequence in step (2) are all A plurality of first attention blocks are each provided with an attention layer, a linear layer, and a normalization layer, and the attention operation is as shown in formula (4) below: (4) Among them, Q represents the position or target that needs to be concerned currently, which represents the position to be matched in the search area here, originating from the linear transformation of the search area features, encoding the features of each position in the search area, and used to match with the template features; K represents the candidate information to be queried, which represents the features of the target template here and serves as the reference for matching; V contains the information actually used for weighted aggregation, which carries the detailed feature information of the target template here and is used for weighted aggregation to generate the final tracking result; both K and V originate from the linear transformation of the template features, retaining the semantic and spatial information of the target; T represents the transpose of the matrix; d k represents the dimension of the K vector.

8. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 4, characterized in that The multi-head attention block a and the multi-head attention block b described in step (2) are both provided with a masked multi-head attention layer, a multi-head attention layer, a feed-forward neural network layer, and a normalization layer; wherein the masked multi-head attention layer uses a causal mask to ensure that the output of each sequence element depends only on the sequence elements preceding it, and the input vector of the multi-head attention layer , contains weight matrices , where , and the final output result is shown in formula group (5): (5) Among them ; head i represents the subspace corresponding to the i-th attention head, and W i is the projection matrix for obtaining the corresponding QKV, W is the output matrix, Concat() is to concatenate the outputs of multiple heads in the feature dimension, and the Mutihead() function is to complete the projection, attention calculation and concatenation of multiple heads to obtain the final result.

9. The intelligent tracking method for surveillance videos based on collaborative optimization of dynamic features according to claim 3, wherein In the dynamic feature attention block a, dynamic feature attention block b, and dynamic feature attention block c described in step (2), each is provided with an adaptive attention layer, a feedforward neural network layer, and a normalization layer.

Images