A low, slow and small target tracking method and tracking system

Abstract

The present invention provides a low-speed, slow, and small target tracking method and tracking system, which relates to the field of target tracking technology. First, a target is selected from the first frame image of a video sequence and a target template and its feature vector are generated. Then, an initial particle set is generated using the first branch of a Siamese network. During the tracking process, the present invention uses a particle filter to predict the new position and new state of the particles in the next frame based on the particle state and motion model of the previous frame, and extracts the candidate area and its feature vector corresponding to each particle from the current frame. Then, the present invention calculates the similarity between the feature vector of each candidate area and the feature vector of the target template, and assigns a weight to each particle according to the similarity, and finally calculates the final position of the target by a weighted average method. After obtaining the final position of the target in the current frame, the present invention also uses the position to dynamically update the target template so as to continue tracking the target in the next frame.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to a low, slow and small target tracking method and tracking system. Background Art

[0002] In the field of modern video processing and computer vision, the tracking technology of low, slow and small targets (such as drones, birds, etc.) has a wide range of application value, including but not limited to security monitoring, air traffic management, wildlife protection, etc. However, since low, slow and small targets often have small size, slow movement speed, and are easily affected by background interference in video sequences, their accurate tracking becomes a very challenging task.

[0003] Traditional target tracking methods, such as optical flow and Kalman filters, perform well in simple scenarios. However, when faced with small, slow and low targets, the tracking accuracy and robustness of these methods are often difficult to meet actual needs due to problems such as unclear target features and susceptibility to noise interference.

[0004] Therefore, it is necessary to provide a low, slow and small target tracking method and tracking system to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a low, slow and small target tracking method and tracking system, which combines the advantages of Siamese network and particle filter to achieve accurate tracking of low, slow and small targets.

[0006] The present invention provides a low, slow and small target tracking method, the tracking method comprising the following steps:

[0007] Select a target from the first frame of the video sequence, extract features of the selected target, and generate a target template and its feature vector;

[0008] Processing the target template using a first branch of a Siamese network to generate an initial particle set, wherein each particle in the initial particle set represents a candidate position and a candidate state of the target;

[0009] For each particle, based on the particle state and motion model of the previous frame, a particle filter is used to predict the new position and state of the particle in the next frame.

[0010] According to the predicted new position and new state of the particle in the next frame, at least one candidate region corresponding to each particle is extracted from the current frame, and the feature vector of each candidate region is extracted using the second branch of the Siamese network;

[0011] Calculating the final position of the target based on a first similarity between the feature vector of each candidate region and the feature vector of the target template;

[0012] The target template is dynamically updated using the final position of the target in the current frame, and the final position is set as the starting position for tracking the next frame, and the previous steps are repeated to achieve continuous tracking of the target in the next frame.

[0013] Preferably, the step of selecting a target from the first frame of the video sequence, extracting features of the selected target, and generating a target template and a feature vector thereof includes:

[0014] Select the target in the first frame of the video sequence;

[0015] Applying a feature extraction algorithm to the selected target to extract target features of the target;

[0016] A target template is constructed according to the extracted target features, and the target template is converted into a feature vector form to obtain a feature vector of the target template.

[0017] Preferably, the target template is processed using the first branch of the Siamese network to generate an initial particle set, wherein each particle in the initial particle set represents a candidate position and a candidate state of the target, including:

[0018] Inputting the target template into the first branch of the Siamese network, and using the first branch of the Siamese network to perform feature learning and matching on the target template, and evaluating the second similarity between the target template and the potential candidate target;

[0019] According to the second similarity, generating a set of initial particles in the current frame of the video sequence, wherein each particle represents a potential candidate position and candidate state;

[0020] The particle distribution in the initial particle set is initialized according to the feature vector of the target template and the expected motion pattern.

[0021] Preferably, for each particle, according to the particle state and motion model of the previous frame, using a particle filter to predict the new position and new state of the particle in the next frame includes:

[0022] Reading the particle state of each particle in the previous frame, wherein the particle state includes position, velocity and acceleration;

[0023] Based on the model mapping table, a motion model is selected according to the target tracking scenario;

[0024] Taking the particle state of the previous frame and the selected motion model as input, a particle filter is applied to perform state prediction;

[0025] The particle filter iteratively updates the position and state of the particles to estimate the new position and state of each particle in the next frame.

[0026] Preferably, extracting at least one candidate region corresponding to each particle from the current frame according to the predicted new position and new state of the particle in the next frame, and extracting a feature vector of each candidate region using the second branch of the Siamese network, comprises:

[0027] For each particle in the current frame, one or more candidate regions are determined based on the predicted new position and new state;

[0028] For each candidate region, a corresponding image block is extracted from the current frame using a predetermined transformation method;

[0029] Each extracted image block is input into the second branch of the Siamese network, and the second branch of the Siamese network is used to extract features of each image block to generate a corresponding feature vector.

[0030] Preferably, the calculating the final position of the target based on the first similarity between the feature vector of each candidate region and the feature vector of the target template includes:

[0031] For each particle in the current frame, the similarity between the feature vector of the corresponding candidate region and the feature vector of the target template is calculated to obtain a first similarity;

[0032] According to the first similarity, a weight is assigned to each particle, wherein the similarity is proportional to the weight;

[0033] Resample the particle set based on the particle weight to generate a new particle set;

[0034] According to the positions and weights of the particles in the new particle set, the final position of the target is calculated by the weighted average method.

[0035] Preferably, the target template is dynamically updated using the final position of the target in the current frame, and the final position is set as the starting position of the next frame tracking, and the previous steps are repeated to achieve continuous tracking of the target in the next frame, including:

[0036] After obtaining the final position of the target in the current frame, updating the target template according to the final position;

[0037] After the update, the initial state of the particle filter in the next frame is set based on the final position and predicted;

[0038] The entire tracking process from target template generation to target final position calculation is repeated until the end of the video sequence, thereby achieving continuous tracking of low, slow and small targets in the entire video sequence.

[0039] The present invention also provides a low-speed, slow, and small target tracking system, which is used to perform a low-speed, slow, and small target tracking method. The tracking system includes:

[0040] The feature extraction module is used to select a target from the first frame image of the video sequence, extract features from the selected target, and generate a target template and its feature vector;

[0041] A particle set generation module, used for processing the target template using the first branch of the Siamese network to generate an initial particle set, wherein each particle in the initial particle set represents a candidate position and a candidate state of the target;

[0042] The particle state prediction module is used to predict the new position and new state of each particle in the next frame using a particle filter according to the particle state and motion model of the previous frame;

[0043] A candidate region feature extraction module is used to extract at least one candidate region corresponding to each particle from the current frame according to the predicted new position and new state of the particle in the next frame, and extract a feature vector of each candidate region using the second branch of the Siamese network;

[0044] A target position calculation module, used for calculating the final position of the target based on the first similarity between the feature vector of each candidate area and the feature vector of the target template;

[0045] The tracking loop module is used to dynamically update the target template using the final position of the target in the current frame, and set the final position as the starting position for tracking the next frame, and repeat the previous steps to achieve continuous tracking of the target in the next frame.

[0046] Compared with the related art, the low, slow and small target tracking method and tracking system provided by the present invention have the following beneficial effects:

[0047] The present invention first selects a target from the first frame image of a video sequence and generates a target template and its feature vector, and then generates an initial particle set using the first branch of a Siamese network. During the tracking process, the present invention uses a particle filter to predict the new position and new state of the particles in the next frame based on the particle state and motion model of the previous frame, and extracts the candidate region and its feature vector corresponding to each particle from the current frame. Next, the present invention calculates the similarity between the feature vector of each candidate region and the feature vector of the target template, and assigns a weight to each particle based on the similarity, and finally calculates the final position of the target by a weighted average method. After obtaining the final position of the target in the current frame, the present invention also uses the position to dynamically update the target template so as to continue tracking the target in the next frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a low, slow, small target tracking method provided by the present invention;

[0049] Figure 2 A module structure diagram of a low, slow and small target tracking system provided by the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.

[0051] It should also be noted that, for ease of description, only the parts related to the present invention, but not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the various operations can be rearranged. When its operations are completed, the processing can be terminated, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0052] Embodiment 1

[0053] The present invention provides a low, slow and small target tracking method, referring to Figure 1 As shown, the tracking method includes the following steps:

[0054] S1: Select a target from the first frame of the video sequence, extract features from the selected target, and generate a target template and its feature vector.

[0055] Specifically, step S1 includes the following steps:

[0056] S11: Select a target in the first frame image of the video sequence.

[0057] In this step, you first need to select the target from the first frame of the video sequence. This step is the basis of the entire tracking process, and its accuracy directly affects the effect of subsequent tracking. The process of selecting the target can be completed by including but not limited to manual selection or automatic recognition using a pre-trained detection model.

[0058] Exemplarily, a method combining user interaction and automatic detection is adopted, that is, the user first roughly selects the approximate position of the target in the first frame image, and then uses the pre-trained target detection model to further refine the boundary of the target. This method not only ensures the accuracy of target selection, but also improves the convenience of operation. The target selected in this way can more accurately reflect the size and shape of the actual target, providing a good foundation for subsequent feature extraction and tracking.

[0059] S12: Apply a feature extraction algorithm to the selected target to extract target features of the target.

[0060] In this application, feature extraction is one of the key steps in the tracking algorithm, and its purpose is to obtain information that can uniquely identify the target from the image.

[0061] In this embodiment, a deep convolutional neural network (DCNN) is used as a feature extractor, especially pre-trained network models, including but not limited to VGG and ResNet, which have been fully trained on large-scale data sets and can effectively capture the deep features of the target. Through these advanced features, the recognition ability of the target in different environments can be significantly improved, especially in complex situations such as lighting changes and occlusions, these features can provide stronger robustness and discrimination.

[0062] S13: construct a target template according to the extracted target features, and convert the target template into a feature vector form to obtain a feature vector of the target template.

[0063] In this application, the purpose of constructing a target template is to provide a standard reference for subsequent matching and tracking.

[0064] In this embodiment, the constructed target template not only includes the visual features of the target, but also takes into account the shape information of the target. The target template is converted into a feature vector to facilitate rapid comparison with the features of the candidate area in subsequent steps. The target template constructed in this way can maintain high precision while also having high real-time performance, laying a solid foundation for the subsequent tracking process. In addition, the feature vector can also be dynamically adjusted as the target appearance changes, ensuring that high tracking accuracy can be maintained throughout the tracking process.

[0065] S2: Use the first branch of the Siamese network to process the target template and generate an initial particle set, where each particle in the initial particle set represents a candidate position and a candidate state of the target.

[0066] Specifically, step S2 includes the following steps:

[0067] S21: Input the target template into the first branch of the Siamese network, and use the first branch of the Siamese network to perform feature learning and matching on the target template, and evaluate the second similarity between the target template and the potential candidate target.

[0068] In this embodiment, the Siamese network is a neural network structure, which consists of two sub-networks with shared weights, which are usually symmetrical and can accept different inputs but output feature vectors of the same dimension. This network architecture is very suitable for comparison tasks, such as target matching in this application.

[0069] In this step, the first branch of the Siamese network is responsible for deep feature learning of the target template, that is, extracting features that can effectively describe the target from the target template. These features include not only visual information such as color and texture, but also higher-level semantic information. In this way, the network can capture the essential characteristics of the target and maintain good recognition capabilities even if the target undergoes certain changes in subsequent frames (such as rotation, scaling, etc.). In addition, the first branch of the Siamese network also calculates the similarity between the target template and other potential candidate targets in the video. This step provides the basis for the generation of subsequent particle sets.

[0070] S22: Generate a set of initial particles in the current frame of the video sequence according to the second similarity, wherein each particle represents a potential candidate position and candidate state.

[0071] In this embodiment, based on the calculated second similarity, a set of initial particles can be generated in the current video frame. The particles here actually refer to the hypothesis of the possible location and state of the target. Each particle represents a possible target candidate, including its location, size and other information in the image.

[0072] By selecting regions with high similarity to the target template to generate these particles, it is ensured that most members of the particle set are valid candidates for the target. This method helps to reduce the search space during tracking and improve tracking efficiency and accuracy. At the same time, since particles are generated based on similarity scores, even if the target undergoes some morphological changes, the consistency and stability of tracking can be maintained to a certain extent.

[0073] S23: Initializing the particle distribution in the initial particle set according to the feature vector of the target template and the expected motion pattern.

[0074] In this embodiment, in order to further optimize the quality of the particle set, the particle distribution in the initial particle set is initialized according to the feature vector of the target template and the preset motion mode.

[0075] The motion mode mentioned here is a series of rules pre-defined based on the target's motion law, such as uniform linear motion, uniformly accelerated motion, etc. By combining the target's historical motion trajectory and current feature information, the possible position of the target in the next frame can be predicted more accurately. This process not only helps to reduce the randomness of particles, but also enhances the representativeness of the particle set, making the particle set more closely around the target's true position. Ultimately, the initialized particle set will provide a more accurate and reliable starting point for the next tracking step, thereby improving the performance of the entire tracking system.

[0076] S3: For each particle, based on the particle state and motion model of the previous frame, a particle filter is used to predict the new position and new state of the particle in the next frame.

[0077] Specifically, step S3 includes the following steps:

[0078] S31: Read the particle state of each particle in the previous frame, where the particle state includes position, velocity and acceleration.

[0079] In this embodiment, the state information of each particle in the previous frame is read, and this state information includes the position, velocity and acceleration of the particle. This is because the movement of the target is usually continuous between consecutive video frames, so the state of the particle in the previous frame is crucial for predicting the possible position of the target in the next frame. The state information of the particle not only helps the system understand the current position of the target, but also provides clues about how the target moves, which is very useful for predicting the future movement path of the target. By accurately reading and storing this state information, reliable data support can be provided for subsequent prediction steps.

[0080] S32: Based on the model mapping table, a motion model is selected according to the tracking scene of the target.

[0081] In this embodiment, a suitable motion model is selected based on a model mapping table to describe the movement of the target. The model mapping table is a predefined database that contains a variety of possible motion models, such as uniform linear motion, uniformly accelerated motion, random walk, etc. The selection of these models depends on the actual motion characteristics of the target and the specific requirements of the tracking scene. For example, in an open area, the target may be more inclined to move in a straight line; while in a complex environment, the movement of the target may be more random. Selecting the correct motion model is crucial to improving the accuracy and efficiency of tracking. By reasonably selecting a motion model, it is possible to better adapt to different motion modes of the target and improve the adaptability and robustness of the tracking system.

[0082] S33: Taking the particle state of the previous frame and the selected motion model as input, a particle filter is applied for state prediction.

[0083] In this embodiment, the particle state information read in the previous frame and the selected motion model are input into the particle filter together to predict the new position and new state of each particle in the next frame. The particle filter is a recursive Bayesian estimation method, which is particularly suitable for handling nonlinear and non-Gaussian problems and is very suitable for application scenarios such as target tracking.

[0084] In this process, the particle filter predicts the future position and state of each particle based on the state information of the previous frame and the selected motion model. This prediction process takes into account the possible movement direction and speed changes of the target, thereby generating a new set of particles that represent the hypothesis of the position and state of the target in the next frame. In this way, the search range can be effectively narrowed and the efficiency and accuracy of tracking can be improved.

[0085] S34: The particle filter iteratively updates the position and state of the particles to estimate the new position and new state of each particle in the next frame.

[0086] In this embodiment, the particle filter iteratively updates the position and state of the particles to further refine the estimation of the new position and new state of each particle in the next frame.

[0087] This iterative process allows the position and state of the particle to be continuously adjusted according to the latest observation information, ensuring the accuracy of the tracking results. Each iteration is based on the current best estimate, and through multiple iterations, the true position of the target can be gradually approached.

[0088] In addition, the iteration process can also help the system adapt to changes in the target's motion characteristics, such as sudden acceleration or change of direction. This dynamic adjustment mechanism is very important for dealing with unpredictable behavior of the target and can significantly improve the stability and reliability of the tracking system. By continuously updating the state of the particles, the consistency and accuracy of the tracking process are ensured, and good tracking results can be maintained even when the target motion is complex and changeable.

[0089] S4: According to the predicted new position and new state of the particle in the next frame, at least one candidate region corresponding to each particle is extracted from the current frame, and a feature vector of each candidate region is extracted using the second branch of the Siamese network.

[0090] Specifically, step S4 includes the following steps:

[0091] S41: For each particle in the current frame, one or more candidate regions are determined according to the predicted new position and new state.

[0092] In this step, first, a reasonable search range needs to be calculated based on the new position and new state of each particle predicted by the particle filter in the previous step. This search range is usually a rectangular area centered on the predicted position of the particle, and its size can be adjusted appropriately according to the expected movement speed and direction of the target. For example, if the target is predicted to move quickly, the candidate area can be set larger to cover a larger possible position change. The purpose of this is to ensure that even if the target has a large displacement between two frames, it can be included in the candidate area, thereby reducing the risk of missed detection.

[0093] In addition, for possible scale changes, the size of the candidate region can also be dynamically adjusted to adapt to changes in the target size. The candidate region determined in this way can not only accurately locate the possible position of the target, but also effectively reduce unnecessary computational burden and improve tracking efficiency.

[0094] S42: For each candidate region, a corresponding image block is extracted from the current frame using a predetermined transformation method.

[0095] In this embodiment, this step involves accurately cutting out the candidate region from the video frame to facilitate subsequent feature extraction. The transformation methods include but are not limited to translation, scaling, rotation, etc., which can be flexibly selected according to the motion characteristics of the target.

[0096] For example, if the target is significantly rotated between the previous and next frames, then the corresponding rotation transformation should be applied when extracting the image block to ensure that the extracted image block is aligned with the target template as much as possible.

[0097] In addition, in order to increase the processing speed, fast image processing technology, such as GPU acceleration, can be used to ensure that image blocks can be extracted efficiently even in real-time tracking. Accurate image block extraction can ensure the accuracy and effectiveness of subsequent feature extraction, thereby improving the overall tracking performance.

[0098] S43: Input each extracted image block into the second branch of the Siamese network, use the second branch of the Siamese network to extract features from each image block, and generate a corresponding feature vector.

[0099] In the specific implementation, the extracted image blocks need to be preprocessed into a format suitable for input into the Siamese network, including adjusting the image resolution, normalizing pixel values, etc. Then, the processed image blocks are input into the second branch of the Siamese network. The second branch of the Siamese network will extract deep feature vectors that can describe the target features from the image blocks through a series of convolutional layers and pooling layers. These feature vectors not only contain visual information such as the color and texture of the target, but also contain higher-level semantic information, which can effectively distinguish the target from the background or other interferences.

[0100] By using a pre-trained deep convolutional neural network (DCNN) as a feature extractor, we can ensure that the extracted features are highly discriminative and robust, and maintain good tracking effects even in complex and changing environments.

[0101] In addition, due to the high efficiency of the Siamese network, this process can be completed in a short time, supporting the needs of real-time tracking. Through this step, high-quality feature representation is provided for the subsequent target position calculation, ensuring the smooth progress of the tracking process.

[0102] S5: Calculate the final position of the target based on the first similarity between the feature vector of each candidate region and the feature vector of the target template.

[0103] Specifically, step S5 includes the following steps:

[0104] S51: For each particle in the current frame, calculate the similarity between the feature vector of the corresponding candidate region and the feature vector of the target template to obtain a first similarity.

[0105] In this embodiment, similarity calculation is usually achieved by measuring the distance between two feature vectors. Considering that the feature vector may have high-dimensional characteristics, cosine similarity is used as a similarity metric because cosine similarity is not affected by the length of the feature vector and can better reflect the directional difference between vectors, thereby better measuring the similarity of features. By calculating the similarity between each candidate region and the target template, the degree of matching between each candidate region and the target can be quantified, providing a basis for subsequent weight allocation.

[0106] In implementation, the feature vector of each candidate region is first obtained from the second branch of the Siamese network. Then, the cosine similarity formula is used to calculate the similarity between the feature vector of the candidate region and the feature vector of the target template.

[0107] S52: assigning a weight to each particle according to the first similarity, wherein the similarity is proportional to the weight.

[0108] In this embodiment, the principle of weight allocation is that the higher the similarity, the greater the weight the particle obtains. This weight mechanism reflects the degree of match between the particle and the target, and particles with high weights are more likely to be close to the true position of the target. In this way, particles with high match with the target can be effectively highlighted, and the influence of particles with low match with the target can be reduced, thereby improving the accuracy of target position estimation. In addition, the introduction of weights also provides a basis for the subsequent resampling steps, ensuring that the resampling process can focus on high-quality particles more efficiently.

[0109] During implementation, the similarity value is converted to a value between 0 and 1 through normalization. The normalization process can ensure that the sum of the weights of all particles is one, which is convenient for subsequent calculations.

[0110] S53: resampling the particle set based on the weight of the particles to generate a new particle set.

[0111] In this embodiment, the particle set is resampled based on the weight of the particle to generate a new particle set. The purpose of resampling is to remove low-weight particles and increase the number of high-weight particles to optimize the quality of the particle set. This process can be seen as a "survival of the fittest" process, which helps the particle set to be more closely distributed around the true position of the target. By resampling, not only can the noise in the particle set be reduced and the purity of the particle set be improved, but it can also ensure that the particle set always has sufficient diversity to cover the possible movement range of the target, which is particularly important for dealing with situations such as sudden changes in direction or speed of the target.

[0112] When implementing, the roulette wheel selection method can be used for resampling. The basic idea of ​​the roulette wheel selection method is that the probability of a particle being selected is proportional to its weight. The specific steps are as follows:

[0113] Calculate the probability of each particle being selected.

[0114] Generate a random number between 0 and 1.

[0115] Accumulate the probability of particles until the accumulated value exceeds the random number, and select the corresponding particle.

[0116] Repeat the above process until the desired number of particles are selected.

[0117] S54: Calculate the final position of the target by weighted average method according to the positions and weights of the particles in the new particle set.

[0118] In this embodiment, the final position of the target is obtained by weighted summing the positions of all particles, where particles with higher weights contribute more to the final position. The weighted average method not only considers the position information of each particle, but also combines the weights of the particles, so it can more accurately reflect the true position of the target. The target position calculated in this way not only has high accuracy, but also can better adapt to the movement mode of the target in different situations, thereby achieving effective tracking of low, slow and small targets.

[0119] S6: dynamically update the target template using the final position of the target in the current frame, and set the final position as the starting position for tracking in the next frame, and repeat the previous steps to achieve continuous tracking of the target in the next frame.

[0120] Specifically, step S6 includes the following steps:

[0121] S61: After obtaining the final position of the target in the current frame, the target template is updated according to the final position.

[0122] In this embodiment, after the final position of the target in the current frame is determined, the next task is to update the target template. The updating method can be to completely replace the old template, or to adopt a smooth updating strategy to gradually merge the information of the new and old templates.

[0123] In this embodiment, a progressive updating method is adopted, that is, according to the final position of the target in the current frame, a new image block of the target is extracted, and it is mixed with the existing target template in a certain proportion to generate a new target template. The advantage of this is that it can avoid tracking failures caused by sudden changes in the target, while retaining the continuity of the template, which helps to improve the stability and robustness of tracking.

[0124] S62: After the update, the initial state of the particle filter in the next frame is set based on the final position and a prediction is performed.

[0125] In this embodiment, after updating the target template, it is necessary to set the initial state prediction of the particle filter in the next frame. This step is mainly to provide a reasonable starting point for tracking the next frame, thereby speeding up the search speed and improving the accuracy.

[0126] In this embodiment, the initial state prediction of the particle filter in the next frame is set to be based on the final position of the target in the current frame. This means that in the next frame, the initial distribution of particles will be concentrated near the current target position, rather than randomly distributed throughout the image. This can effectively narrow the search range, reduce unnecessary computing resource consumption, and also increase the possibility of finding the target.

[0127] In addition, the target's movement direction and speed information can be combined to further optimize the initial distribution of particles to make it more consistent with the target's actual movement trend.

[0128] S63: Repeat the entire tracking process from target template generation to target final position calculation until the video sequence ends, thereby achieving continuous tracking of low, slow and small targets in the entire video sequence.

[0129] In this embodiment, in order to realize continuous tracking of low, slow and small targets in the entire video sequence, it is necessary to repeatedly execute the entire tracking process from target template generation to target final position calculation. This is a cyclic process, and each time a frame of image is processed, the next frame is immediately prepared for processing.

[0130] In this embodiment, this process includes but is not limited to the steps of updating the target template, generating a particle set, predicting the particle state, extracting features of the candidate area, and calculating the target position. By continuously iterating these steps, the tracking system can continuously track the target even if the target undergoes complex motion or environmental changes. This loop mechanism not only ensures the continuity of tracking, but also enhances the adaptability of the system to cope with various challenging tracking scenarios. Ultimately, this method can achieve efficient and accurate tracking of low, slow, and small targets in the entire video sequence.

[0131] Embodiment 2

[0132] The present invention also provides a low-speed, slow, small target tracking system for executing a low-speed, slow, small target tracking method, referring to Figure 2 As shown, the tracking system includes:

[0133] The feature extraction module 100 is used to select a target from the first frame image of the video sequence, and perform feature extraction on the selected target to generate a target template and a feature vector thereof.

[0134] The particle set generation module 200 is used to process the target template using the first branch of the Siamese network to generate an initial particle set, wherein each particle in the initial particle set represents a candidate position and a candidate state of the target.

[0135] The particle state prediction module 300 is used to predict the new position and new state of each particle in the next frame using a particle filter according to the particle state and motion model of the previous frame.

[0136] The candidate region feature extraction module 400 is used to extract at least one candidate region corresponding to each particle from the current frame according to the predicted new position and new state of the particle in the next frame, and extract the feature vector of each candidate region using the second branch of the Siamese network.

[0137] The target position calculation module 500 is used to calculate the final position of the target based on the first similarity between the feature vector of each candidate area and the feature vector of the target template.

[0138] The tracking loop module 600 is used to dynamically update the target template using the final position of the target in the current frame, and set the final position as the starting position for tracking the next frame, and repeat the previous steps to achieve continuous tracking of the target in the next frame.

[0139] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0141] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or includes elements that are inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.

Claims

1. A low, slow, small target tracking method, characterized in that: The tracking method comprises the following steps: Select a target from the first frame of the video sequence, extract features of the selected target, and generate a target template and its feature vector; Processing the target template using a first branch of a Siamese network to generate an initial particle set, wherein each particle in the initial particle set represents a candidate position and a candidate state of the target; For each particle, based on the particle state and motion model of the previous frame, a particle filter is used to predict the new position and state of the particle in the next frame. According to the predicted new position and new state of the particle in the next frame, at least one candidate region corresponding to each particle is extracted from the current frame, and the feature vector of each candidate region is extracted using the second branch of the Siamese network; Calculating the final position of the target based on a first similarity between the feature vector of each candidate region and the feature vector of the target template; Dynamically update the target template using the final position of the target in the current frame, set the final position as the starting position for tracking the next frame, and repeat the previous steps to achieve continuous tracking of the target in the next frame; The step of calculating the final position of the target based on the first similarity between the feature vector of each candidate region and the feature vector of the target template includes: For each particle in the current frame, the similarity between the feature vector of the corresponding candidate region and the feature vector of the target template is calculated to obtain a first similarity; According to the first similarity, a weight is assigned to each particle, wherein the similarity is proportional to the weight; Resample the particle set based on the particle weight to generate a new particle set; According to the positions and weights of the particles in the new particle set, the final position of the target is calculated by the weighted average method.

2. A low, slow, small target tracking method according to claim 1, characterized in that: The step of selecting a target from the first frame of a video sequence, extracting features of the selected target, and generating a target template and a feature vector thereof includes: Select the target in the first frame of the video sequence; Applying a feature extraction algorithm to the selected target to extract target features of the target; A target template is constructed according to the extracted target features, and the target template is converted into a feature vector form to obtain a feature vector of the target template.

3. A low, slow, small target tracking method according to claim 2, characterized in that: The first branch of the Siamese network is used to process the target template to generate an initial particle set, wherein each particle in the initial particle set represents a candidate position and a candidate state of the target, including: Inputting the target template into the first branch of the Siamese network, and using the first branch of the Siamese network to perform feature learning and matching on the target template, and evaluating the second similarity between the target template and the potential candidate target; According to the second similarity, generating a set of initial particles in the current frame of the video sequence, wherein each particle represents a potential candidate position and candidate state; The particle distribution in the initial particle set is initialized according to the feature vector of the target template and the expected motion pattern.

4. A low, slow, small target tracking method according to claim 3, characterized in that: For each particle, according to the particle state and motion model of the previous frame, a particle filter is used to predict the new position and new state of the particle in the next frame, including: Reading the particle state of each particle in the previous frame, wherein the particle state includes position, velocity and acceleration; Based on the model mapping table, a motion model is selected according to the target tracking scenario; Taking the particle state of the previous frame and the selected motion model as input, a particle filter is applied to perform state prediction; The particle filter iteratively updates the position and state of the particles to estimate the new position and state of each particle in the next frame.

5. A low, slow and small target tracking method according to claim 4, characterized in that: The method extracts at least one candidate region corresponding to each particle from the current frame according to the predicted new position and new state of the particle in the next frame, and extracts a feature vector of each candidate region using the second branch of the Siamese network, including: For each particle in the current frame, one or more candidate regions are determined based on the predicted new position and new state; For each candidate region, a corresponding image block is extracted from the current frame using a predetermined transformation method; Each extracted image block is input into the second branch of the Siamese network, and the second branch of the Siamese network is used to extract features of each image block to generate a corresponding feature vector.

6. A low, slow, small target tracking method according to claim 5, characterized in that: The target template is dynamically updated using the final position of the target in the current frame, and the final position is set as the starting position of the next frame tracking, and the previous steps are repeated to achieve continuous tracking of the target in the next frame, including: After obtaining the final position of the target in the current frame, updating the target template according to the final position; After the update, the initial state of the particle filter in the next frame is set based on the final position and predicted; The entire tracking process from target template generation to target final position calculation is repeated until the end of the video sequence, thereby achieving continuous tracking of low, slow and small targets in the entire video sequence.

7. A low-speed, slow, small target tracking system, used to execute a low-speed, slow, small target tracking method as claimed in any one of claims 1 to 6, characterized in that: The tracking system includes: The feature extraction module is used to select a target from the first frame image of the video sequence, extract features from the selected target, and generate a target template and its feature vector; A particle set generation module, used for processing the target template using the first branch of the Siamese network to generate an initial particle set, wherein each particle in the initial particle set represents a candidate position and a candidate state of the target; The particle state prediction module is used to predict the new position and new state of each particle in the next frame using a particle filter according to the particle state and motion model of the previous frame; A candidate region feature extraction module is used to extract at least one candidate region corresponding to each particle from the current frame according to the predicted new position and new state of the particle in the next frame, and extract a feature vector of each candidate region using the second branch of the Siamese network; A target position calculation module, used for calculating the final position of the target based on the first similarity between the feature vector of each candidate area and the feature vector of the target template; The tracking loop module is used to dynamically update the target template using the final position of the target in the current frame, and set the final position as the starting position for tracking the next frame, and repeat the previous steps to achieve continuous tracking of the target in the next frame.

Images