Target tracking method based on feature association and dynamic model

Abstract

The invention discloses a target tracking method based on feature correlation and a dynamic model, which comprises the following steps: firstly, converting an image acquired by a tracking system into a Lab color space, calculating a color moment to describe target color distribution, extracting texture features by using a local binary pattern (LBP), and performing initial modeling by using a Gaussian mixture model in combination with an expectation maximization (EM) algorithm; target initial features are accurately obtained; then, predicting the motion state of the target based on a uniform motion model by adopting a self-adaptive particle filter algorithm, and improving the prediction accuracy; thirdly, candidate areas are divided in the prediction area, feature correlation matching is carried out by calculating the mahalanobis distance, and a target is accurately positioned; then, calculating speed and acceleration according to target position change, and dynamically updating model parameters by combining feature matching similarity to adapt to target dynamic change; and predicting the position by using particle filtering, and re-initializing the model to recover tracking after shielding is removed.

Description

Technical Field

[0001] The present invention belongs to the technical field of target tracking, and particularly relates to a target tracking method based on feature association and dynamic model, aiming to overcome the problem of accurate target in complex real-world scenarios. Background Art

[0002] In many actual complex scenarios, achieving accurate tracking of targets is of crucial significance. For example, in the field of intelligent security, it is necessary to continuously track people or vehicles to ensure the safety of public areas; in the logistics distribution scenario, it is necessary to track transportation vehicles or goods in real time to ensure the smoothness of the logistics process. However, the complexity of the real application environment brings many challenges to target tracking. From the perspective of the target's own behavior, the target to be tracked often changes its action path and speed frequently. In urban traffic, vehicles often change lanes, accelerate or decelerate due to traffic conditions, driving habits, etc.; in crowded places, the action routes of pedestrians are also quite random. At the same time, interference factors in the environment further increase the tracking difficulty. Occlusion is extremely common. In the urban environment, there are many buildings, and the target is easily blocked by high-rise buildings, billboards, etc.; changes in light conditions will also affect tracking. From strong sunlight during the day to dim light at night, the light intensity difference is significant, and the direction and color of light will also change significantly at different times and under different weather conditions, which makes the visual features of the target change greatly, resulting in deviations in tracking algorithms based on visual features.

[0003] Early target tracking methods were mainly based on simple motion models and feature matching principles. Take the method based on Kalman filtering as an example. It predicts the target state by establishing a linear target motion model and combining measurement data. However, this method relies on the linear assumption and fixed parameters of target motion. Once the target shows complex maneuvers, such as sudden direction changes or speed changes, its prediction error will increase rapidly, and the tracking accuracy will drop sharply. Tracking algorithms based on template matching use the initial template of the target to perform matching and positioning in subsequent images. However, when the target undergoes pose changes (such as tumbling and turning of the target resulting in a change in perspective) and scale changes (when the target approaches or moves away from the tracking system), the matching degree between the template and the target will be greatly reduced, resulting in tracking failure.

[0004] With the rise of deep learning technology, tracking algorithms based on convolutional neural networks (CNNs) have gradually become a research hotspot. These algorithms leverage the powerful feature extraction ability of CNNs to improve tracking performance to a certain extent. For example, some trackers based on the Siamese network structure perform tracking by learning the similarity between the target and candidate regions. However, such algorithms have many limitations. First, they usually have a high computational complexity, demanding high computational power and memory of hardware devices, and it is difficult to meet the real-time requirements of reconnaissance weapons. Second, deep learning models rely heavily on training data. If the training data coverage is incomplete, the generalization ability of the model is insufficient when encountering unseen scenarios or target changes. In addition, in extremely complex battlefield environments, such as severe occlusion and strong background interference, CNN-based trackers are prone to misjudgment and tracking failure, and cannot guarantee the stability and reliability of tracking.

[0005] The deficiencies of the above existing technologies seriously affect the actual application effect of the existing target tracking system and cannot meet the strict requirements of high precision, high reliability, and real-time of target tracking in actual scenarios. It is urgent to study a target tracking method that can effectively adapt to complex environments. Summary of the Invention

[0006] The present invention aims to overcome the defects of the existing technology and proposes a target tracking method based on feature association and dynamic model. By establishing the association relationship between target features and combining with a dynamically updated tracking model, stable and accurate tracking of the target is achieved to meet the actual application requirements.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions: A target tracking method based on feature association and dynamic model, and the implementation steps are as follows:

[0008] Step 1: Extract features of the target in the acquired initial image. Use color moments to describe the color distribution features of the target, and adopt local binary pattern (LBP) to extract the texture features of the target. Based on these features, construct an initial target tracking model, and use Gaussian mixture model (GMM) to model the feature distribution of the target.

[0009] Step 2: Use the adaptive particle filter algorithm to predict the motion state of the target. Initialize the particle set of the particle filter according to the historical position and velocity information of the target. During the prediction process, dynamically adjust the weights and quantities of the particles according to the uncertainty of the target motion, and predict the possible position area of the target at the next moment.

[0010] Step 3: Extract the features of the candidate regions within the predicted target position area, associate and match them with the initially established target feature model. Measure the similarity between the two by calculating the Mahalanobis distance between the features of the candidate regions and the target model features, and select the candidate region with the smallest distance as the current position estimate of the target;

[0011] Step 4: During the tracking process, continuously monitor the changes in the features and motion states of the target, calculate the motion speed and acceleration of the target. If the change in the motion state of the target exceeds a certain threshold or the similarity of feature matching is lower than the set value, re-extract the features of the target, update the parameters of the Gaussian mixture model, and at the same time adjust the relevant parameters of the particle filter algorithm to adapt to the dynamic changes of the target;

[0012] Step 5: Determine whether occlusion occurs by comparing the feature differences between the target region and adjacent regions and the coherence of the target motion. When occlusion is judged, suspend the model update, and use the prediction results of the particle filter to continue tracking the possible position of the target. Once the occlusion is lifted, re-extract the target features, re-initialize the relevant parameters of the Gaussian mixture model and the particle filter algorithm, and resume normal tracking.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] First, extract the color moments and LBP texture features of the target in the initial image to construct the initial tracking model, comprehensively and accurately depict the initial features of the target. Subsequently, use the adaptive particle filter algorithm to predict the target position and determine the current position estimate, effectively combine the target motion information, adapt to the complex motion of the target, and improve the accuracy of position prediction and positioning. Then, dynamically update the model parameters according to the motion state and feature changes of the target, so that the tracking model can keep up with the dynamic changes of the target in time, ensuring the stability and reliability of tracking. Finally, identify occlusion by calculating feature differences and judging motion coherence and take corresponding measures, effectively coping with occlusion interference and still being able to maintain the tracking of the target under occlusion conditions. The present invention has the characteristics of high precision, strong stability and good adaptability, enabling the tracking system to track the target efficiently and stably in complex scenarios. Description of the Drawings

[0015] Figure 1 is the implementation flowchart of the target tracking method based on feature association and dynamic model of the present invention;

[0016] Figure 2 is the flowchart of initial feature extraction and model initialization of the present invention. Detailed Embodiments

[0017] The following further describes the embodiments and effects of the present invention in detail with reference to the drawings.

[0018] AsFigure 1 As shown in the figure, the object tracking method based on feature association and dynamic model of the present invention comprises the following steps:

[0019] Step 1: As Figure 2 shown, to accurately extract the target color features of a color RGB image, it is converted to the Lab color space, which consists of three channels: luminance (L), color opponent dimensions (a, b). For color features, a color moment feature vector is constructed by calculating the first moment (mean), second moment (variance), and third moment (skewness) of the image in the L, a, and b channels to describe the target color distribution. Taking the L channel as an example, the first moment (mean) where N is the total number of image pixels, the mean reflects the average color brightness, the second moment (variance) is used to measure the degree of dispersion of color values, and the third moment (skewness) describes the asymmetry of color distribution. Arranging these three moments of the three channels forms a color moment feature vector. For texture features, the local binary pattern (LBP) operator is used in the target area. Taking a certain pixel p as the center, P pixels in a circular neighborhood with a radius of R are taken, and the gray value g i of the neighborhood pixels is compared with the gray value g c of the central pixel, and the expression formula is:

[0020]

[0021] Based on this, an LBP code is generated The histogram of the LBP codes of all pixels in the target area is statistically calculated as the target texture feature. Then, based on the extracted color moments and LBP texture features, the Gaussian mixture model (GMM) is used to model the target features. The GMM consists of K Gaussian distributions, and the probability density function expression formula of each Gaussian distribution is:

[0022]

[0023] where x is a d-dimensional feature vector composed of color moments and LBP texture features, μ k is the mean vector, which determines the position of the distribution center, and ∑ k is the covariance matrix, which affects the shape of the distribution and the dimensional correlation. The weight of each Gaussian distribution is ω k , and represents its contribution degree to the mixture model. Then, the expectation maximization (EM) algorithm is used to estimate these parameters to complete the model initialization. The EM includes the E step and the M step. In the E step, the posterior probability i that each data point x belongs to the k-th Gaussian distribution is calculated according to the current model parameters. In the M step, the parameters are updated using the results of the E step, and the weight is updated to The mean is updated to The covariance is updated to Continuously repeat the E-step and M-step until the model parameters converge (the parameter change is less than the preset threshold), and complete the initialization of the target tracking model;

[0024] Step 2: After completing the initial feature extraction and model initialization, in order to achieve accurate tracking of the target, the next step is to use an adaptive particle filter algorithm to predict the motion state of the target. First, define the state vector of the target as [x, y, v x , v y , where (x, y) represents the position coordinates of the target at the current moment, and (v x , v y ) represents the velocities of the target in the x and y directions. Then, initialize the particle filter. In this process, N particles are generated, and the state s i of each particle is = [x i , y i , v xi , v yi , and an initial weight is assigned to each particle Then, based on the historical position information of the target, a uniform motion model is constructed, and its state transition equation is as follows:

[0025]

[0026] In this set of equations, Δt represents the time difference between two adjacent moments, and are the accelerations of particle i in the x and y directions at time t, respectively. In the uniform motion model, the acceleration is 0. Based on this state transition equation, the state of each particle at the next moment can be predicted;

[0027] Step 3: After completing the target motion prediction through the adaptive particle filter algorithm and obtaining the possible position area of the target at the next moment, in order to accurately locate the target position within the predicted position area, feature correlation matching is required. The predicted target position area is divided into multiple candidate areas. For each candidate area, the same method as the initial feature extraction is used - extracting color moments and LBP texture features. In order to determine which candidate area is most likely to be the target position, it is necessary to calculate the similarity between the candidate area feature vector and the target model feature vector. The Mahalanobis distance D comprehensively considers the mean and covariance information of the data and can more accurately measure the similarity between the candidate area feature and the target model feature. Its calculation formula is:

[0028]

[0029] Among them, f is the feature vector of the candidate region, which is a vector composed of the color moments and LBP texture features extracted from the candidate region. It comprehensively reflects the feature information of the candidate region in terms of color and texture. μ is the mean vector of the target model features, which represents the central position of the target features in the feature space. ∑ is the covariance matrix of the target model features, which describes the correlation between the various dimensions of the target features and the degree of data dispersion;

[0030] Step 4: After completing the feature association matching and determining the estimation of the current position of the target, since the state and features of the target may change over time during the actual movement process, in order to ensure the accuracy and stability of subsequent tracking, it is necessary to dynamically update the tracking model. Based on the current position (x t , y t ) of the target and the position (x t-1 , y t-1 ) at the previous moment, calculate the motion state information of the target. First, calculate the velocities of the target in the x and y directions. The calculation formulas for velocities v x and v y are as follows:

[0031]

[0032] Among them, Δt represents the time interval between two adjacent moments. Then, further calculate the accelerations based on the calculated velocities. The calculation formulas for accelerations a x and a y are as follows:

[0033]

[0034] Among them, and are the velocities of the target in the x and y directions calculated at the previous moment. To determine whether it is necessary to update the tracking model, set the change thresholds T v and T a for velocities and accelerations, as well as the feature matching similarity threshold T s . The velocity change threshold T v is used to measure whether the change in the target velocity between adjacent moments is too large. The acceleration threshold T a is used to determine whether the target acceleration exceeds the normal range. The feature matching similarity threshold T s is based on similarity measurement methods such as the Mahalanobis distance calculated in the feature association matching step. When any one of the thresholds is exceeded, the model update is triggered. At this time, it is necessary to re-extract the color moments and LBP texture features of the current position of the target. Then, use the EM algorithm to update the parameters of the Gaussian mixture model. At the same time, according to the new motion state of the target, adjust the parameters of the state transition model and the observation model of the particle filter algorithm;

[0035] Step 5: After the dynamic model is updated to adapt the tracking model to the dynamic changes of the target, during the actual tracking process, the target may still be occluded. Therefore, an effective occlusion handling and recovery mechanism is required. To determine whether the target is occluded, first calculate the feature difference between the target region and the adjacent region. Here, the method of calculating the sum d of the Euclidean distances of the color moment and the LBP texture feature is adopted. Let the color moment feature vector of the target region be C t =(c t1 , c t2 , …, c tn ), and the color moment feature vector of the adjacent region be C n =(c n1 , c n2 , …, c nn ). The Euclidean distance d c between them is calculated by the formula:

[0036]

[0037] Similarly, let the LBP texture feature vector of the target region be L t =(l t1 , l t2 , …, l tm ), and the LBP texture feature vector of the adjacent region be L n =(l n1 , l n2 , …, l nm ). The Euclidean distance d l between them is calculated by the formula:

[0038]

[0039] Then the feature difference is d = d c + d l . At the same time, set an occlusion judgment threshold T d . This threshold is an empirical value determined according to a large number of experiments and actual application scenarios, and is used to distinguish the feature difference between the target region and the adjacent region under normal circumstances and the feature difference when the target is occluded. If d > T d , it means that the feature difference between the target region and the adjacent region exceeds the normal range, and it may be caused by the target being occluded. In addition, it is also necessary to comprehensively judge in combination with the coherence of the target movement. In step 4 of the dynamic model update, the speed v x , v y and acceleration a x , a y of the target are calculated. If the coherence of the target movement is abnormal, that is, the speed change exceeds the speed change threshold T vor the acceleration exceeds the acceleration threshold T a , and d > T is satisfied d When this occurs, it is determined that occlusion has occurred. After determining that the target is occluded, in order to avoid the influence of incorrect information during occlusion on the tracking model, the update of the parameters of the Gaussian mixture model and the particle filter algorithm is paused, and only the prediction result of the particle filter algorithm (step 2) is used to continue tracking the possible position of the target. Once the occlusion is lifted, the target features are re-extracted, the relevant parameters of the Gaussian mixture model and the particle filter algorithm are re-initialized, and normal tracking is resumed.

[0040] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. Obviously, for professionals in the field, after understanding the content and principle of the present invention, various modifications and changes in form and details may be made without departing from the principle and structure of the present invention. However, these modifications and changes based on the idea of the present invention are still within the protection scope of the claims of the present invention.

Claims

1. A target tracking method based on feature association and dynamic model, characterized in that It includes the following steps: Step 1: To accurately extract the target color features of a color RGB image, it is converted to the Lab color space, which consists of three channels: luminance L, color opponent dimensions a, and b. For color features, a color moment feature vector is constructed by calculating the first moment (i.e., mean), second moment (i.e., variance), and third moment (i.e., skewness) of the image in the L, a, and b channels to describe the target color distribution. Taking the L channel as an example, the first moment where N is the total number of image pixels, the mean reflects the average brightness of the color, and the second moment is used to measure the degree of dispersion of the color values, and the third moment describes the asymmetry of the color distribution. Arranging these three moments of the three channels forms a color moment feature vector. For texture features, the Local Binary Pattern (LBP) operator is used in the target area. Taking a certain pixel p as the center, P pixels in a circular neighborhood with a radius of R are taken, and the gray values g i of the neighborhood pixels are compared with the gray value g c of the central pixel. The representation formula is: Generate LBP codes accordingly Statistically calculate the LBP code histogram of all pixels within the target area as the target texture feature. Then, based on the extracted color moments and LBP texture features, a Gaussian mixture model (GMM) is used to model the target features. The GMM consists of K Gaussian distributions, and the expression formula for the probability density function of each Gaussian distribution is: Among them, x is a d-dimensional feature vector composed of color moments and LBP texture features, and μ k is the mean vector, which determines the position of the distribution center. Σ k is the covariance matrix, which affects the distribution shape and dimensional correlation. The weight of each Gaussian distribution is ω k , and ω k ≥0, indicating its contribution to the mixture model. Then, the Expectation-Maximization (EM) algorithm is used to estimate these parameters to complete the model initialization. EM includes the E-step and the M-step. In the E-step, the posterior probability that each data point x i belongs to the k-th Gaussian distribution is calculated according to the current model parameters In the M-step, the parameters are updated using the results of the E-step. The weight is updated to The mean is updated to The covariance is updated to The E-step and the M-step are continuously repeated until the model parameters converge (the parameter change is less than the preset threshold), and the initialization of the target tracking model is completed; Step 2: After completing the initial feature extraction and model initialization, in order to achieve accurate tracking of the target, the next step is to use an adaptive particle filter algorithm to predict the motion state of the target. First, define the state vector of the target as [x, y, v x , v y , where (x, y) represents the position coordinates of the target at the current moment, and (v x , v y ) represents the velocities of the target in the x and y directions. Then, initialize the particle filter. In this process, N particles will be generated, and the state of each particle Assign an initial weight to each particle Then, according to the historical position information of the target, construct a uniform motion model, and its state transition equation is as follows: In this system of equations, Δt represents the time difference between two adjacent moments. and are the accelerations of particle i in the x - direction and y - direction at time t respectively. In the uniform motion model, the acceleration is 0. Based on this state - transition equation, the state of each particle at the next moment can be predicted. Step 3: After completing the target motion prediction through the adaptive particle filtering algorithm and obtaining the possible position area of the target at the next moment, in order to accurately locate the target position within the predicted position area, feature association matching needs to be performed. The predicted target position area is divided into multiple candidate areas. For each candidate area, the same method as the initial feature extraction is used - extracting color moments and LBP texture features. To determine which candidate area is most likely to be the location of the target, it is necessary to calculate the similarity between the feature vector of the candidate area and the feature vector of the target model. The Mahalanobis distance D comprehensively considers the mean and covariance information of the data and can more accurately measure the similarity between the candidate area feature and the target model feature. Its calculation formula is: where f is the feature vector of the candidate area, which is a vector composed of the color moments and LBP texture features extracted from this candidate area, and it comprehensively reflects the feature information of the candidate area in terms of color and texture. μ is the mean vector of the target model features, which represents the central position of the target features in the feature space. Σ is the covariance matrix of the target model features, which describes the correlation between the various dimensions of the target features and the degree of data dispersion; Step 4: After completing the feature association matching and determining the estimation of the target's current position, since the state and features of the target may change over time during the actual movement, in order to ensure the accuracy and stability of subsequent tracking, the tracking model needs to be dynamically updated. Based on the target's current position (x t , y t ) and the position at the previous moment (x t-1 , y t-1 ), calculate the motion state information of the target. First, calculate the velocity of the target in the x and y directions. The calculation formulas for velocities v x and v y are as follows: where Δt represents the time interval between two adjacent moments. Then, the acceleration is further calculated based on the calculated speed. The acceleration a x and a y are calculated according to the following formulas respectively: Among them, and are the velocities of the target in the x and y directions calculated at the previous moment. In order to determine whether it is necessary to update the tracking model, change thresholds T v and T a for velocity and acceleration, as well as a feature matching similarity threshold T s are set. The velocity change threshold T v is used to measure whether the change in the target velocity between adjacent moments is too large, and the acceleration threshold T a is used to determine whether the target acceleration exceeds the normal range. The feature matching similarity threshold T s is based on similarity measurement methods such as the Mahalanobis distance calculated in the feature association matching step. When any of the thresholds is exceeded, the model update is triggered. At this time, it is necessary to re-extract the color moments and LBP texture features of the current position of the target. Then, the parameters of the Gaussian mixture model are updated using the EM algorithm. At the same time, according to the new motion state of the target, the parameters of the state transition model and the observation model of the particle filter algorithm are adjusted; Step 5: After the dynamic model is updated to adapt the tracking model to the dynamic changes of the target, during the actual tracking process, the target may still be occluded. Therefore, an effective occlusion handling and recovery mechanism is required. To determine whether the target is occluded, first calculate the feature difference between the target region and the adjacent regions. Here, the method of calculating the sum d of the Euclidean distances of the color moment and the LBP texture feature is adopted. Let the color moment feature vector of the target region be C t =(c t1 ,c t2 ,…,c tn ), and the color moment feature vector of the adjacent region be C n =(c n1 ,c n2 ,…,c nn ). The calculation formula for their Euclidean distance d c is as follows: Similarly, let the LBP texture feature vector of the target region be L t =(l t1 ,l t2 ,…,l tm ), and the LBP texture feature vector of the adjacent region be L n =(l n1 ,l n2 ,…,l nm ). The calculation formula for the Euclidean distance d l between them is as follows: Then the feature difference is d = d c + d l , and at the same time, a occlusion judgment threshold T d is set. This threshold is an empirical value determined based on a large number of experiments and actual application scenarios, and is used to distinguish the feature differences between the target area and the adjacent area under normal circumstances and the feature differences when the target is occluded. If d > T d , it indicates that the feature difference between the target area and the adjacent area exceeds the normal range, which may be caused by the occlusion of the target. In addition, it is necessary to comprehensively judge in combination with the coherence of the target movement. In step 4 of the dynamic model update, the speed v x 、v y and acceleration a x 、a y of the target are calculated. If the coherence of the target movement is abnormal, that is, the speed change exceeds the speed change threshold T v or the acceleration exceeds the acceleration threshold T a , and d > T d is satisfied, it is judged that occlusion occurs. When it is judged that the target is occluded, in order to avoid the influence of incorrect information during occlusion on the tracking model, the parameter updates of the Gaussian mixture model and the particle filter algorithm are suspended, and only the prediction results of the particle filter algorithm are used to continue tracking the possible position of the target; once the occlusion is removed, the target features are re-extracted, the relevant parameters of the Gaussian mixture model and the particle filter algorithm are re-initialized, and normal tracking is restored.

Images