Target tracking method, system and device based on multi-level loss recovery mechanism

Abstract

The application discloses a target tracking method, system and device based on a multi-level loss recovery mechanism, comprising the following steps: collecting a video frame sequence and initial target region information; using a kernel correlation filter to perform target tracking on a current video frame, extracting HOG features of the target and generating a response map; using a Kalman filter to correct the motion state of the tracking result of the kernel correlation filter and outputting a corrected target position; evaluating the tracking confidence of the current target, and when the reliability evaluation result is lower than a preset threshold, starting a backup template mechanism for secondary verification; when a plurality of backup templates in succession fail, triggering a target loss redetection mechanism to redetect the target, obtaining a candidate region of the target, and performing multi-scale search and correction on the redetected candidate region to recover normal tracking of the target. The application can realize high-robustness and real-time embedded target tracking.

Description

Technical Field

[0001] This invention relates to the field of computer vision tracking technology, specifically to a target tracking method, system, and apparatus based on a multi-level loss recovery mechanism. Background Technology

[0002] With the rapid development of computer vision technology, target tracking has become one of the core technologies in fields such as intelligent monitoring, autonomous driving, human-computer interaction, and mobile robotics. Especially in embedded devices and mobile platform applications, higher demands are placed on the real-time performance, robustness, and resource efficiency of target tracking algorithms.

[0003] The kernelized correlation filter (KCF) algorithm is widely used in real-time target tracking due to its high computational efficiency and simple implementation. KCF performs correlation filtering operations within a cyclic matrix structure and utilizes the fast Fourier transform to accelerate computation, achieving real-time performance while maintaining high tracking accuracy.

[0004] However, KCF is still prone to problems such as target drift, tracking loss, and template contamination in complex scenarios, significantly weakening the stability and robustness of tracking. Specifically:

[0005] 1. Traditional KCF algorithm relies solely on the peak height of the response map as the single criterion for judging tracking quality. When the target is occluded, moves rapidly, or changes in appearance, the response map may show a multi-peak distribution or a decrease in peak value. However, a single indicator cannot effectively distinguish between target deformation and tracking failure, which is not conducive to a comprehensive assessment of the tracking status.

[0006] 2. Traditional KCF algorithms employ a per-frame update strategy. When the target is occluded or undergoes drastic changes, error information rapidly contaminates the template, leading to a continuous decline in template quality. Once the primary template is contaminated, subsequent tracking becomes difficult to repair, lacking an effective protection mechanism to maintain template quality. Furthermore, existing methods lack backup template design, failing to provide a reliable alternative when the primary template fails.

[0007] 3. When the target moves rapidly, changes direction suddenly, or its appearance changes, the KCF algorithm is prone to positioning errors and trajectory jitter. Because KCF performs response map peak localization independently in each frame, it lacks constraints on the continuity of target motion. When the target moves at high speed or changes direction abruptly, the search area may not cover the actual position of the target, leading to tracking failure.

[0008] 4. When the target is occluded for a long time or completely lost, the traditional KCF algorithm lacks an effective re-detection mechanism. Although some improved algorithms introduce re-detection modules, they usually adopt a full-graph dense search approach, which has a large computational overhead and is difficult to achieve real-time response in resource-constrained environments such as embedded devices. At the same time, it is difficult to achieve a balance between the accuracy and computational complexity of re-detection, which affects the overall performance of the system.

[0009] To improve performance, many studies have attempted to introduce mechanisms such as spatial regularization, multi-feature fusion, or deep learning. However, these methods typically involve high computational overhead, making it difficult to meet the real-time requirements of embedded devices. Furthermore, most algorithms still rely on a single metric to evaluate tracking quality, lacking a multi-dimensional comprehensive judgment mechanism. Template update strategies also lack effective loss recovery designs; once the main template is contaminated, it is difficult to repair it in a timely manner. The re-detection module also struggles to achieve a balance between detection accuracy and computational complexity, especially in resource-constrained environments, making it even more difficult to guarantee real-time response.

[0010] Therefore, given the existing problems in the technology, there is an urgent need to propose a target tracking method based on a multi-level loss recovery mechanism that can maintain high robustness in complex scenarios. Summary of the Invention

[0011] The purpose of this invention is to provide a target tracking method, system, and apparatus based on a multi-level loss recovery mechanism, thereby solving the aforementioned problems existing in the prior art.

[0012] To address the aforementioned technical problems, in a first aspect, the present invention provides a target tracking method based on a multi-level loss recovery mechanism, comprising the following steps:

[0013] Acquire video frame sequences and initial target area information;

[0014] The target is tracked using a kernel correlation filter in the current video frame, and the HOG features of the target are extracted and a response map is generated.

[0015] The Kalman filter is used to correct the motion state of the tracking results of the kernel correlation filter, and the corrected target position is output.

[0016] Assess the tracking confidence of the current target. When the tracking confidence level is at this point When the tracking confidence level is greater than or equal to a first preset threshold, the current tracking is determined to be reliable, and the tracking result of the target is output and updated; when the tracking confidence level is... If the value is less than the first preset threshold, the backup template mechanism will be activated to restore the target.

[0017] Reassess the tracking confidence of the recovered target. When the tracking confidence level is at this point When the tracking confidence level is greater than or equal to the first preset threshold, the recovered target bounding box is determined to be reliable, and the tracking result of the recovered target is output and updated; when the tracking confidence level is... If the value is still less than the first preset threshold, the failure counter is incremented by 1 and the filter state is not updated. When the backup template verification fails for multiple consecutive frames, the target loss re-detection mechanism is triggered to re-detect the target, obtain the candidate region of the target, and perform multi-scale search and correction on the re-detected candidate region to restore normal tracking of the target.

[0018] Furthermore, the kernel function used in the kernel correlation filter is a Gaussian radial basis function kernel.

[0019] Furthermore, motion state correction includes:

[0020] The target state is modeled using a six-dimensional state vector. ,in( ( ) represents the center coordinates of the target. These are the width and height of the target, respectively; These are the target's velocities in the x and y directions, respectively;

[0021] A target motion model is established using a Kalman filter, and the position of the current video frame is predicted using a state transition matrix.

[0022] The tracking results of the kernel correlation filter are used as observations, and the motion velocity components of the target state are smoothed using a sliding window.

[0023] The observation residuals are calculated based on the smoothed observation vectors, and the Kalman gain is calculated by combining the prediction covariance matrix and the observation noise covariance matrix.

[0024] The predicted position is corrected by posterior estimation using Kalman gain, and the corrected target position is output.

[0025] Furthermore, the tracking confidence The evaluation model is as follows:

[0026] ;

[0027] in, For the tracking confidence of the response graph, The peak signal-to-noise ratio of the response plot, where K is a constant. The response entropy of the response graph, The correlation coefficient between the corrected target location HOG features and the historical template HOG features is given. , and These are the weights.

[0028] Furthermore, the backup template mechanism includes:

[0029] When the tracking confidence of the current target When the current tracking is greater than or equal to the first preset threshold, the tracking result of the kernel correlation filter is used as the main template for continuous updating, and the current frame is saved as a historical high-reliability frame as a backup template.

[0030] When the tracking confidence of the current target If the value is less than the first preset threshold, the target's response map is recalculated based on the backup template, and the peak value of the backup template's response map is compared with the peak value of the main template's response map. If the peak value of the backup template's response map is greater than the peak value of the main template's response map, the recovered target box is determined to be more reliable, and the tracking confidence of the recovered target is reassessed. ;

[0031] When the tracking confidence of the recovered target If the value is greater than or equal to the first preset threshold, the recovered target is determined to be more reliable, and the tracking result of the recovered target is output and updated.

[0032] Furthermore, the target loss re-detection mechanism adopts a two-stage search strategy, which includes: in the rapid screening stage, firstly, the search area is expanded with the predicted location as the center, and the color histogram and HOG features of the candidate area are extracted using a sliding window with a step size of s1; then, the color area overlap rate and HOG feature correlation between the candidate area and the historical target template are calculated, and candidate areas that meet the preset conditions in terms of color area overlap rate and HOG feature correlation are initially screened.

[0033] In the fine-grained matching stage, a local search region is constructed centered on the candidate regions selected quickly. A sliding window with a step size of s2 is used for fine-grained matching, and the best candidate region is selected based on the color area overlap rate and HOG feature correlation; where s2 <s1。

[0034] Furthermore, multi-scale corrections include:

[0035] After fine matching is completed, multiple search windows of different scales are constructed for the best candidate regions output.

[0036] Recalculate the fusion confidence of HOG feature correlation and color area overlap rate for each scale window;

[0037] Select the scale with the highest fusion confidence as the current scale of the target to restore normal tracking.

[0038] Secondly, the present invention provides a target tracking system based on a multi-level loss recovery mechanism, comprising:

[0039] The information acquisition module is used to acquire video frame sequences and initial target area information;

[0040] The feature extraction module is used to perform target tracking on the current video frame using a kernel correlation filter, extract the target's HOG features, and generate a response map;

[0041] The motion correction module is used to correct the motion state of the target by using the Kalman filter to correct the tracking results of the kernel correlation filter, and outputs the corrected target position.

[0042] The reliability assessment module is used to calculate the peak signal-to-noise ratio (PSR) and response entropy of the response graph. And the correlation coefficient between the corrected target location HOG features and the historical template HOG features. And based on the peak signal-to-noise ratio (PSR) and response entropy and correlation coefficient Construct a multi-dimensional reliability assessment model to calculate the target tracking confidence. ;

[0043] The backup template mechanism startup module is used to assess the tracking confidence of the current target. When the tracking confidence level is greater than or equal to a first preset threshold, the current tracking is determined to be reliable, and the tracking result of the target is output and updated; when the tracking confidence level is... If the target value is less than the first preset threshold, the backup template mechanism is activated to recover the target; then the tracking confidence of the recovered target is reassessed. When the tracking confidence level is at this point When the value is greater than or equal to the first preset threshold, the recovered target bounding box is determined to be reliable, and the tracking result of the recovered target is output and updated.

[0044] The lost target re-detection module is used to assess the tracking confidence of the recovered target. If the value is still less than the first preset threshold, the failure counter is incremented by 1 and the filter state is not updated. When the backup template verification fails for multiple consecutive frames, the target loss re-detection mechanism is triggered to re-detect the target, obtain the candidate region of the target, and perform multi-scale search and correction on the re-detected candidate region to restore normal tracking of the target.

[0045] Thirdly, the present invention also provides a target tracking device based on a multi-level loss recovery mechanism, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the target tracking method provided in the first aspect above.

[0046] The beneficial effects of this invention are as follows:

[0047] 1. By constructing a multi-level loss recovery framework that includes Kalman filter motion compensation, multi-dimensional reliability assessment, backup template mechanism, and target loss re-detection, the tracking robustness and stability of the kernel correlation filter algorithm in complex scenarios are effectively improved. It is particularly suitable for real-time target tracking applications in embedded devices and mobile robot platforms.

[0048] 2. By adopting a backup template mechanism, template contamination can be effectively prevented. Through dynamic switching and update control strategies, template quality is kept stable, and the continuous decline in tracking performance caused by the accumulation of error information is avoided.

[0049] 3. By adopting a two-stage re-detection mechanism, it can successfully relocate the target after long-term occlusion or complete loss. The fast screening stage improves the coverage of candidate areas, the fine matching stage ensures the positioning accuracy, and the multi-scale correction mechanism adapts to changes in target scale. Moreover, the two-stage search strategy significantly reduces the computational cost compared to full-map dense search. The fast screening process quickly eliminates irrelevant areas, and the fine matching process accurately locates the target only within the candidate area, making it suitable for resource-constrained embedded environments.

[0050] 4. By adopting a multi-dimensional reliability assessment model, the tracking status is comprehensively evaluated from three perspectives: response map quality, distribution concentration, and feature consistency. Compared with single-index assessment, this is more accurate and reliable, providing a scientific basis for loss recovery strategies.

[0051] 5. By selecting features with low computational overhead and fast extraction speed, such as HOG features and color area overlap rate, it maintains good recognition performance while having high computational efficiency, making it particularly suitable for deployment on embedded platforms such as Jetson Nano for moving target tracking applications. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, use the same reference numerals to denote the same or similar parts. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0053] Figure 1 This is a flowchart illustrating a target tracking method according to an embodiment of the present invention;

[0054] Figure 2 This is a diagram illustrating a target tracking process based on a multi-level loss recovery mechanism according to an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of a target tracking device according to an embodiment of the present invention;

[0056] Figure 4This is a performance comparison chart of a target tracking method according to an embodiment of the present invention on the OTB100 dataset. Detailed Implementation

[0057] like Figure 1 The target tracking method based on a multi-level loss recovery mechanism, as shown, includes the following steps:

[0058] Acquire video frame sequences and initial target area information;

[0059] Kernel correlation filter is used to track targets in the current video frame, extract HOG features of the target and generate a response map. The peak position of the response map corresponds to the predicted position of the target. The core of the KCF algorithm is to use the property of cyclic matrix to transform matrix operations in the time domain into element-wise operations in the frequency domain, and combine it with fast Fourier transform to achieve efficient computation. In the feature extraction stage, HOG features are used for target description. HOG features have good local geometric invariance and illumination robustness, and can effectively capture the structural information of the target.

[0060] The Kalman filter is used to correct the motion state of the tracking results of the kernel correlation filter, and the smoothed target position is output. The Kalman filter can correct the motion state to address the problem that the KCF algorithm is prone to positioning errors when the target moves quickly or changes direction suddenly.

[0061] Assess the tracking confidence of the current target. When the tracking confidence level is at this point When the tracking confidence level is greater than or equal to a first preset threshold, the current tracking is determined to be reliable, and the tracking result of the target is output and updated; when the tracking confidence level is... If the value is less than the first preset threshold, the backup template mechanism will be activated to restore the target.

[0062] Reassess the tracking confidence of the recovered target. When the tracking confidence level is at this point When the tracking confidence level is greater than or equal to the first preset threshold, the recovered target bounding box is determined to be reliable, and the tracking result of the recovered target is output and updated; when the tracking confidence level is... If the value is still less than the first preset threshold, the failure counter is incremented by 1 and the filter state is not updated. When the backup template verification fails for multiple consecutive frames, the target loss re-detection mechanism is triggered to re-detect the target, obtain the candidate region of the target, and perform multi-scale search and correction on the re-detected candidate region to restore normal tracking of the target.

[0063] This invention employs a multi-layered target loss recovery mechanism, progressively increasing the complexity of the recovery methods based on the severity of tracking failure: the first layer is Kalman filter motion compensation, utilizing motion continuity constraints to smooth position prediction; the second layer is a backup template mechanism, activating the historically optimal template when response quality deteriorates; the third layer is a target loss re-detection mechanism, initiating a two-stage search strategy to relocate the target when multiple consecutive frames fail. All modules are unified and coordinated through a multi-dimensional reliability assessment model, which integrates three evaluation metrics: peak signal-to-noise ratio, response entropy, and HOG feature correlation, enabling real-time quantitative assessment of the tracking status and triggering of the loss recovery strategy.

[0064] According to one embodiment of this application, the kernel function used in the kernel correlation filter is a Gaussian radial basis function kernel. The Gaussian radial basis function kernel is defined as:

[0065] ;

[0066] in, For kernel bandwidth parameters, Indicates the inverse Fourier transform. Indicates element-wise multiplication; adaptively set according to feature type. Value: When using HOG features =0.6, when using grayscale features =0.2.

[0067] This application uses KCF as the basic tracking framework. In its implementation, the target region is divided into 4×4 pixel cells, the HOG features within each cell are calculated, and the gradient magnitude distribution of the nine bins is statistically analyzed. Subsequently, adjacent 2×2 cells are grouped into a block, and L2-Hys normalization is performed within the block to enhance robustness to illumination changes.

[0068] To adapt to changes in target scale, KCF introduces a multi-scale detection mechanism. During each frame's detection process, the system performs target detection at three levels: the current scale, a scale reduction of 1 / scale_step, and a scale increase of scale_step. Here, scale_step = 1.05 represents a 5% step size between adjacent scales. By comparing the peak responses at the three scales, the scale with the highest weighted response value is selected as the current frame result, where scale_weight = 0.96 is used to attenuate the weights of responses from scales other than the current one.

[0069] According to one embodiment of this application, motion state correction includes:

[0070] The target state is modeled using a six-dimensional state vector. ,in( ( ) represents the center coordinates of the target. These are the width and height of the target, respectively; These are the target's velocities in the x and y directions, respectively;

[0071] A target motion model is established using a Kalman filter, and the position of the current video frame is predicted using the state transition matrix. The Kalman filter employs a uniform motion model, assuming that the target maintains uniform linear motion within short time intervals. The time step is considered. The state transition matrix F is defined as a 6×6 matrix:

[0072]

[0073] This matrix indicates that the position of the current frame is determined by the position and velocity of the previous frame, while scale information is propagated statically.

[0074] The tracking result of the kernel correlation filter is used as the observation vector. The velocity components are smoothed using a sliding window of length N. The formula for calculating the smoothed velocity is as follows:

[0075] ;

[0076] in,( ( ) represents the center coordinates of the target. These are the width and height of the target, respectively; These represent the target's speed of movement in the x and y directions (i.e., the number of pixels moved per frame). These are the velocity estimates from historical frames; this smoothing mechanism effectively suppresses high-frequency noise in velocity estimation, providing more stable motion predictions.

[0077] Calculate the observation residuals based on the smoothed observation vectors. The Kalman gain is calculated by combining the prediction covariance matrix and the observation noise covariance matrix. ;

[0078] The predicted position is corrected by posterior estimation using Kalman gain, and the corrected target position is output.

[0079] Kalman filtering employs a classic two-step recursive process of prediction and update;

[0080] The prediction step estimates the current state based on the motion model:

[0081] ;

[0082]

[0083] in, For the prior state estimate at time k, For the posterior state estimate at time k-1, For control vectors, Here is the state transition matrix. To control the input matrix, The prior error covariance matrix;

[0084] The correction step optimizes the prediction results using observed values:

[0085]

[0086]

[0087]

[0088] in, For Kalman gain, For the observation matrix, For the posterior state estimate at time k, Let be the observation vector at time k. Let be the posterior error covariance matrix.

[0089] Calculate the observation residuals based on the smoothed observation vectors. The Kalman gain is calculated by combining the prediction covariance matrix and the observation noise covariance matrix. Process noise covariance matrix Control the level of trust in the motion model, set to ,in =0.01; The observation noise covariance matrix R controls the degree of confidence in the observations, and is set to 0.01. ,in =0.07. Relatively small. Values ​​and larger ones The value indicates a higher level of confidence in the motion model than in a single observation, which is beneficial for trajectory smoothing.

[0090] According to one embodiment of this application, confidence level is tracked. The evaluation model is as follows:

[0091] ;

[0092] in, For the tracking confidence of the response graph, The peak signal-to-noise ratio of the response plot, where K is a constant. The response entropy of the response graph, The correlation coefficient between the corrected target location HOG features and the historical template HOG features is given. , and These are the weights; the weights can be assigned as follows: =0.3, =0.2, =0.5; the K value can be 10 to saturate and normalize the PSR term to avoid excessively large values, the entropy term is inverted to ensure consistent directionality, and the HOG correlation directly uses its original value. Higher values... The weights reflect the important role of feature consistency in reliability assessment. This tracking confidence level... The evaluation model quantifies the reliability of tracking from three dimensions: response map quality, distribution concentration, and feature consistency, thereby improving the accuracy of tracking status assessment.

[0093] The formula for calculating the peak signal-to-noise ratio (PSR) of the response plot is:

[0094]

[0095] in, The maximum value of the response graph, The mean value of the sidelobe region is calculated after excluding the 5×5 neighborhood of the main peak. The standard deviation of the side lobe region. To ensure numerical stability and prevent division by zero errors, it is set to... Peak signal-to-noise ratio (PSR) is used to evaluate the significance of the main peak in the response graph, reflecting the distinction between the target and the background. The higher the PSR value, the more prominent the main peak and the more reliable the target localization.

[0096] Response entropy of the response graph The calculation formula is:

[0097] ;

[0098] in, This is the normalized response value. For numerical stability; response entropy The uncertainty of the response distribution is measured from an information theory perspective; to eliminate the influence of response maps of different sizes, the entropy value is standardized.

[0099]

[0100] in, This represents the total number of pixels in the response map. Low entropy values ​​indicate a concentrated response and a high degree of certainty regarding the target location; high entropy values ​​indicate a dispersed response and the existence of multiple possible locations.

[0101] The correlation coefficient between the corrected target location HOG features and the historical template HOG features. The calculation formula is:

[0102] ;

[0103] in, This is the HOG feature vector of the current video frame. For template HOG feature vectors, Describing covariance, The standard deviation is represented by HOG feature correlation, which directly measures the feature consistency between the target region in the current frame and the historical template. A correlation coefficient close to 1 indicates that the features are highly consistent, while a correlation coefficient close to 0 indicates that the features have changed significantly, and the target may be lost or its appearance may have changed drastically.

[0104] According to one embodiment of this application, the alternative template mechanism includes:

[0105] When the tracking confidence of the current target When the reliability score is greater than or equal to the first preset threshold (e.g., 0.5), the current tracking is determined to be reliable. The tracking result of the kernel correlation filter is used as the main template for continuous updating, and the current frame is saved as a historical high-reliability frame as a backup template. That is, only when the reliability score of the main template is... Template updates are only performed when the value is greater than or equal to the first preset threshold. The template update formula is:

[0106] ;

[0107] in, The target region features extracted for the current frame. These are the newly trained filter coefficients. The learning rate is adjusted by... Balancing the stability and adaptability of updates, a smaller learning rate is beneficial for template stability and avoids excessive impact of single-frame noise on the template. Larger sizes allow for rapid adaptation to changes in the target appearance; the main template is only updated when its own score is reliable, while the backup template always retains the state of the most recent high-quality main template.

[0108] When the tracking confidence of the current target If the value is less than the first preset threshold, the target's response map is recalculated based on the backup template, and the peak value of the backup template's response map is compared with the peak value of the main template's response map. If the peak value of the backup template's response map is greater than the peak value of the main template's response map, the recovered target is determined to be more reliable, and the tracking confidence of the recovered target is reassessed. ;

[0109] When the tracking confidence of the recovered target If the value is greater than or equal to the first preset threshold, the recovered target is determined to be more reliable, and the tracking result of the recovered target is output and updated; at this time, the update of the backup template is paused.

[0110] The backup template mechanism used in this embodiment can effectively prevent template contamination. Through dynamic switching and update control strategies, it ensures stable template quality and avoids the continuous decline in tracking performance caused by the accumulation of error information.

[0111] According to one embodiment of this application, this application introduces a continuous spare template usage counter. Track the consecutive usage counts of the backup template. When When < 0.5, Increment the counter by 1; otherwise The count is 0. When > (set to) When =2), it is determined that the target may be lost, and a target loss re-detection mechanism is implemented.

[0112] The target loss re-detection mechanism employs a two-stage search strategy, which includes:

[0113] In the rapid screening stage, the search area is first expanded with the predicted location as the center, and the color histogram and HOG features of the candidate area are extracted using a sliding window with a large step size s1. Then, the color area overlap rate and HOG feature correlation between the candidate area and the historical target template are calculated, and candidate areas that meet the preset conditions in terms of color area overlap rate and HOG feature correlation are initially screened.

[0114] In the fine-grained matching stage, a local search region is constructed centered on the candidate regions selected quickly. A sliding window with a small step size s² is used for fine-grained matching, and the optimal candidate region is selected based on color area overlap rate and HOG feature correlation. Wherein, s² <s1。

[0115] The dual-stage re-detection mechanism used in this embodiment can successfully relocate the target after it has been occluded for a long time or completely lost. The rapid screening stage improves the coverage of the candidate area, the fine matching stage ensures the positioning accuracy, and the multi-scale correction mechanism adapts to changes in the target scale.

[0116] In practice, the rapid screening phase uses the last reliable tracking location as the center to construct an expanded search area:

[0117] ;

[0118] in( , ) is the predicted center location, ( , ( ) represents the target size. =2.192 is the search area expansion factor. Within the search area, the step size is... =9 Perform a sliding window scan, extract HOG features and color area features for each candidate window, calculate the similarity with the initial template and the backup template respectively, and retain the maximum similarity between the two as the initial score of the window.

[0119] Perform dual threshold validation on the initially screened candidate regions:

[0120] ;

[0121] in, =0.5 is the HOG similarity threshold. =0.62 is the color area overlap rate threshold. The verified candidate regions are further combined with a distance penalty to recalculate the score.

[0122] ;

[0123] in, =0.8 is the weight balancing factor. Let be the Euclidean distance between the center of the candidate region and the predicted location. This is the normalization factor. Finally, the N=3 candidate regions with the highest scores are selected for the fine-tuning matching stage.

[0124] In the fine-matching stage, for each candidate region that passes the quick screening, a local search region is constructed centered on that region:

[0125] ;

[0126] in, =1.3 is the expansion factor for fine-grained search, which is smaller than the expansion factor for quick filtering. Within each local search region, the step size is... =5 performs dense scanning, calculates the comprehensive similarity score, and selects the position with the highest global score as the re-detection result.

[0127] In practice, HOG feature similarity uses normalized cosine similarity:

[0128] ;

[0129] This measurement method is sensitive to the geometry of the target and effectively reflects the consistency of the shape contour. Color area overlap is calculated using a quantized HSV color space, and may specifically include:

[0130] Convert the image to the HSV color space;

[0131] The H, S, and V channels are quantized separately, with the H channel quantized to 180 levels, the S channel to 256 levels, and the V channel to 256 levels.

[0132] Construct color identifier IDs;

[0133] Count the number of pixels for each color ID to form the color area distribution;

[0134] Calculate the color area overlap rate:

[0135]

[0136] in, Let represent the color area distribution of the candidate region and the reference template in the quantized HSV space, respectively, and k represent the quantized color identifier. This method measures the consistency of the color distribution by calculating the ratio of the overlapping area to the total area of ​​the reference template.

[0137] The formula for comprehensive similarity fusion is:

[0138]

[0139] in, =0.67 is the HOG feature weight, reflecting the dominant role of shape features.

[0140] According to one embodiment of this application, multi-scale correction includes:

[0141] After fine matching is completed, multiple search windows of different scales (such as 0.9, 1.0, and 1.1) are constructed for the best candidate region output.

[0142] Recalculate the HOG feature correlation for each scale window. Overlap rate with color area Fusion confidence ;

[0143] Choose the scale with the highest fusion confidence as the current scale of the target;

[0144] If the highest confidence level If the value is ≥ 0.65, the re-detection is considered successful.

[0145] When the number of consecutive failed re-detection attempts exceeds the maximum threshold max_failures=5, the system marks the re-detection function as disabled to avoid unnecessary computational overhead. At this point, the system reverts to the basic KCF tracing mode, awaiting manual intervention or natural recovery after a change in the scenario.

[0146] Secondly, the present invention provides a target tracking system based on a multi-level loss recovery mechanism, comprising:

[0147] The information acquisition module is used to acquire video frame sequences and initial target area information;

[0148] The feature extraction module is used to perform target tracking on the current video frame using a kernel correlation filter, extract the target's HOG features, and generate a response map;

[0149] The motion correction module is used to correct the motion state of the target by using the Kalman filter to correct the tracking results of the kernel correlation filter, and outputs the corrected target position.

[0150] The reliability assessment module is used to calculate the peak signal-to-noise ratio (PSR) and response entropy of the response graph. And the correlation coefficient between the corrected target location HOG features and the historical template HOG features. And based on the peak signal-to-noise ratio (PSR) and response entropy and correlation coefficient Construct a multi-dimensional reliability assessment model to calculate the target tracking confidence. ;

[0151] The backup template mechanism startup module is used to assess the tracking confidence of the current target. When the tracking confidence level is greater than or equal to a first preset threshold, the current tracking is determined to be reliable, and the tracking result of the target is output and updated; when the tracking confidence level is... If the target value is less than the first preset threshold, the backup template mechanism is activated to recover the target; then the tracking confidence of the recovered target is reassessed. When the tracking confidence level is at this point When the value is greater than or equal to the first preset threshold, the recovered target bounding box is determined to be reliable, and the tracking result of the recovered target is output and updated.

[0152] The lost target re-detection module is used to assess the tracking confidence of the recovered target. If the value is still less than the first preset threshold, the failure counter is incremented by 1 and the filter state is not updated. When the backup template verification fails for multiple consecutive frames, the target loss re-detection mechanism is triggered to re-detect the target, obtain the candidate region of the target, and perform multi-scale search and correction on the re-detected candidate region to restore normal tracking of the target.

[0153] The various modules in this application are coordinated through a three-dimensional reliability assessment model: Kalman filtering is executed every frame to provide motion constraints; the backup template ZAI is used when... Activated when < 0.5; re-detection mechanism when > Triggered at time 2. This layered design ensures that the system employs appropriate recovery strategies based on the degree of target loss, avoiding overcomputation.

[0154] Thirdly, the present invention also provides a target tracking device based on a multi-level loss recovery mechanism, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the target tracking method provided in the first aspect above.

[0155] like Figure 3 As shown, in addition to the memory and processor, this device can also work collaboratively with a display module, input / output interfaces, communication interfaces, power supply, communication bus, and image sensor. These components are connected via the communication bus to achieve functions such as video acquisition, feature extraction, motion state correction, and target relocalization. This device can be integrated into a mobile robot platform and adopts a modular design for easy expansion and maintenance.

[0156] The memory is a component used to store target tracking algorithm programs, operating systems, and related data. This includes: computer programs implementing algorithms such as KCF, Kalman filtering, and target re-detection; operating systems (such as Windows, Unix, and Linux) that support algorithm execution; and runtime data such as video frames, feature data, and tracking results. The memory can use flash memory or SSDs as non-volatile storage media and is equipped with high-speed interfaces and caching mechanisms to ensure efficient data read / write operations at 50+ FPS.

[0157] The processor is an embedded multi-core processing unit with CPU computing capabilities, supporting parallel feature extraction and filtering calculations. The processor executes programs in memory to perform steps such as HOG feature extraction and target response calculation based on kernel correlation filtering, motion state correction based on Kalman filtering, multi-dimensional reliability assessment (including peak signal-to-noise ratio, response entropy, and feature correlation), and backup templates and loss re-detection mechanisms. The processor may also include a main processor and a coprocessor. The main processor, also known as the CPU, processes data in the wake-up state; the coprocessor processes data in the standby state.

[0158] The display module is a component used to output target tracking results and system operating status information. It is connected to the processor via a communication bus and supports real-time result visualization or remote debugging. This module can use an embedded OLED screen or an external display interface to display content including the tracking screen and the tracked target bounding box.

[0159] The input / output interfaces are standard high-speed data interface components, including a USB interface for connecting image sensors and transmitting video data. This interface also supports the connection of external debugging devices or additional sensors, enhancing system scalability.

[0160] The communication interface is a component used for communication between the device and external devices or networks. It uses Gigabit Ethernet or wireless communication modules (such as Wi-Fi) to realize ROS message exchange and remote data transmission, thereby supporting real-time sharing of target tracking results and remote system monitoring.

[0161] The power supply is a component that provides a stable energy supply. It can be a battery pack or an external DC power supply. Through a voltage regulator module, it supplies power to core components such as processors, memory, and image sensors, ensuring the continuous and stable operation of the algorithm on the mobile robot platform.

[0162] The communication bus is a high-speed data transmission bus component used for data interaction between components such as processors, memory, image sensors, and display modules, ensuring smooth data flow for real-time target tracking.

[0163] An image sensor is a component that acquires environmental video frames or image sequences as input data for target tracking. This sensor preferably supports a video camera in RGB mode with a frame rate of at least 30fps and a resolution selectable between 640×480 and 1920×1080 to meet image quality requirements in complex environments. Through input / output interfaces, the image sensor transmits video data to a processor in real time for feature extraction and target tracking.

[0164] During operation, the image sensor continuously acquires video data. The processor extracts target features and generates a response map based on the kernel correlation filter method, corrects the target trajectory using Kalman filtering, and further calculates the confidence level through a multi-dimensional reliability assessment mechanism. When the confidence level falls below a set threshold, a backup template mechanism is activated; if verification fails for multiple consecutive frames, a loss re-detection mechanism is executed, achieving target relocalization through a two-stage search and multi-scale correction. The display module outputs tracking results in real time, the communication interface supports remote transmission and interaction with the robot system, and the power supply ensures stable operation of the entire device.

[0165] Compared with existing technologies, this device, through the integration and optimization of the aforementioned components and combined with a multi-level loss recovery mechanism, achieves a target tracking processing speed of 50+ FPS under limited computing resources, balancing real-time performance and robustness, and is suitable for embedded scenarios such as mobile robots. For more detailed information on the working processes of each of the aforementioned components, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0166] This invention underwent comprehensive testing on the OTB100 standard dataset. OTB100 contains 100 video sequences with different challenge attributes, covering 11 challenging scenarios including lighting variations, scale variations, occlusion, deformation, motion blur, fast motion, in-plane rotation, out-of-plane rotation, out-of-view, background clutter, and low resolution. Combined with... Figure 4 The overall performance of the method of the present invention is as follows:

[0167] The accuracy reached 0.746, which is 10.4% higher than the KCF algorithm's 0.676; the success rate reached 0.546, which is 7.7% higher than the KCF algorithm's 0.507.

[0168] Specifically, this tracking method demonstrates the following improvements across various challenging scenarios: accuracy is increased by 17.9% and 18.0% in fast-moving and motion-blurred scenarios, respectively, with success rates also improved by 14.4% and 12.1%, respectively. Regarding target shape changes, accuracy is improved by 14.4%, 16.1%, and 14.3% in occlusion, deformation, and scale change scenarios, respectively, with corresponding increases in success rates of 10.4%, 13.0%, and 10.9%. In addition to the aforementioned attributes, performance improvements are also achieved to varying degrees in most other attributes, such as illumination changes, in-plane rotation, and out-of-view tracking.

[0169] The method of this invention achieves an average frame rate of 67.7 FPS while maintaining high accuracy, which is significantly better than most improved algorithms and is suitable for deployment in embedded devices and real-time application scenarios.

[0170] This invention has been successfully deployed on the Jetson Nano embedded platform, demonstrating excellent real-time performance and stability in mobile robot target tracking applications. Even with limited computing resources, it can maintain a processing speed of 50+ FPS, meeting real-time tracking requirements.

[0171] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatuses, devices, and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant details can be found in the method section. Those skilled in the art will further recognize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A target tracking method based on a multi-level loss recovery mechanism, characterized in that, Includes the following steps: Acquire video frame sequences and initial target area information; The target is tracked using a kernel correlation filter in the current video frame, and the HOG features of the target are extracted and a response map is generated. The Kalman filter is used to correct the motion state of the tracking results of the kernel correlation filter, and the corrected target position is output. Assess the tracking confidence of the current target. When the tracking confidence level is at this point If the tracking confidence level is greater than or equal to the first preset threshold, the current tracking is determined to be reliable, and the tracking result of the target is output and updated; when the tracking confidence level is... If the value is less than the first preset threshold, the backup template mechanism will be activated to restore the target. Reassess the tracking confidence of the recovered target. When the tracking confidence level is at this point If the tracking confidence level is greater than or equal to the first preset threshold, the recovered target is determined to be more reliable, and the tracking result of the recovered target is output and updated; when the tracking confidence level is... If the value is still less than the first preset threshold, the failure counter is incremented by 1, and the filter state is not updated. When the backup template verification fails for multiple consecutive frames, the target loss re-detection mechanism is triggered to re-detect the target, obtain the candidate region of the target, and perform multi-scale search and correction on the re-detected candidate region to restore normal target tracking.

2. The target tracking method based on a multi-level loss recovery mechanism according to claim 1, characterized in that, The kernel function used in the kernel correlation filter is a Gaussian radial basis function kernel. 3.The target tracking method based on a multi-level loss recovery mechanism according to claim 1, characterized in that, The motion state correction includes: The target state is modeled using a six-dimensional state vector. ,in( ( ) represents the center coordinates of the target. These are the width and height of the target, respectively; These are the target's velocities in the x and y directions, respectively; A target motion model is established using a Kalman filter, and the position of the current video frame is predicted using a state transition matrix. The tracking results of the kernel correlation filter are used as observations, and the motion velocity components of the target state are smoothed using a sliding window. The observation residuals are calculated based on the smoothed observation vectors, and the Kalman gain is calculated by combining the prediction covariance matrix and the observation noise covariance matrix. The predicted position is corrected by posterior estimation using the Kalman gain, and the corrected target position is output.

4. The target tracking method based on a multi-level loss recovery mechanism according to claim 1, characterized in that, the tracking confidence The evaluation model is: ； in, For the tracking confidence of the response graph, The peak signal-to-noise ratio of the response plot, where K is a constant. The response entropy of the response graph, The correlation coefficient between the corrected target location HOG features and the historical template HOG features is given. , and These are the weights.

5. The target tracking method based on a multi-level loss recovery mechanism according to claim 1, characterized in that, The backup template mechanism includes: When the tracking confidence of the current target When the current tracking is greater than or equal to the first preset threshold, the tracking result of the kernel correlation filter is used as the main template for continuous updating, and the current frame is saved as a historical high-reliability frame as a backup template. When the tracking confidence of the current target If the value is less than the first preset threshold, the target's response map is recalculated based on the backup template, and the peak value of the backup template's response map is compared with the peak value of the main template's response map. If the peak value of the backup template's response map is greater than the peak value of the main template's response map, the recovered target box is determined to be more reliable, and the tracking confidence of the recovered target is reassessed. ; When the tracking confidence of the recovered target If the value is greater than or equal to the first preset threshold, the recovered target is determined to be more reliable, and the tracking result of the recovered target is output and updated.

6. The target tracking method based on a multi-level loss recovery mechanism according to claim 1, characterized in that, The target loss re-detection mechanism adopts a two-stage search strategy; The two-stage search strategy includes: In the rapid screening stage, the search area is first expanded with the predicted location as the center, and the color histogram and HOG features of the candidate area are extracted using a sliding window with a step size of s1. Then, the color area overlap rate and HOG feature correlation between the candidate area and the historical target template are calculated, and the maximum similarity between the two is retained as the initial score of the window. Candidate areas that meet the preset conditions in terms of color area overlap rate and HOG feature correlation are initially screened. In the fine-grained matching stage, a local search region is constructed centered on the candidate regions selected quickly. A sliding window with a step size of s2 is used for fine-grained matching, and the best candidate region is selected based on the color area overlap rate and HOG feature correlation; where s2 <s1。 7. The target tracking method based on a multi-level loss recovery mechanism according to claim 6, characterized in that, In the rapid screening stage, after initially screening out candidate regions, the initially screened candidate regions are subjected to dual threshold verification, and then the scores are recalculated in combination with distance penalty. The N candidate regions with the highest scores are selected to enter the fine matching stage.

8. The target tracking method based on a multi-level loss recovery mechanism according to claim 6 or 7, characterized in that, The multi-scale correction includes: After fine matching is completed, multiple search windows of different scales are constructed for the best candidate regions output. Recalculate the fusion confidence of HOG feature correlation and color area overlap rate for each scale window; Select the scale with the highest fusion confidence as the current scale of the target to restore normal tracking.

9. A target tracking system based on a multi-level loss recovery mechanism, characterized in that, include: The information acquisition module is used to acquire video frame sequences and initial target area information; The feature extraction module is used to perform target tracking on the current video frame using a kernel correlation filter, extract the target's HOG features, and generate a response map; The motion correction module is used to correct the motion state of the tracking results of the kernel correlation filter using a Kalman filter and output the corrected target position. a reliability evaluation module configured to calculate a peak signal-to-noise ratio (PSR) of the response map, a response entropy of the response map and a correlation coefficient of the modified target position HOG feature and the historical template HOG feature , according to the peak signal-to-noise ratio (PSR), the response entropy and the correlation coefficient to construct a multi-dimensional reliability evaluation model and calculate a tracking confidence of the target ; The standby template mechanism starting module is configured to start a standby template mechanism to recover the target when the tracking confidence of the current target is less than the first preset threshold value. When the tracking confidence is greater than or equal to the first preset threshold value, it is determined that the current tracking is reliable, and the tracking result of the target is output and updated. When the tracking confidence is less than the first preset threshold value, the standby template mechanism is started to recover the target. Then re-evaluate the tracking confidence of the recovered target When the tracking confidence at this time is greater than or equal to the first preset threshold, it is determined that the recovered target frame is reliable, and the tracking result of the recovered target is output and updated. a loss re-detection module for detecting a loss of the target when the tracking confidence of the target after recovery is still less than the first preset threshold, the failure counter is incremented by 1, and the state of the filter is not updated. When the backup template verification fails for multiple consecutive frames, the target loss re-detection mechanism is triggered to re-detect the target, obtain the candidate region of the target, and perform multi-scale search and correction on the re-detected candidate region to restore normal target tracking.

10. A target tracking device based on a multi-level loss recovery mechanism, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the target tracking method according to any one of claims 1-8.

Images