A method and system for long-term tracking of single targets with anti-occlusion combined with trajectory prediction

Abstract

The present invention discloses a method and system for long-term tracking of a single target with anti-occlusion combined with trajectory prediction. The method comprises: first manually specifying the position and size of the target, determining the area to be tracked, extracting features from the target area and mapping the feature vector to a high-dimensional feature space to obtain an initial target template; locating the target in the tracking area according to the feature information of the initial target model, calculating the similarity between the target candidate area and the initial target template, selecting the most likely target position, and entering a search state when the target is occluded in combination with an occlusion determination algorithm, and using the historical trajectory data of the target for prediction; dynamically updating the template information of the target according to the obtained target position in combination with the current state and motion of the target. The system comprises a target initialization module, a target positioning module and a target template update module. The present invention can update the target search range in real time to achieve accurate and stable long-term target tracking.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an anti-occlusion single target long-term tracking method and system combined with trajectory prediction. Background Art

[0002] Single target tracking technology is a long-term research hotspot in the field of computer vision. With the development of computer vision technology, single target tracking plays an important role in many practical applications, such as video surveillance, autonomous driving, robot navigation, and human-computer interaction. The goal of single target tracking is to specify and continuously track a specific target object from a video sequence, while dealing with challenges such as target appearance changes, occlusion, rapid movement, and background interference. The development of single target tracking technology can be traced back to early model-based methods, which usually rely on predefined templates or target models to track targets by matching. However, this method is very sensitive to changes in target appearance and has difficulty in dealing with complex scenes. With the rise of machine learning and deep learning technologies, single target tracking has been significantly improved in terms of model robustness and accuracy.

[0003] Single target tracking methods based on deep learning usually use convolutional neural networks (CNNs) to extract deep features of the target, thereby improving the ability to adapt to complex backgrounds and lighting changes. For example, the application of Siamese network structures in single target tracking has received widespread attention. The Siamese network inputs the target and the search area into the twin network at the same time and calculates the similarity between the two, thereby achieving efficient target positioning. In addition to the Siamese network, single target tracking methods based on generative adversarial networks (GANs) and attention mechanisms have also made significant research progress. GAN can generate realistic target samples through adversarial training of the generator and the discriminator, enhancing the tracker's ability to adapt to changes in the target's appearance. The attention mechanism improves the accuracy and robustness of tracking by focusing on important areas in the image.

[0004] Although single target tracking technology has made great progress, it still faces many challenges. In particular, how to efficiently deal with the rapid movement, drastic deformation and complex occlusion of the target is still a hot topic in current research. In addition, how to design lightweight tracking algorithms to adapt to resource-constrained embedded devices is also an important research direction.

[0005] In long-term tracking, the ability to effectively handle occlusion, especially severe occlusion, is an important aspect of evaluating the performance of target tracking algorithms and is of great significance to improving the robustness of target tracking algorithms. However, existing tracking algorithms do not solve the problem of target occlusion in real scenarios well or can only deal with the target being partially occluded for a short time (target occlusion area ≤ 50% of the total target area), and the existing occlusion criteria cannot be well integrated with the tracking algorithm. In many cases, the occlusion criteria will make incorrect judgments, which seriously affects the performance of the tracking algorithm. In view of the fact that existing tracking algorithms are difficult to effectively deal with the problem of long-term target occlusion in real scenarios, we propose a long-term tracking method for single targets with occlusion resistance combined with trajectory prediction. First, the template update strategy of the target tracking algorithm is improved to ensure that the target tracking template always has a high confidence level and avoid the target template being disturbed by occlusion or other environmental factors. Combined with a new occlusion judgment method, it accurately determines whether the target is occluded. When the target is confirmed to be occluded, the tracking strategy is changed in time. When the target is occluded, the trajectory prediction algorithm is used to predict the target position in real time, and the target search range is dynamically adjusted to ensure that the target is captured in time when it appears. Summary of the invention

[0006] The purpose of the present invention is to provide a method and system for long-term tracking of a single target with anti-occlusion combined with trajectory prediction. Aiming at the problem that the current target tracking algorithm cannot effectively track occluded targets for a long time, the target template update strategy is improved, and a fast occlusion judgment method suitable for low computing power resource platforms is proposed. At the same time, combined with the trajectory prediction algorithm based on Kalman filtering, the target position is predicted when the target is occluded, and the target search range is updated in real time to achieve accurate and stable long-term target tracking.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A method for long-term tracking of a single target with anti-occlusion combined with trajectory prediction, comprising:

[0009] Target initialization: first manually specify the position and size of the target, determine the area to be tracked, extract features from the target area and map the feature vector to a high-dimensional feature space to obtain the initial target template;

[0010] Target positioning: locate the target in the tracking area based on the feature information of the initial target model, calculate the similarity between the target candidate area and the initial target template, select the most likely target position, and enter the search state when the target is blocked by the occlusion judgment algorithm. Use the historical trajectory data of the target to make predictions to deal with the situation where the target is blocked.

[0011] Target template update: According to the obtained target position, combined with the current state and motion of the target, the target template information is dynamically updated to adapt to the possible appearance changes and motion trajectory adjustments of the target, thereby achieving continuous tracking and positioning of the target.

[0012] A further improvement of the present invention is that target initialization specifically includes:

[0013] At the beginning of the tracking task, the position and size of the target are manually specified in the first frame. The tracking algorithm uses the directional gradient histogram and color histogram to extract features of the target area on the image according to the specified position and size, and maps the extracted features to a high-dimensional feature space; then the feature data of the target area is used to train the correlation filter, including calculating the Fourier transform of the target feature, calculating the response of the correlation filter, and updating the filter parameters; finally, an initial target template is generated, which contains the trained correlation filter and feature mapping results. The initial target template is used to detect and locate the target in the subsequent tracking process; the gradient value of each pixel in the target area With the gradient direction The calculation formula is as follows:

[0014]

[0015]

[0016] in Represents the horizontal gradient of a pixel. Represents the vertical gradient of a pixel.

[0017] A further improvement of the present invention is that target positioning includes:

[0018] In subsequent frames, according to the feature information of the current target template, the target is located in the search area, the similarity between the target candidate area and the initial target template is calculated, and the most likely target position is selected on this basis. Specifically, a circulant matrix is ​​used to collect positive and negative samples in the area around the target, and the target detector is trained using ridge regression. The diagonal property of the circulant matrix in Fourier space is used to convert matrix operations into point products of vector elements. At the same time, the ridge regression in linear space is mapped to nonlinear space through a kernel function, the response matrix in the frequency domain is calculated, and then the inverse Fourier transform is calculated to obtain the final response matrix. The largest position in the response matrix is ​​the position of the target in the current image.

[0019] A further improvement of the present invention is that a target occlusion judgment algorithm is added to determine whether the target is occluded by calculating the structural similarity coefficient between the current target and the template target. When the structural similarity coefficient is less than a set threshold, it is considered that the target is occluded at this time, and the template is no longer updated, and the search state is entered; in the search state, the target will be relocated through image features and historical trajectory information, including changes in color, texture features, motion trajectory and the target's surrounding environment, until the target reappears and the structural similarity coefficient reaches above the set threshold again; at this time, the algorithm will exit the search state.

[0020] A further improvement of the present invention is that, when predicting the target trajectory, when the target occlusion determination algorithm detects that the target is occluded, it enters a search state, predicts the target trajectory in combination with the target historical position information, and dynamically adjusts the target search range in real time according to the predicted position; the trajectory prediction adopts Kalman filtering, through a dynamic model and observation data, based on the fusion of dynamic evolution and observation data, to achieve the optimal estimation of the target position; through a dynamic state estimation and error adjustment process, in the presence of dynamic changes and observation errors, the optimal estimation of the target position is provided, thereby achieving accurate prediction and tracking of the target trajectory.

[0021] A further improvement of the present invention is that the target template is updated, including:

[0022] Based on the current target position, if the target size is , the size of the image is The image block is taken, the directional gradient features of the image block are calculated, and the directional gradient feature map of the target is obtained. The feature map is weighted by the cosine window to reduce the image smoothness caused by the boundary shift. The length, width and feature are obtained by using the two-dimensional Gaussian function. Figure 1 The template matrix is ​​updated by linear interpolation according to the set learning rate.

[0023] A further improvement of the present invention is that the target template matrix update strategy is modified, and the calculation of the structural similarity coefficient is introduced before the template matrix is ​​updated; through the calculation of the structural similarity coefficient, the target can be updated only when it is not occluded and has a sufficiently similar structure to the template target, avoiding the introduction of external interference factors in the template update; a threshold is set as a judgment basis to screen out the template target that needs to be updated; when the target is not occluded, the judgment is made based on the structural similarity coefficient between the current target and the template target; if the similarity coefficient is higher than the set threshold, the template matrix will be updated to ensure the matching degree and accuracy of the template and the actual target.

[0024] An anti-occlusion single target long-term tracking system combined with trajectory prediction, comprising:

[0025] The target initialization module first manually specifies the position and size of the target, determines the area to be tracked, extracts features from the target area and maps the feature vector to a high-dimensional feature space to obtain the initial target template;

[0026] The target positioning module locates the target in the tracking area according to the feature information of the initial target model, calculates the similarity between the target candidate area and the initial target template, selects the most likely target position, and enters the search state when the target is blocked by the occlusion judgment algorithm. It uses the historical trajectory data of the target to make predictions to deal with the situation where the target is blocked.

[0027] The target template update module dynamically updates the target template information according to the obtained target position and the current state and motion of the target to adapt to the possible appearance changes and motion trajectory adjustments of the target, thereby achieving continuous tracking and positioning of the target.

[0028] A further improvement of the present invention is that the target initialization module specifically includes:

[0029] At the beginning of the tracking task, the position and size of the target are manually specified in the first frame. The tracking algorithm uses the directional gradient histogram and color histogram to extract features of the target area on the image according to the specified position and size, and maps the extracted features to a high-dimensional feature space; then the feature data of the target area is used to train the correlation filter, including calculating the Fourier transform of the target feature, calculating the response of the correlation filter, and updating the filter parameters; finally, an initial target template is generated, which contains the trained correlation filter and feature mapping results. The initial target template is used to detect and locate the target in the subsequent tracking process; the gradient value of each pixel in the target area With the gradient direction The calculation formula is as follows:

[0030]

[0031]

[0032] in Represents the horizontal gradient of a pixel. Represents the vertical gradient of a pixel.

[0033] A further improvement of the present invention is that the target positioning module specifically includes:

[0034] In subsequent frames, according to the feature information of the current target template, the target is located in the search area, the similarity between the target candidate area and the initial target template is calculated, and the most likely target position is selected on this basis. Specifically, a circulant matrix is ​​used to collect positive and negative samples in the area around the target, and the target detector is trained using ridge regression. The diagonal property of the circulant matrix in Fourier space is used to convert matrix operations into point products of vector elements. At the same time, the ridge regression in linear space is mapped to nonlinear space through a kernel function, the response matrix in the frequency domain is calculated, and then the inverse Fourier transform is calculated to obtain the final response matrix. The largest position in the response matrix is ​​the position of the target in the current image.

[0035] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0036] The present invention proposes an anti-occlusion single-target long-term tracking method and system combined with trajectory prediction. Through a fast occlusion judgment method suitable for low-computing power computing platforms, the present invention can quickly respond in complex scenes, promptly discover whether the target is occluded, and take corresponding measures accordingly. The characteristic of this method is the efficient use of computing power resources. Under the premise of ensuring tracking accuracy, it can minimize computing time and resource consumption, so that the present invention can still maintain efficient operation in a resource-constrained environment.

[0037] Furthermore, the present invention proposes to dynamically adjust the target search range in real time when the target is blocked, in combination with a trajectory prediction algorithm based on Kalman filtering, so as to improve the success rate of target recapture. The trajectory prediction algorithm based on Kalman filtering can effectively handle the dynamic changes and noise interference of the present invention, providing a more accurate target position estimation for the present invention, while the real-time dynamic adjustment of the target search range can ensure that the present invention can still effectively track the target in a complex environment.

[0038] Furthermore, the present invention proposes a new template updating strategy, which introduces the calculation of the structural similarity coefficient before updating the template matrix. By evaluating the structural similarity between the current target and the template target, intelligent control of the template update is achieved to ensure the matching degree and accuracy between the template and the actual target, thereby improving the sensitivity and accuracy of the present invention to the target features, and also enhancing the adaptability and generalization ability of the present invention to different targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a method for long-term tracking of a single target with anti-occlusion combined with trajectory prediction according to the present invention;

[0040] Figure 2 (a) and (b) are schematic diagrams of feature extraction of the target area by dividing the cell into cells using the directional gradient histogram;

[0041] Figure 3 is a schematic diagram of a circulant matrix;

[0042] Figure 4 Flowchart for target trajectory prediction;

[0043] Figure 5 It is the cosine window waveform;

[0044] Figure 6 The present invention is a structural block diagram of an anti-occlusion single target long-term tracking system combined with trajectory prediction. DETAILED DESCRIPTION

[0045] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0046] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0047] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0048] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0049] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0050] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] Example 1

[0052] The present invention provides an anti-occlusion single target long-term tracking method combined with trajectory prediction, comprising:

[0053] Target initialization: first manually specify the position and size of the target, determine the area to be tracked, extract features from the target area and map the feature vector to a high-dimensional feature space to obtain the initial target template;

[0054] Target positioning: locate the target in the tracking area based on the feature information of the initial target model, calculate the similarity between the target candidate area and the initial target template, select the most likely target position, and enter the search state when the target is blocked by the occlusion judgment algorithm. Use the historical trajectory data of the target to make predictions to deal with the situation where the target is blocked.

[0055] Target template update: According to the obtained target position, combined with the current state and motion of the target, the target template information is dynamically updated to adapt to the possible appearance changes and motion trajectory adjustments of the target, thereby achieving continuous tracking and positioning of the target.

[0056] In this embodiment, target initialization specifically includes:

[0057] At the beginning of the tracking task, the position and size of the target are manually specified in the first frame. The tracking algorithm uses the directional gradient histogram and color histogram to extract features of the target area on the image according to the specified position and size, and maps the extracted features to a high-dimensional feature space; then the feature data of the target area is used to train the correlation filter, including calculating the Fourier transform of the target feature, calculating the response of the correlation filter, and updating the filter parameters; finally, an initial target template is generated, which contains the trained correlation filter and feature mapping results. The initial target template is used to detect and locate the target in the subsequent tracking process; the gradient value of each pixel in the target area With the gradient direction The calculation formula is as follows:

[0058]

[0059]

[0060] in Represents the horizontal gradient of a pixel. Represents the vertical gradient of a pixel.

[0061] In this embodiment, target positioning includes:

[0062] In subsequent frames, according to the feature information of the current target template, the target is located in the search area, the similarity between the target candidate area and the initial target template is calculated, and the most likely target position is selected on this basis. Specifically, a circulant matrix is ​​used to collect positive and negative samples in the area around the target, and the target detector is trained using ridge regression. The diagonal property of the circulant matrix in Fourier space is used to convert matrix operations into point products of vector elements. At the same time, the ridge regression in linear space is mapped to nonlinear space through a kernel function, the response matrix in the frequency domain is calculated, and then the inverse Fourier transform is calculated to obtain the final response matrix. The largest position in the response matrix is ​​the position of the target in the current image.

[0063] In this embodiment, a target occlusion determination algorithm is added to determine whether the target is occluded by calculating the structural similarity coefficient between the current target and the template target. When the structural similarity coefficient is less than the set threshold, the target is considered to be occluded at this time, and the template is no longer updated, and the search state is entered; in the search state, the target will be relocated through image features and historical trajectory information, including changes in color, texture features, motion trajectory and the target's surrounding environment, until the target reappears and the structural similarity coefficient reaches above the set threshold again; at this time, the algorithm will exit the search state.

[0064] In this embodiment, when predicting the target trajectory, when the target occlusion determination algorithm detects that the target is occluded, it enters the search state, predicts the target trajectory in combination with the target historical position information, and dynamically adjusts the target search range in real time according to the predicted position; the trajectory prediction adopts Kalman filtering, through the dynamic model and observation data, based on the fusion of dynamic evolution and observation data, to achieve the optimal estimation of the target position; through the dynamic state estimation and error adjustment process, in the presence of dynamic changes and observation errors, the optimal estimation of the target position is provided, thereby achieving accurate prediction and tracking of the target trajectory.

[0065] In this embodiment, the target template is updated, including:

[0066] Based on the current target position, if the target size is , the size of the image is The image block is taken, the directional gradient features of the image block are calculated, and the directional gradient feature map of the target is obtained. The feature map is weighted by the cosine window to reduce the image smoothness caused by the boundary shift. The length, width and feature are obtained by using the two-dimensional Gaussian function. Figure 1 The template matrix is ​​updated by linear interpolation according to the set learning rate.

[0067] In this embodiment, the target template matrix update strategy is modified, and the calculation of the structural similarity coefficient is introduced before the template matrix is ​​updated; through the calculation of the structural similarity coefficient, the target can be updated only when it is not occluded and has a sufficiently similar structure to the template target, avoiding the introduction of external interference factors when updating the template; a threshold is set as a judgment basis to screen out the template targets that need to be updated; when the target is not occluded, the judgment is made based on the structural similarity coefficient between the current target and the template target; if the similarity coefficient is higher than the set threshold, the template matrix will be updated to ensure the matching degree and accuracy of the template and the actual target.

[0068] Example 2

[0069] The present invention provides an anti-occlusion single target long-term tracking method combined with trajectory prediction, comprising:

[0070] 1) Overall process

[0071] The anti-occlusion single target long-term tracking method combined with trajectory prediction proposed in the present invention mainly includes three stages, namely target initialization, target positioning and target template update. In the target initialization stage, the position and size of the target are first manually specified to determine the area to be tracked. Then, in the target positioning stage, the target is positioned in the search area according to the feature information of the current template, the similarity between the target candidate area and the initial target template is calculated, the most likely target position is selected, and the occlusion judgment algorithm is combined to enter the search state when the target is occluded. The historical trajectory data of the target will be used for prediction to deal with the situation where the target is occluded, ensuring the continuity and accuracy of the tracking. Finally, in the target template update stage, the template information of the target is dynamically updated according to the current state and motion of the target to adapt to the possible appearance changes and motion trajectory adjustments of the target, thereby achieving continuous tracking and positioning of the target.

[0072] 1) Target initialization

[0073] In the target initialization stage, you first need to manually specify the position and size of the target in the first frame to determine the image area that needs to be tracked. Next, the tracking algorithm uses the directional gradient histogram and color histogram to extract features from the target area, and maps these features in a high-dimensional feature space to enhance the expression ability, making the features easier to distinguish and model. Subsequently, the correlation filter is trained using the feature data of the target area, which includes calculating the Fourier transform of the target features, calculating the response of the correlation filter, and updating the parameters. Finally, an initial target model is formed, which contains the trained correlation filter and feature mapping results. This model is used for target detection and positioning in the subsequent tracking process. The gradient value of the pixel in the target area With the gradient direction The calculation formula is as follows:

[0074]

[0075]

[0076] in Represents the horizontal gradient of a pixel. Represents the vertical gradient of a pixel. and The calculation formula is as follows:

[0077]

[0078]

[0079] in Representing the image The pixel value at the coordinate.

[0080] After obtaining the gradient of each pixel in the target area, the image of the target area is divided into several cells, and adjacent cells do not overlap. In each cell, all gradient directions are divided into n intervals (usually n=8), and the modulus lengths in the corresponding intervals are accumulated to obtain the feature vector of a single interval. The feature vectors of each interval are concatenated and normalized to obtain the feature vector of the entire target area, such as Figure 2 shown.

[0081] 2) Target positioning

[0082] like Figure 3 As shown in the figure, in the target positioning stage, a circulant matrix is ​​used to collect positive and negative samples in the area around the target, and the target detector is trained using ridge regression. The ridge regression in the linear space is mapped to the nonlinear space through the kernel function, and the response matrix in the nonlinear space is calculated. Then, the final response matrix is ​​obtained by inverse Fourier transform. The largest position in the response matrix is ​​the position of the target in the current image. After obtaining the target position in the current image, the structural similarity coefficient between the current target and the template target is calculated. The structural similarity coefficient is used to measure the similarity between the two images. Compared with traditional image quality measurement indicators, the structural similarity coefficient is more consistent with the human eye's judgment of image quality in measuring image quality.

[0083] The calculation of the image structural similarity coefficient is based on a sliding window. Each time a window of size 10×10 is taken from the image, and the structural similarity coefficient is calculated based on the window. After traversing the entire image, the values ​​of all windows are averaged and used as the SSIM indicator of the entire image.

[0084] Assume that x represents the data in the first image window, and y represents the data in the second image window. The similarity of the images consists of three parts: brightness similarity, contrast similarity, and structural similarity. The formula for calculating brightness similarity is:

[0085]

[0086] The formula for calculating contrast similarity is:

[0087]

[0088] The formula for calculating structural similarity is:

[0089]

[0090] in and represents the mean of x and y, and represents the variance of x and y, represents the covariance of x and y, , and are three constants, namely:

[0091]

[0092]

[0093]

[0094] Where L represents the range of image pixel values.

[0095] The calculation formula of the final structural similarity coefficient is:

[0096]

[0097] make All are 1, simplifying to get:

[0098]

[0099] If the structural similarity coefficient between the current target and the template target is less than the set threshold (0.65), the target is considered to be blocked, the template is no longer updated, and the search state is entered. Figure 4 As shown, in the search state, a trajectory prediction mechanism is added, and the Kalman filter algorithm is used to predict the target's motion trajectory. The target motion state equation is established in the image coordinate system as follows:

[0100] The target state quantities are the position and velocity in the x direction, and the position and velocity in the y direction, recorded as:

[0101]

[0102] The target state transfer equation is:

[0103]

[0104] Where T is the sampling interval.

[0105] Based on the target position predicted by the Kalman filter, the present invention dynamically adjusts the target search range and periodically searches for the target in the image to ensure that the present invention tracks the position change of the target and captures the target in time when it reappears. When the target reappears and the structural similarity coefficient with the template target reaches a set threshold (0.65) or above, the present invention will confirm that the target has been recaptured, exit the search state, and enter the normal tracking state.

[0106] Get the target location

[0107] 3) Target template update

[0108] References

[0109] After obtaining the target position of the current frame, the structural similarity coefficient between the current target and the template target is calculated. If the calculated result is higher than the set threshold (0.8), the present invention will update the template matrix. Specifically, according to the position of the current target, an image block is intercepted in the corresponding area on the image, and the directional gradient feature of the image block is calculated to obtain a directional gradient feature map, such as Figure 5 As shown, the feature map is weighted by a cosine window. The cosine window is used to smooth the boundaries of the image, which can make the image transition smoothly at the boundary and reduce the error caused by the edge effect. The definition of the cosine window is as follows:

[0110]

[0111] Where n is the value of each point in the feature map, and N is the size of the feature map

[0112] The target template is updated as follows:

[0113]

[0114] in is the updated template matrix, is the template matrix of the previous frame, is the sample generated by the circulant matrix based on the current target position, is the learning rate.

[0115] Example 3

[0116] like Figure 6As shown, the present invention provides an anti-occlusion single target long-term tracking system combined with trajectory prediction, comprising:

[0117] The target initialization module first manually specifies the position and size of the target, determines the area to be tracked, extracts features from the target area and maps the feature vector to a high-dimensional feature space to obtain the initial target template;

[0118] The target positioning module locates the target in the tracking area according to the feature information of the initial target model, calculates the similarity between the target candidate area and the initial target template, selects the most likely target position, and enters the search state when the target is blocked by the occlusion judgment algorithm. It uses the historical trajectory data of the target to make predictions to deal with the situation where the target is blocked.

[0119] The target template update module dynamically updates the target template information according to the obtained target position and the current state and motion of the target to adapt to the possible appearance changes and motion trajectory adjustments of the target, thereby achieving continuous tracking and positioning of the target.

[0120] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

[0121] In addition, it should be understood that although this specification is described in accordance with the implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation modes that can be understood by those skilled in the art. The above content is only to illustrate the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for long-term tracking of a single target with anti-occlusion combined with trajectory prediction, characterized in that: include: Target initialization: first manually specify the position and size of the target, determine the area to be tracked, extract features from the target area and map the feature vector to a high-dimensional feature space to obtain the initial target template; specifically, it includes: At the beginning of the tracking task, the position and size of the target are manually specified in the first frame. The tracking algorithm uses the directional gradient histogram and color histogram to extract features of the target area on the image according to the specified position and size, and maps the extracted features to a high-dimensional feature space. Then, the feature data of the target area is used to train the correlation filter, including calculating the Fourier transform of the target feature, calculating the response of the correlation filter, and updating the filter parameters. Finally, an initial target template is generated, which contains the trained correlation filter and feature mapping results. The initial target template is used to detect and locate the target in the subsequent tracking process. The calculation formula of the gradient value g and the gradient direction θ of each pixel in the target area is as follows: where g x Represents the horizontal gradient of the pixel, g y Represents the vertical gradient of a pixel; Target positioning: locate the target in the tracking area based on the feature information of the initial target model, calculate the similarity between the target candidate area and the initial target template, select the most likely target position, and enter the search state when the target is blocked by the occlusion judgment algorithm. Use the historical trajectory data of the target to make predictions to deal with the situation where the target is blocked. Specifically, it includes: In subsequent frames, according to the feature information of the current target template, the target is located in the search area, the similarity between the target candidate area and the initial target template is calculated, and the most likely target position is selected on this basis. Specifically, positive and negative samples are collected in the area around the target using a circulant matrix, the target detector is trained using ridge regression, and the matrix operation is converted into the dot product of vector elements using the diagonalizable property of the circulant matrix in Fourier space. At the same time, the ridge regression in the linear space is mapped to the nonlinear space through the kernel function, the response matrix in the frequency domain is calculated, and then the inverse Fourier transform is calculated to obtain the final response matrix. The largest position in the response matrix is ​​the position of the target in the current image. Target template update: According to the obtained target position, combined with the current state and motion of the target, the target template information is dynamically updated to adapt to the possible appearance changes and motion trajectory adjustments of the target, thereby achieving continuous tracking and positioning of the target.

2. The method for long-term tracking of a single target with anti-occlusion combined with trajectory prediction according to claim 1, characterized in that: Add a target occlusion judgment algorithm to determine whether the target is occluded by calculating the structural similarity coefficient between the current target and the template target. When the structural similarity coefficient is less than the set threshold, the target is considered to be occluded, the template is no longer updated, and the search state is entered; in the search state, the target will be relocated through image features and historical trajectory information, including changes in color, texture features, motion trajectory, and the target's surrounding environment, until the target reappears and the structural similarity coefficient reaches above the set threshold again; at this time, the algorithm will exit the search state.

3. The method for long-term tracking of a single target with anti-occlusion combined with trajectory prediction according to claim 1, characterized in that: When predicting the target trajectory, when the target occlusion judgment algorithm detects that the target is occluded, it enters the search state, predicts the target trajectory in combination with the target's historical position information, and dynamically adjusts the target search range in real time according to the predicted position; the trajectory prediction adopts Kalman filtering, through dynamic models and observation data, based on the fusion of dynamic evolution and observation data, to achieve the optimal estimation of the target position; through the dynamic state estimation and error adjustment process, in the presence of dynamic changes and observation errors, it provides the optimal estimate of the target position, thereby achieving accurate prediction and tracking of the target trajectory.

4. The method for long-term tracking of a single target with anti-occlusion combined with trajectory prediction according to claim 1, characterized in that: Target template updates, including: Based on the current target position, if the target size is m*n, an image block of size m*n is intercepted on the image, the directional gradient features of the image block are calculated, and the directional gradient feature map of the target is obtained. The feature map is weighted by a cosine window to reduce the image roughness caused by boundary shift. A two-dimensional Gaussian function is used to obtain a template matrix with the same length and width as the feature map. According to the set learning rate, the target template is updated using linear interpolation.

5. The method for long-term tracking of a single target with anti-occlusion combined with trajectory prediction according to claim 4, characterized in that: Modify the target template matrix update strategy, introduce the calculation of the structural similarity coefficient before updating the template matrix; through the calculation of the structural similarity coefficient, the target can be updated only when it is not occluded and has a sufficiently similar structure to the template target, avoiding the introduction of external interference factors in the template update; set a threshold as a judgment basis to filter out the template targets that need to be updated; when the target is not occluded, make a judgment based on the structural similarity coefficient between the current target and the template target; If the similarity coefficient is higher than the set threshold, the template matrix will be updated to ensure the matching degree and accuracy between the template and the actual target.

6. An anti-occlusion single target long-term tracking system combined with trajectory prediction, characterized in that: include: The target initialization module first manually specifies the position and size of the target, determines the area to be tracked, extracts features from the target area and maps the feature vector to a high-dimensional feature space to obtain the initial target template. Specifically, it includes: At the beginning of the tracking task, the position and size of the target are manually specified in the first frame. The tracking algorithm uses the directional gradient histogram and color histogram to extract features of the target area on the image according to the specified position and size, and maps the extracted features to a high-dimensional feature space. Then, the feature data of the target area is used to train the correlation filter, including calculating the Fourier transform of the target feature, calculating the response of the correlation filter, and updating the filter parameters. Finally, an initial target template is generated, which contains the trained correlation filter and feature mapping results. The initial target template is used to detect and locate the target in the subsequent tracking process. The calculation formula of the gradient value g and the gradient direction θ of each pixel in the target area is as follows: where g x Represents the horizontal gradient of the pixel, g y Represents the vertical gradient of a pixel; The target positioning module locates the target in the tracking area according to the feature information of the initial target model, calculates the similarity between the target candidate area and the initial target template, selects the most likely target position, and enters the search state when the target is blocked by the occlusion judgment algorithm. It uses the historical trajectory data of the target to make predictions to deal with the situation where the target is blocked. Specifically, it includes: In subsequent frames, according to the feature information of the current target template, the target is located in the search area, the similarity between the target candidate area and the initial target template is calculated, and the most likely target position is selected on this basis. Specifically, positive and negative samples are collected in the area around the target using a circulant matrix, the target detector is trained using ridge regression, and the matrix operation is converted into the dot product of vector elements using the diagonalizable property of the circulant matrix in Fourier space. At the same time, the ridge regression in the linear space is mapped to the nonlinear space through the kernel function, the response matrix in the frequency domain is calculated, and then the inverse Fourier transform is calculated to obtain the final response matrix. The largest position in the response matrix is ​​the position of the target in the current image. The target template update module dynamically updates the target template information according to the obtained target position and the current state and motion of the target to adapt to the possible appearance changes and motion trajectory adjustments of the target, thereby achieving continuous tracking and positioning of the target.

Images