A correlation filter based target tracking method, system, device and medium

Abstract

The application discloses a target tracking method, system, device and medium based on a correlation filter, and the target tracking method comprises the following steps: acquiring a first frame image and target tracking related information; wherein the target tracking related information specifically comprises at least position information and scale information of a target to be tracked; initializing a correlation filter based on the acquired first frame image and target tracking related information, and obtaining an initialized correlation filter; frame by frame acquiring images and performing target tracking and correlation filter updating based on the obtained initialized correlation filter, and finally obtaining a target tracking result. In the technical scheme of the application, the correlation filter combined with multi-feature fusion and online re-detection is used for target tracking, so that more robust and stable target tracking can be realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of video image processing, and relates to the field of target tracking, and particularly relates to a target tracking method, system, device and medium based on a correlation filter. BACKGROUND

[0002] Target tracking is one of the research hotspots in the field of video image processing, and is widely applied in the fields of aerospace, security and protection, transportation, military, etc. Target tracking is to estimate the relevant information of the selected target continuously according to the target information concerned in the first frame. Due to the motion change of the target itself and the environment where the target is located and many other factors, target tracking has always been a challenging problem, and the main challenges are the disappearance of the field of view, occlusion, deformation, etc.

[0003] With the continuous development of tracking algorithms, the discriminative target tracker based on correlation filtering has attracted widespread attention from scholars due to its relatively accurate tracking results and excellent tracking efficiency. It regards the tracking problem as a classification task and distinguishes the target from the background around the target.

[0004] In recent years, great progress has been made in the research on the discriminative target tracking algorithm based on correlation filtering. Among them, the Minimum Output Sum of Squared Error (MOSSE) filter proposed by D.S. Blome lays the foundation for the development of subsequent algorithms as the pioneer of the correlation filtering tracking algorithm. Since MOSSE causes the problem of sample redundancy by initializing the filter with a set of samples obtained through affine transformation, J.F. Henriques proposed a correlation filter tracker based on kernel circulant structure (Exploiting the Circulant Structure of Tracking-by-Detection with Kernels, CSK), which mainly includes an approximate dense sampling method based on circular shift, ridge regression and kernel method. It fully utilizes the circulant matrix and kernel method, and converts to the frequency domain for calculation, which not only realizes dense sampling but also reduces the time complexity.

[0005] However, the above-mentioned MOSSE and CSK only use single-channel image grayscale features. M. Danelljan et al. added color features on the basis of CSK, thereby obtaining an adaptive color feature target tracker (Adaptive Color Attributes for Real-Time Visual Tracking, CN); and high-speed kernel correlation filtering (High-Speed Tracking with Kernelized Correlation Filters, KCF) uses multi-channel Histograms of Oriented Gradients (HOG) features, thereby improving the ability to represent the appearance of a target and achieving more excellent tracking performance.

[0006] On the basis of the above research, due to the limitations of a single feature, the appearance of a target cannot be sufficiently described, which leads to inaccurate estimation of tracking results. Therefore, many scholars have carried out research on multi-feature fusion in order to achieve the purpose of taking the advantages and making up the shortcomings. Among them, L. Bertinett et al. proposed a complementary learner (Sum of Template And Pixel-wise Learners, Staple) for real-time tracking, which combines HOG features and color histogram features, and achieves good comprehensive tracking results. However, the fixed weight weighting method is used to fuse the two features, which cannot realize adaptive tracking of the tracker, and the robustness is poor in various tracking scenes.

[0007] In addition, the boundary effect problem caused by the way of cyclic sampling to obtain training samples of the kernel correlation filter suppresses the performance of the tracker. Many scholars have carried out research on this problem. Among them, H. K. Galoogahi et al. proposed a background-aware correlation filter (Background-Aware Correlation Filters, BACF) to improve the proportion of real samples by constraining the space, thereby weakening the influence of the boundary effect. However, it only uses HOG features, and performs poorly in complex tracking scenes such as motion blur and fast motion. The tracker is prone to drift, and the robustness is poor. In the existing target tracking process, online model updating is usually performed. The above-mentioned trackers update the model by weighting the information of the current frame and the previous frame, and do not judge the estimated results of the tracker. If the current frame tracking fails, the tracker will learn the wrong target samples, thereby causing rapid drift of the model, and subsequent tracking of the target will fail, i.e., tracking failure. SUMMARY

[0008] The present application aims to provide a target tracking method, system, device and medium based on a correlation filter, to solve one or more technical problems existing in the prior art. In the technical solution of the present application, target tracking is performed based on a correlation filter combined with multi-feature fusion and online re-detection, so that more robust and stable target tracking can be achieved.

[0009] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0010] The present application provides a target tracking method based on a correlation filter in the first aspect, comprising the following steps:

[0011] Obtaining a first frame image and target tracking related information; wherein the target tracking related information specifically comprises at least position information and scale information of the target to be tracked;

[0012] Initializing the correlation filter based on the obtained first frame image and target tracking related information, to obtain an initialized correlation filter;

[0013] Based on the obtained initialized correlation filter, obtaining an image frame by frame and performing target tracking and correlation filter updating, to finally obtain a target tracking result;

[0014] The step of initializing the correlation filter comprises:

[0015] Based on the obtained first frame image and target tracking related information, obtaining original image data containing the target to be tracked and part of the background information and performing feature extraction, to obtain three feature images of the first frame image, i.e., a histogram of oriented gradients, color naming and grayscale; based on the three feature images of the first frame image, i.e., the histogram of oriented gradients, color naming and grayscale, minimizing a target function of a real-time filter to obtain an initialized real-time filter; based on the obtained first frame image and target tracking related information, obtaining original image data containing only the target to be tracked, and based on the original image data containing only the target to be tracked, minimizing a target function of a template filter to obtain an initialized template filter; based on the obtained first frame image and target tracking related information, obtaining positive and negative target samples with the same scale size as the target to be tracked, and by minimizing a target function of a target sample classifier, an initialized target sample classifier is obtained; thus, the initialization of the correlation filter is completed.

[0016] The method is further improved in that, in the step of obtaining an image frame by frame based on the obtained initialized correlation filter, performing target tracking and correlation filter updating, and finally obtaining a target tracking result, the step of obtaining the A-th frame image, performing target tracking and correlation filter updating comprises:

[0017] Obtaining the A-th frame image to be tracked;

[0018] According to the to-be-tracked target related information of the A-th frame image and the A-1-th frame image, original image data containing the to-be-tracked target and part of background information in the A-th frame image is acquired and feature extraction is performed, to obtain three feature images of a direction gradient histogram, color naming and grayscale corresponding to the original image data containing the to-be-tracked target and part of background information in the A-th frame image; the three feature images of the direction gradient histogram, color naming and grayscale corresponding to the original image data containing the to-be-tracked target and part of background information in the A-th frame image are respectively subjected to correlation operation with the real-time filter obtained based on the A-1-th frame image, to obtain respective feature responses; based on the obtained respective feature responses, feature response evaluation is performed to obtain a feature response weight matrix; based on the feature response weight matrix, a feature response adaptive weight is obtained; based on the feature response adaptive weight, the respective feature responses are adaptively fused to obtain a multi-feature adaptive fusion response; according to the multi-feature adaptive fusion response, a target initial estimation result of the A-th frame image is obtained.

[0019] According to the to-be-tracked target related information of the A-th frame image and the A-1-th frame image, original image data containing only the to-be-tracked target in the A-th frame image is acquired and feature extraction is performed, to obtain three feature images of a direction gradient histogram, color naming and grayscale corresponding to the original image data containing only the to-be-tracked target in the A-th frame image; the three feature images of the direction gradient histogram, color naming and grayscale corresponding to the original image data containing only the to-be-tracked target in the A-th frame image are subjected to correlation operation with the template filter obtained based on the A-1-th frame image, to obtain a first consistency parameter; whether to perform online re-detection is determined according to the first consistency parameter, if online re-detection is performed, sample collection is performed on the A-th frame image through a sliding window, to obtain a to-be-classified target sample; the to-be-classified target sample is subjected to high-score target positive sample acquisition through a sample classifier obtained based on the A-1-th frame image, target consistency verification is performed on the high-score positive sample through the template filter obtained based on the A-1-th frame image, to obtain a second consistency parameter, whether to update the target initial estimation result of the A-th frame image as a target final estimation result of the A-th frame image is determined based on the second consistency parameter;

[0020] Whether to update the template filter and the sample classifier is determined according to the first consistency parameter or the second consistency parameter, if yes, the same steps as the initialization are adopted, online training and update learning are completed based on the A-th frame image and the target final estimation result of the A-th frame image, to obtain the template filter and the sample classifier of the A-th frame image; if no, the template filter and the sample classifier of the A-th frame image are the template filter and the sample classifier of the A-1-th frame image; the same steps as the initialization are adopted, online training and update learning are completed based on the A-th frame image and the target final estimation result of the A-th frame image, to obtain a real-time filter of the A-th frame image.

[0021] The method of the present application is further improved in that the expression of the objective function of the real-time filter and the template filter is,

[0022]

[0023] In the formula, is a binary mask matrix, and is used to crop a sample of size Mx1 from , N>>M; [Delta]tau j is a circular shift operator, Px d [Delta]tau j represents that a sample of size Mx1 is cropped from all the shifted samples of x d ; the superscript T of the filter represents the conjugate transpose of a complex vector or matrix, y is an ideal response; N is the length of x, D is the number of characteristic channels, lambda is a regularization parameter; j=[1,…,N], d=[1,…,D]; E(·) represents an objective function;

[0024] When the objective function is solved, formula (1) is converted into the following frequency domain expression according to the Parseval theorem,

[0025]

[0026] In the formula, I D is a unit matrix of size DxD; represents a Kronecker product; represents that a discrete Fourier transform is performed on g, is an orthogonal matrix used to convert a sample to the frequency domain;

[0027] Formula (2) is solved by using the augmented Lagrange method, and the Lagrange expression is,

[0028]

[0029] In the formula, mu is a penalty coefficient; is a Lagrange vector defined in the frequency domain;

[0030] Formula (3) is solved by the alternating direction multiplier method, and the problem is converted into a sub-problem with an analytical solution. When the sub-problem is solved, it is decomposed into multiple independent objective functions, and the solution is,

[0031]

[0032] In the formula, the complexity is D is the number of channels, Nlog(N) is the calculation overhead of inverse Fourier transform;

[0033]

[0034] In the formula, b is a scalar size are respectively elements in the matrix, n = [1, …, N];

[0035] The update strategy of the Lagrange vector is:

[0036]

[0037] In the formula, The superscript (k+1) represents the k+1th iteration of the ADMM algorithm; the value strategy of the penalty coefficient μ is μ (k+1) = min(μ max , χμ (k) ), μ max is the maximum value of the penalty coefficient, χ is a constant coefficient, and min(·) is the minimum value.

[0038] Further improvement of the method is that the expression of the objective function of the target sample classifier is

[0039]

[0040] In the formula, <h,q> represents the inner product between h and q; {(q f ,b f )|f = 1, 2, …, F} is a training set, q f is the HOG and CN feature vector generated by the fth sample, b f ∈{+1,-1} is a label, F is the total number of samples; max{·} is the maximum value.

[0041] Further improvement of the method is that, based on the obtained feature response, the feature response evaluation is performed to obtain a feature response weight matrix; based on the feature response weight matrix, a feature response adaptive weight is obtained; based on the feature response adaptive weight, the feature responses are adaptively fused to obtain a multi-feature adaptive fusion response; in the step of obtaining the target initial estimation result of the A frame image according to the multi-feature adaptive fusion response,

[0042] The peak side lobe ratio, the average peak correlation energy and the main peak significance are used as three evaluation indexes for feature response evaluation; wherein,

[0043]

[0044] In the formula, the subscript {1, 2, …, i} is the serial number of arranging all response peaks in descending order of response value; O represents the Euclidean distance between the main peak coordinate and the rest of the peak coordinates, S represents the maximum response value of each peak, sigma is the standard deviation, and lambda SOMP is a normalized constant coefficient;

[0045] The obtained characteristic response weight matrix is

[0046]

[0047] The adaptive weight weighted sum of the characteristic response according to the weight matrix is obtained, and the multi-feature fusion response result and the target initial estimation result are obtained, and the expression is

[0048] R fusion = wH * R HOG + wC * R CN + wG * R Gray ;

[0049] In the formula, R fusion is a multi-feature adaptive fusion response, R HOG is a HOG feature response, R CN is a CN feature response, and R Gray is a gray scale feature response; wH is a direction gradient histogram feature response weight, wC is a color naming feature response weight, and wG is a gray scale feature response weight.

[0050] Further improvement of the method of the application is that in the step of obtaining the first consistency parameter and the second consistency parameter, the calculation expression of the consistency parameter is

[0051]

[0052] In the formula, max(·) represents the maximum value of the response, represents the consistency response of the A-th frame, mean(·) is the mean value, A is the current frame number, and a is any constant integer between 2 and A-1.

[0053] Further improvement of the method of the application is that in the step of obtaining the first consistency parameter and the second consistency parameter, the calculation expression of the consistency parameter is

[0054] When the first consistency parameter is less than the threshold T rd , an online re-detection operation is performed; if the online re-detection operation is performed, the high-score target sample obtained by the sample classifier based on the A-1th frame image is selected, the first L samples are passed through the template filter again for re-determination, if the second consistency parameter of the re-detected high-score positive sample is greater than T rp , the initial estimated target information is updated by using the high-score positive sample;

[0055] In the step of determining whether to update the template filter and the sample classifier based on the first consistency parameter or the second consistency parameter, and completing online training and update learning based on the Ath frame image and the final estimated result of the target of the Ath frame image to obtain the real-time filter of the Ath frame image,

[0056] When the first consistency parameter or the second consistency parameter is greater than T up , the template filter and the sample classifier are updated;

[0057] The update strategy of the real-time filter and the template filter is,

[0058]

[0059] In the formula, is a model obtained after the Ath frame is updated, is a model trained according to the final target estimated information of the Ath frame, and η is an update learning rate;

[0060] The sample classifier is updated, and the hyperplane h update strategy is,

[0061]

[0062] In the formula, is the gradient of the loss function with respect to the hyperplane, and γ is a learning rate parameter.

[0063] The second aspect of the present application provides a target tracking system based on a correlation filter, comprising:

[0064] A data acquisition module is configured to acquire a first frame image and target tracking related information; wherein the target tracking related information specifically includes at least position information and scale information of the target to be tracked.

[0065] An initialization module is configured to initialize the correlation filter based on the acquired first frame image and target tracking related information, and obtain an initialized correlation filter.

[0066] A tracking and updating module is configured to acquire images frame by frame and perform target tracking and correlation filter updating based on the obtained initialized correlation filter, and finally obtain a target tracking result.

[0067] The third aspect of the present application provides an electronic device, comprising:

[0068] at least one processor; and,

[0069] a memory in communication with the at least one processor; wherein,

[0070] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the target tracking method based on the correlation filter according to any one of the first aspect of the present application.

[0071] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the target tracking method based on the correlation filter according to any one of the first aspect of the present application.

[0072] Compared with the prior art, the present application has the following beneficial effects:

[0073] In the technical solution of the present application, the target tracking is performed based on the correlation filter combined with multi-feature fusion and online re-detection, which can realize more robust and stable target tracking. Specifically, in view of the problem that the single HOG feature is limited in describing the target appearance under some tracking challenges, thereby greatly increasing the risk of tracking failure, the technical means of jointly enhancing the description ability and discrimination ability of the target appearance of the tracker by multiple features is adopted in the present application, which has the characteristics of fully exploiting the target information and fully exerting the inherent characteristics of different features, and can effectively improve the stability and robustness of the tracker under different tracking scenes to solve the above problems. In view of the problems that the target tracking fails due to the challenges such as target occlusion and field of view disappearance, and the accumulation of tracking errors, and the target cannot be continuously tracked after recovery, the technical means of online re-detection of the tracking target is adopted in the technical solution of the present application, which has the characteristics of adaptively judging the target tracking state and online re-detecting the target related position and scale information, and can improve the stability of the tracker in the tracking process, reduce the risk of tracker drift, and improve the overall tracking performance.

[0074] In view of the problem that the linear weighted fusion of multi-feature response fusion is not good, and the complementary properties of multiple features cannot be fully exerted, a novel feature response evaluation index SOMP is designed in the present application, which describes the geometric characteristics of the response graph and the peak information of the response graph, and pays more attention to the robustness of the response, and can effectively improve the accuracy of the tracker in discriminating and fusing multiple feature responses to realize more accurate target tracking. BRIEF DESCRIPTION OF DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following briefly introduces the drawings needed to be used in the embodiments or prior art description; obviously, the drawings described below are some embodiments of the present application, and the ordinary skilled in the art can also obtain other drawings according to these drawings without creative effort.

[0076] Figure 1 is a flowchart of a target tracking method based on a correlation filter provided by an embodiment of the present application;

[0077] Figure 2 is a schematic diagram of the overall structure of a correlation filter in an embodiment of the present application;

[0078] Figure 3 is a schematic diagram of an SOMP response evaluation index in an embodiment of the present application;

[0079] Figure 4 is a schematic diagram of the main structure of a re-detection module in an embodiment of the present application;

[0080] Figure 5 is a schematic diagram of the decision flow of a decision module in an embodiment of the present application;

[0081] Figure 6 is a schematic diagram of the accuracy rate curve and the success rate curve of a correlation filter in an embodiment of the present application; wherein, Figure 6 (a) in the figure is a schematic diagram of a comprehensive tracking accuracy rate curve, Figure 6 (b) in the figure is a schematic diagram of a comprehensive tracking success rate curve;

[0082] Figure 7 is a schematic diagram of experimental results under a representative tracking challenge in an embodiment of the present application; wherein, Figure 7 (a) in the figure is a schematic diagram of a tracking accuracy rate curve under a deformation challenge, Figure 7 (b) in the figure is a schematic diagram of a tracking success rate curve under a deformation challenge, Figure 7 (c) in the figure is a schematic diagram of a tracking accuracy rate curve under a field-of-view disappearance challenge, Figure 7 (d) in the figure is a schematic diagram of a tracking success rate curve under a field-of-view disappearance challenge, Figure 7 (e) in the figure is a schematic diagram of a tracking accuracy rate curve under an occlusion challenge, Figure 7 (f) in the figure is a schematic diagram of a tracking success rate curve under an occlusion challenge;

[0083] Figure 8 is a tracking schematic diagram of a correlation filter in an embodiment of the present application; wherein, Figure 8 (a) in the figure is a schematic diagram of the tracking results of a video sequence ball_ce2, Figure 8Fig. 2 is a schematic diagram of tracking results of a video sequence Lemming, Figure 8 Fig. 3 is a schematic diagram of tracking results of a video sequence Woman. DETAILED DESCRIPTION

[0084] The application will be further described below in conjunction with the accompanying drawings:

[0085] Referring to Figure 1 The target tracking method based on the correlation filter provided by the embodiment of the application specifically comprises the following steps:

[0086] Step 1: obtaining a first frame image and target tracking related information; wherein the target tracking related information specifically comprises position information and scale information of the target to be tracked;

[0087] Step 2: initializing the correlation filter based on the first frame image and the target tracking related information obtained in step 1, and obtaining an initialized correlation filter;

[0088] Step 3: obtaining an image frame by frame and tracking the target based on the initialized correlation filter obtained in step 2, and obtaining a target tracking result;

[0089] In step 2, the step of initializing the correlation filter specifically comprises:

[0090] Step 2.1: obtaining original image data containing the target to be tracked and partial background information based on the obtained first frame image and the target tracking related information; extracting a histogram of oriented gradients, color naming and grayscale of the first frame image based on the original image data containing the target to be tracked and partial background information; for example, the first frame image is cropped according to the target tracking related information in the first frame image, and the original image data containing the target to be tracked and partial background information is obtained;

[0091] Step 2.2: minimizing and solving a target function of a real-time filter based on the histogram of oriented gradients, color naming and grayscale of the first frame image, and obtaining an initialized real-time filter;

[0092] Step 2.3: obtaining original image data containing only the target to be tracked based on the obtained first frame image and the target tracking related information; minimizing and solving a target function of a template filter based on the original image data containing only the target to be tracked, and obtaining an initialized template filter;

[0093] Step 2.4, based on the first frame image and the information related to the target to be tracked, obtaining positive and negative target samples with the same size as the target to be tracked, and obtaining the initialized target sample classifier by minimizing the objective function of the target sample classifier; finally completing the initialization of the correlation filter.

[0094] In step 3, the step of obtaining images frame by frame and tracking the target, the step of tracking the target in the A-th frame image among the second frame image to the Z-th frame image includes:

[0095] Step 3.1, obtaining the A-th frame image to be tracked;

[0096] Step 3.2, obtaining the original image data containing target and partial background information in the A-th frame image according to the information related to the target to be tracked in the A-th frame image and the A-1-th frame image; based on the original image data containing target and partial background information in the A-th frame image, extracting three feature images of direction gradient histogram, color naming and grayscale corresponding to the original image data containing target and partial background information in the A-th frame image; performing correlation operation on the three feature images of direction gradient histogram, color naming and grayscale corresponding to the original image data containing target and partial background information in the A-th frame image and the real-time filter obtained based on the A-1-th frame image respectively to obtain each feature response; based on the obtained each feature response, performing feature response evaluation to obtain a feature response weight matrix; based on the feature response weight matrix, obtaining a feature response adaptive weight; based on the feature response adaptive weight, adaptively fusing each feature response to obtain a multi-feature adaptive fusion response; and obtaining the initial estimation result of the target in the A-th frame image according to the multi-feature adaptive fusion response.

[0097] Step 3.3, obtaining original image data containing only the target in the A-th frame image according to the target-related information of the A-th frame image and the A-1-th frame image; extracting three feature images of a direction gradient histogram, color naming and grayscale corresponding to the original image data containing only the target in the A-th frame image; performing correlation operation on the three feature images corresponding to the original image data containing only the target in the A-th frame image and the template filter obtained based on the A-1-th frame image to obtain a first consistency parameter; judging whether online re-detection is needed according to the first consistency parameter; if online re-detection is needed, performing sample collection on the A-th frame image through a sliding window to obtain a target sample to be classified; obtaining a high-score positive sample of the target sample to be classified through a sample classifier obtained based on the A-1-th frame image, performing target consistency test on the high-score positive sample through the template filter obtained based on the A-1-th frame image to obtain a second consistency parameter, and judging whether to update the target initial estimation result of the A-th frame image as the target final estimation result of the A-th frame image based on the second consistency parameter;

[0098] Step 3.4, judging whether to update the template filter and the sample classifier according to the first consistency parameter or the second consistency parameter, if yes, the same steps as the initialization are adopted to complete online training and update learning based on the A-th frame image and the target final estimation result of the A-th frame image to obtain the template filter and the sample classifier of the A-th frame image, if no, the template filter and the sample classifier of the A-th frame image are the template filter and the sample classifier of the A-1-th frame image, and the same steps as the initialization are adopted to complete online training and update learning based on the A-th frame image and the target final estimation result of the A-th frame image to obtain the real-time filter of the A-th frame image.

[0099] In the target tracking method described above, the target is tracked based on the correlation filter combining multi-feature fusion and online re-detection, and more robust and stable target tracking can be realized.

[0100] The target tracking method based on the correlation filter provided in the embodiment of the application specifically includes the following steps.

[0101] Step one, obtaining a first high-resolution image of a target through a visible light image acquisition device, and framing a target to be tracked in the initial first frame;

[0102] Step two, cropping the first frame image according to the related information of the first target to obtain image original data containing the target and part of background information, and extracting three feature images of a direction gradient histogram, color naming and grayscale;

[0103] Step three, initializing the real-time filter; wherein, a Gaussian target response label is set according to a feature image containing a target and a background, a cyclic sampling of the obtained feature image is realized in a frequency domain, a target function is optimized and solved by an alternating direction multiplier method, relevant parameter information of the real-time filter is obtained, online training of the real-time filter is realized, and initialization of the real-time filter is completed;

[0104] Step four, initializing the template filter; wherein, a feature image containing only a target is obtained by cropping the feature image according to target related information, and the template filter is initialized synchronously with step three;

[0105] Step five, initializing the target sample classifier; wherein, positive and negative target samples of the same size as the target are obtained by cropping the feature image according to target related information, relevant parameter information of a classifier hyperplane is obtained by minimizing a classifier target function, and the target sample classifier is initialized;

[0106] Step six, a new frame of high-resolution image is obtained by the optical image acquisition device, and the image is subjected to the operation of step two according to the final target related information of the previous frame to obtain relevant feature images;

[0107] Step seven, each feature response is obtained by performing relevant operations on the feature images and the real-time filter, a feature response weight matrix is obtained by performing feature response evaluation, a feature response adaptive weight is obtained, multi-feature adaptive fusion is completed, and an initial target estimation result is obtained according to the fusion response;

[0108] Step eight, a relevant operation is performed on the feature image containing only the target cropped according to the initial target estimation result and the template filter to obtain a consistency parameter, and it is determined whether to perform online re-detection according to the consistency parameter;

[0109] Step nine, if online re-detection is needed, a sample is collected by a sliding window on the current frame feature image, a high-resolution positive sample is obtained by the sample classifier, a target consistency test is performed on the high-resolution positive sample by the template filter, a consistency parameter is obtained, and it is determined whether to update the initial target estimation result as the final target estimation result of the current frame;

[0110] Step ten, it is determined whether to update the template filter and the sample classifier according to the consistency parameter, if yes, online training and update learning are completed synchronously with steps four and five, and online training and update learning of the real-time filter are completed synchronously with step three according to the target estimation result;

[0111] Step eleven, subsequent steps six to ten are performed to continuously track the target.

[0112] Please refer to Figure 2 , Figure 2The related filter overall structure logic diagram disclosed by the embodiment of the present application mainly comprises three parts: a multi-feature response analysis module, a multi-feature adaptive fusion module and an online re-detection module.

[0113] Among them, the workflow is first to extract multiple features of a target from an input image, to obtain multiple feature responses through a real-time correlation filter, to then distribute adaptive weights through the multi-feature adaptive fusion module to obtain a fusion response and initial target estimation information, and finally to analyze the initial target estimation information through the online re-detection module and determine whether re-detection and model updating operations are needed; the above steps are repeated in subsequent video frames.

[0114] In the embodiment of the present application, in the multi-feature response analysis module:

[0115] Features are tools for describing the appearance of a target, and various features have different characteristics and will obtain more accurate appearance information in the tracking scene they adapt to. The correlation filter based on background perception can effectively handle the boundary effect problem in the correlation filter-based target tracker, but only a single HOG feature is used to describe the target, and due to the limitations of the feature itself, the risk of tracking failure will greatly increase in some tracking challenges such as motion blur and fast motion. By combining multiple features, accurate description of the target appearance in different tracking scenes is achieved, the robustness of the tracker is enhanced, and the overall tracking performance is improved.

[0116] The HOG feature can capture the edge or gradient structure of the local shape, and when the target translation or rotation is less than the size of the local region, the change of the feature description is small, and the stability is excellent, but it is very sensitive to fast motion and deformation. The color naming (CN) feature representing color visual information is robust to appearance changes and fast motion, but it is sensitive to light changes and background clutter. Since the inherent properties of the HOG feature and the CN feature can complement each other well, the gray feature is not sensitive to scale changes and has good robustness. Therefore, in order to enhance the discrimination ability of the tracker to the target and optimize the description of the target appearance, multi-feature analysis is performed on the target, so that more robust target tracking is achieved.

[0117] In order to realize a continuous target tracking process, it is very important to train a real-time correlation filter, which can obtain a feature response graph corresponding to different features by correlating different feature images with the current frame, so as to perform subsequent operations in tracking. In order to deal with the boundary effect problem and improve the tracking performance, the real-time correlation filter in the present application is obtained by minimizing the objective function shown in formula (1):

[0118]

[0119] where is a binary mask matrix, which can be cropped from with size M x 1, N » M; [Δτ j ] is a circular shift operator, Px d [Δτ j ] represents cropping from all shifted samples of x d with size M x 1; the superscript T of filter denotes the conjugate transpose of a complex vector or matrix, and y is the ideal response.

[0120] To improve computational efficiency, the correlation filter is usually trained in the frequency domain. Equation (1) can be converted to the following frequency domain expression according to the Parseval theorem:

[0121]

[0122] where I D is a D x D identity matrix; denotes the Kronecker product; denotes the discrete Fourier transform of g, which is specifically where is an orthogonal matrix used to convert the sample to the frequency domain.

[0123] The augmented Lagrangian method (ALM) is used to solve equation (2), and the Lagrangian expression is as follows:

[0124]

[0125] where μ is a penalty coefficient; is a Lagrangian vector defined in the frequency domain.

[0126] Equation (3) can be solved iteratively by the alternating direction multiplier method (ADMM), which can convert the problem into multiple sub-problems with analytical solutions. When solving the sub-problem , it is decomposed into multiple independent target solutions, and the solution result is:

[0127] The sub-problem w * :

[0128]

[0129] where The complexity is where D is the number of channels, and Nlog(N) is the computational overhead of the inverse Fourier transform;

[0130] The sub-problem

[0131]

[0132] wherein, b is a scalar size of wherein are elements in , n = [1, … N].

[0133] The update strategy of the Lagrange vector is:

[0134]

[0135] wherein, The superscript (k+1) represents the k+1th iteration of the ADMM algorithm; the value strategy of the penalty coefficient μ is μ (k+1) = min(μ max , χμ (k) ).

[0136] In the embodiment of the application, the corresponding feature response map can be obtained by performing correlation operation on different feature images of the current frame respectively through the trained real-time correlation filter, so as to realize adaptive multi-feature fusion operation through the multi-feature fusion module, and obtain a fusion response map which can fully exert the internal advantages of different features.

[0137] The multi-feature adaptive fusion module of the embodiment of the application comprises:

[0138] Under different tracking scenes, the estimated target displacement vectors obtained from the three feature response maps are different, and the fixed value weighting operation on the response map cannot fully exert the effect of multi-feature complementation; in this part, the application designs a more accurate response evaluation index, Significance Of Main Peak (SOMP), combines the remaining several existing evaluation indexes, realizes adaptive fusion of two feature responses through the construction of a feature response weight matrix, and further improves the tracking performance.

[0139] The feature response evaluation index is the key to multi-feature response fusion, and currently there are two evaluation indexes, Peak to Sidelobe Ratio (PSR) and Average Peak-to-Correlation Energy (APCE).

[0140] Both of the above two evaluation indexes reflect the quality of the response to some extent, but both have certain defects, and the ideal template class feature response is a single-peak three-dimensional shape, and both PSR and APCE only start from the response value and do not evaluate the geometric characteristics of the response map.

[0141] In the embodiment of the present application, a new response evaluation index SOMP is specifically disclosed, which mainly describes the geometric characteristics of the response map and the peak information of the response map, pays more attention to the robustness of the response than PSR and APCE, and makes a more rigorous judgment on the response map, and its expression is:

[0142]

[0143] In the formula, the subscript {1, 2, …, i} is the serial number of arranging all response peaks in descending order of response value; O represents the Euclidean distance between the main peak coordinates and the remaining peak coordinates, S represents the maximum response value of each peak, σ is the standard deviation, and λ SOMP is a normalized constant coefficient.

[0144] Please refer to Figure 3 , occlusion is a common tracking challenge, and different degrees of occlusion of the target will produce response maps of different quality; as Figure 3 shown, when the target is not disturbed too much, that is, the corresponding response quality is better, the SOMP value is larger, and when the target is disturbed by factors such as occlusion, the shape of the response map will change, and the SOMP value will also be significantly reduced; by using multiple response evaluation indexes to construct a multi-feature response weight matrix, the tracker can adaptively assign weights to three kinds of feature response maps in different tracking scenes, effectively utilize different feature response information, and avoid the risk of tracking failure; the weight matrix is shown in Table 1.

[0145] Table 1. Feature response weight matrix

[0146]

[0147]

[0148] The feature response weight matrix is evaluated by using three evaluation indexes for each feature respectively, to obtain the corresponding index feature weights wSH, wAH, wPH, wSC, wAC, wPC, wSG, wAG, and wPG. The index feature weights of the same feature are added to obtain the corresponding adaptive weights wH, wC, and wG. Finally, the multi-feature fusion response result and the initial target estimation result can be obtained by weighting and summing the feature responses according to the adaptive weights obtained by the weight matrix through formula (8).

[0149] R fusion = wH*R HOG + wC*R CN + wG*R Gray (8)

[0150] In the formula, R fusion , R HOG , and R CNR Gray are fusion response, HOG feature response, CN feature response and gray feature response respectively.

[0151] In the embodiment of the application, the robustness of the filter can be enhanced to a certain extent and the stability in different tracking scenes can be improved through adaptive fusion of multiple features.

[0152] In the online re-detection module of the embodiment of the application,

[0153] After obtaining the initial estimation result of the target, if the real-time filter is directly updated using the initial estimation result and continuous tracking is performed, the risk of tracker drift will greatly increase and the tracking may fail quickly, therefore, it is necessary to construct an online re-detection module to reduce the risk of such problems and increase the stability of the tracker.

[0154] Please refer to Figure 4 and Figure 5 , the main structure of the online re-detection module is shown in Figure 4 , which mainly includes three parts: template filter, decision module and SVM sample classification; wherein the template filter is trained by the target function shown in formula (1), mainly focusing on the target region of the image, by sending the initial estimation result into the template filter, a consistency response can be obtained, which can be obtained by formula (9):

[0155]

[0156] In the formula, max(·) represents the maximum value of the response, represents the consistency response of the A-th frame, and mean(·) is the mean value.

[0157] The consistency parameter reflects the reliability of the initial estimation result of the current frame, therefore, a series of decisions can be made according to the consistency parameter, and the decision flow of the decision module is shown in Figure 5 , in which the consistency parameter is used to make decisions such as re-detection and update. When the consistency parameter C is less than the threshold T rd , the re-detection operation will be performed, if the re-detection operation is performed, the high-score target sample obtained by the support vector machine (SVM) classification will be obtained, the first L samples are selected to pass through the template filter again to make a decision, if the consistency parameter of the re-detected high-score positive sample is greater than T rp , the high-score positive sample is used to update the initial estimation target information. In order to ensure the stability of the template filter and the SVM classifier, when the consistency parameter C is greater than Tup The real-time filter is updated in each frame, and the update strategy of the real-time filter and the template filter is as follows:

[0158]

[0159] wherein, is a model obtained after the tth frame is updated, is a model trained according to the final target estimation information of the tth frame, and η is an update learning rate.

[0160] When the consistency parameter of the initial estimation result of the target is low and re-detection is needed, the re-detection sample needs to be classified by the SVM classifier, and therefore an SVM classifier needs to be trained in the first frame. The training of the classifier needs a large number of positive and negative samples, and the intersection over union with the real target frame is greater than 0.9 and less than 0.1 is set as the positive sample and the negative sample, and therefore a training set {(p f ,b f )|f=1,2,…,F} with F samples can be obtained; wherein, p f is the HOG and CN feature vector generated by the fth sample, b f ∈{+1,-1} is a label, and the SVM classifier hyperplane h can be obtained by minimizing the objective function as shown in the following formula:

[0161]

[0162] wherein, <h,q> represents the inner product between h and q.

[0163] When the consistency parameter is greater than T up , the SVM classifier model is updated, and the update strategy of the hyperplane h is as follows:

[0164]

[0165] wherein, is the gradient of the loss function with respect to the hyperplane, and γ is a learning rate parameter.

[0166] When the target needs to be re-detected, the image region around the initial estimation target position is sampled by a sliding window and the feature information is extracted to form a re-detection sample; the high-score positive sample is selected by the SVM classifier, and then the template filter is used for determination again, and the decision module is further operated according to the consistency parameter, if greater than T rp , the re-detected target information is used to replace the initial tracking result, if greater than T up , the current target information is used to update the classifier online, and the re-detection process is realized.

[0167] In the embodiment of the present application, the initial estimation result of the filter can be further judged and corrected in real time by the re-detection module, which can greatly reduce the risk of tracking failure, improve the stability of tracking, and achieve more efficient, longer and more stable target tracking.

[0168] In a specific embodiment of the present application, the experimental operating system is Windows 10, the programming environment is MATLAB 2016, both of which run on a CPU (3.7 GHz), and the main experimental parameters are shown in Table 2:

[0169] Table 2. Main experimental parameters

[0170]

[0171] Experiments are performed on the TC128 dataset, which contains 11 tracking challenges, including fast motion, cluttered background, motion blur, deformation, illumination change, in-plane rotation, low resolution, occlusion, out-of-plane rotation, field of view disappearance, and scale change. The dataset toolbox mainly evaluates the tracker through two indicators, accuracy and success rate. The number of video frames whose estimated target position is less than a given threshold is counted, and the percentage of the total number of video frames is taken as the accuracy value corresponding to the threshold. According to different thresholds, an accuracy curve can be drawn. The success rate is the percentage of the number of successfully tracked frames to the total number of video frames. When the overlap rate of the estimated target box a and the real target box b is greater than a certain threshold, it is defined as successful tracking, where the overlap rate is defined as OS = |a∩b| / |a∪b|, and |·| represents the number of pixels in the region. Similarly, a success rate curve can be drawn according to different thresholds.

[0172] Please refer to Figure 6 Compared with multiple trackers, the experimental evaluation results are output by the dataset toolbox, and the obtained accuracy curve and success rate curve are shown in Figure 6 The tracker (ORMCF) involved in the embodiment of the present application has a large improvement compared with the remaining listed trackers, and the accuracy and success rate are improved by 11.2% and 7.1% respectively compared with the benchmark BACF tracker. The accuracy and success rate results under different challenges are shown in Tables 3 and 4.

[0173] Table 3. Accuracy under different tracking challenges of TC128

[0174]

[0175] Table 4. Success rate of algorithm under different tracking challenges of OTB2015

[0176]

[0177] Please refer toFigure 7 and Figure 8 As can be seen from Table 3 and Table 4, the tracker involved in the embodiment of the application has excellent performance under all challenges in the data set, and the experimental results under representative challenges are shown in Table 4, wherein the accuracy and success rate are significantly improved under challenges such as field of view disappearance and occlusion, and the accuracy and success rate are improved by 15.2% and 9.5% respectively for the field of view disappearance challenge, and by 12.5% and 8.9% respectively for the occlusion challenge. The tracking examples of the tracker involved in the embodiment of the application in some video sequences are shown in Table 5. Through comprehensive experiments, the tracker involved in the embodiment of the application can track the target in real time at a speed of 26fps, and has good tracking accuracy, and is a high-efficiency, stable and accurate real-time target tracker. Figure 7 Figure 8 As can be seen from Table 3 and Table 4, the tracker involved in the embodiment of the application has excellent performance under all challenges in the data set, and the experimental results under representative challenges are shown in Table 4, wherein the accuracy and success rate are significantly improved under challenges such as field of view disappearance and occlusion, and the accuracy and success rate are improved by 15.2% and 9.5% respectively for the field of view disappearance challenge, and by 12.5% and 8.9% respectively for the occlusion challenge. The tracking examples of the tracker involved in the embodiment of the application in some video sequences are shown in Table 5. Through comprehensive experiments, the tracker involved in the embodiment of the application can track the target in real time at a speed of 26fps, and has good tracking accuracy, and is a high-efficiency, stable and accurate real-time target tracker.

[0178] In summary, the embodiment of the application discloses a correlation filter combining multi-feature fusion and online re-detection, which can effectively solve the problem that the traditional tracker performs poorly in some complex tracking scenes. A main peak saliency feature response evaluation index is designed to improve the accuracy of multi-feature discrimination and fusion; a multi-feature adaptive fusion strategy is constructed to enhance the discrimination ability of the tracker to the target and improve the robustness of the tracker in different tracking scenes; an online re-detection module is constructed to enhance the stability of the tracker in the tracking process, reduce the risk of tracker drift, and improve the overall tracking performance. Through experiments on the TC128 data set, the experimental results show that the tracker has good overall performance and can realize relatively accurate real-time target tracking.

[0179] The following is an apparatus embodiment of the application, which can be used to execute the method embodiment of the application. For details not disclosed in the apparatus embodiment, please refer to the method embodiment of the application.

[0180] The target tracking system based on the correlation filter provided by the embodiment of the application comprises:

[0181] The data acquisition module is configured to acquire a first frame of image and target-to-be-tracked related information; wherein the target-to-be-tracked related information specifically comprises at least position information and scale information of the target-to-be-tracked.

[0182] The initialization module is configured to initialize the correlation filter based on the acquired first frame of image and target-to-be-tracked related information, and obtain an initialized correlation filter.

[0183] The tracking and updating module is configured to acquire images frame by frame and perform target tracking and correlation filter updating based on the obtained initialized correlation filter, and finally obtain a target tracking result.

[0184] ​In another embodiment of the present application, a computer device is provided, which comprises a processor and a memory, the memory is configured to store a computer program, the computer program comprises program instructions, and the processor is configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are particularly suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method process or a corresponding function; the processor in the embodiments of the present application can be used for the operation of the target tracking method based on the correlation filter.

[0185] In another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in the computer device, and is configured to store programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the computer device, and of course can also include the expansion storage medium supported by the computer device. The computer readable storage medium provides a storage space, and the storage space stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. One or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the target tracking method based on the correlation filter in the above embodiments.

[0186] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application rather than limit them, and although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A target tracking method based on a correlation filter, characterized in that, Includes the following steps: Acquire the first frame image and related information of the target to be tracked; wherein, the related information of the target to be tracked specifically includes at least the position information and scale information of the target to be tracked; Based on the acquired first frame image and the relevant information of the target to be tracked, the correlation filter is initialized to obtain the initialized correlation filter; Based on the obtained initial correlation filters, images are acquired frame by frame and target tracking and correlation filter updates are performed to finally obtain the target tracking result. The steps for initializing the correlation filter include: Based on the acquired first frame image and information related to the target to be tracked, raw image data containing the target and some background information is acquired and features are extracted to obtain three feature images: histogram of directional gradients, color naming, and grayscale. Based on the histogram of directional gradients, color naming, and grayscale feature images of the first frame image, the objective function of solving the real-time filter is minimized to obtain the initialized real-time filter. Based on the acquired first frame image and information related to the target to be tracked, raw image data containing only the target to be tracked is acquired. Based on the raw image data containing only the target to be tracked, the objective function of solving the template filter is minimized to obtain the initialized template filter. Based on the acquired first frame image and information related to the target to be tracked, positive and negative target samples of the same scale as the target to be tracked are acquired. By minimizing the objective function of solving the target sample classifier, the initialized target sample classifier is obtained. Thus, the initialization of the relevant filters is completed.

2. The target tracking method based on a correlation filter according to claim 1, characterized in that, In the step of acquiring images frame by frame and performing target tracking and correlation filter updates based on the obtained initial correlation filter, the steps of acquiring the A-th frame image, performing target tracking, and updating the correlation filter include: Acquire the A-th frame image of the target to be tracked; Based on the target-to-be-tracked information in the A-th and A-1-th frames, the original image data containing the target and some background information in the A-th frame is obtained and its features are extracted. This yields three feature images: histogram of directional gradients, color naming, and grayscale, corresponding to the original image data. These feature images are then correlated with the real-time filter obtained from the A-1-th frame to obtain various feature responses. Based on these responses, a feature response weight matrix is ​​obtained through feature response evaluation. Adaptive weights are then derived from this matrix. Finally, adaptive fusion of these weights yields a multi-feature adaptive fusion response. Finally, the initial target estimation result for the A-th frame is obtained based on this multi-feature adaptive fusion response. Based on the target-to-be-tracked information in the A-th and A-1-th frames, the original image data containing only the target in the A-th frame is obtained and its features are extracted to obtain three feature images: histogram of orientation gradients, color name, and grayscale corresponding to the original image data containing only the target in the A-th frame. These three feature images are then correlated with a template filter obtained from the A-1-th frame to obtain a first consistency parameter. Based on the first consistency parameter, it is determined whether online re-detection should be performed. If online re-detection is performed, samples are collected from the A-th frame using a sliding window to obtain target samples to be classified. The target samples to be classified are then processed by a sample classifier obtained from the A-1-th frame to obtain high-scoring positive samples. The template filter obtained from the A-1-th frame is used to perform target consistency checks on the high-scoring positive samples to obtain a second consistency parameter. Based on the second consistency parameter, it is determined whether to update the initial target estimation result of the A-th frame as the final target estimation result of the A-th frame. Whether to update the template filter and sample classifier is determined based on the first consistency parameter or the second consistency parameter. If updated, the same steps as initialization are used to complete online training and update learning based on the A-th frame image and the final target estimation result of the A-th frame image to obtain the template filter and sample classifier of the A-th frame image. If not updated, the template filter and sample classifier of the A-th frame image are the template filter and sample classifier of the (A-1)-th frame image. The same steps as initialization are used to complete online training and update learning based on the A-th frame image and the final target estimation result of the A-th frame image to obtain the real-time filter of the A-th frame image.

3. The target tracking method based on a correlation filter according to claim 2, characterized in that, The objective function expressions for both the real-time filter and the template filter are . In the formula, It is a binary mask matrix used to extract data from... The sample size is M×1, where N >> M; [Δτ] j ] is the cyclic shift operator, Px d [Δτ j ] indicates from x d A sample of size M×1 is obtained by cropping from all shifted samples; filter The superscript T denotes the conjugate transpose of a complex vector or matrix; y is the ideal response; N is the length of x; D is the number of feature channels; λ is the regularization parameter; j = [1,…,N]; d = [1,…,D]; E(·) represents the objective function. When training and solving the objective function, equation (1) is transformed into the following frequency domain expression according to Pasvald's theorem. In the formula, I D It is an identity matrix of size D×D; Indicates the Kronecker product; This indicates that a discrete Fourier transform is performed on g. It is an orthogonal matrix used to transform samples to the frequency domain; The augmented Lagrange method is used to solve equation (2), and its Lagrange expression is: In the formula, μ is the penalty coefficient; It is a Lagrange vector defined in the frequency domain; Equation (3) is solved iteratively using the alternating direction multiplier method, transforming the problem into a subproblem w with an analytical solution. * , Solving subproblems When this is done, it is decomposed into multiple independent objectives for solution, and the solution result is: In the formula, The complexity is D is the number of channels, and Nlog(N) is the computational cost of the inverse Fourier transform. In the formula, b is a scalar of size They are The elements in the array, n = [1, ..., N]; The update strategy for the Lagrange vector is as follows: In the formula, The superscript (k+1) indicates the (k+1)th iteration of the ADMM algorithm; the strategy for choosing the value of the penalty coefficient μ is μ (k+1) =min(μ max ,χμ (k) ), μ max χ represents the maximum value of the penalty coefficient, χ is a constant coefficient, and min(·) represents the minimum value.

4. The target tracking method based on a correlation filter according to claim 3, characterized in that, The objective function of the target sample classifier is expressed as follows: In the formula, <h,q> represents the inner product between h and q; {(q f ,b f Let f = 1, 2, ..., F be the training set, and q be the training set. f b is the HOG and CN feature vector generated from the f-th sample. f ∈{+1,-1} are the labels, F is the total number of samples; max{·} is the maximum value.

5. The target tracking method based on a correlation filter according to claim 4, characterized in that, Based on the obtained feature responses, feature response evaluation is performed to obtain the feature response weight matrix; Based on the feature response weight matrix, adaptive weights for the feature response are obtained. Based on the adaptive weights of the feature responses, the feature responses are adaptively fused to obtain a multi-feature adaptive fusion response; In the step of obtaining the initial target estimation result of the A-th frame image based on the multi-feature adaptive fusion response... The characteristic response was evaluated using three evaluation indicators: peak-to-sidelobe ratio, mean peak correlation energy, and main peak significance. In the formula, the subscripts {1,2,…,i} represent the sequence numbers of all response peaks arranged in descending order of their response values; O represents the Euclidean distance between the coordinates of the main peak and the coordinates of the other peaks; S represents the maximum response value of each peak; σ is the standard deviation; and λ is the maximum response value of each peak. SOMP For normalized constant coefficients; The obtained feature response weight matrix contains two dimensions: features and metrics. The features include HOG, CN, and Gray, and the metrics include SOMP, APCE, PSR, and adaptive weights. The weights corresponding to SOMP are wSH, wSC, and wSG, respectively; the weights corresponding to APCE are wAH, wAC, and wAG, respectively; the weights corresponding to PSR are wPH, wPC, and wPG, respectively; and the weights corresponding to adaptive weights are wH, wC, and wG, respectively. The feature responses are adaptively weighted and summed according to the weight matrix to obtain the multi-feature fusion response result and the initial target estimation result, expressed as follows: R fusion =wH*R HOG +wC*R CN +wG*R Gray ; In the formula, R fusion For multi-feature adaptive fusion response, R HOG For HOG characteristic response, R CN For CN characteristic response, R Gray Gray-scale feature response; wH is the weight of the histogram of oriented gradients feature response, wC is the weight of the color naming feature response, and wG is the weight of the gray-scale feature response.

6. The target tracking method based on a correlation filter according to claim 5, characterized in that, In the steps of obtaining the first consistency parameter and the second consistency parameter, the expression for calculating the consistency parameter is as follows: In the formula, max(·) represents the maximum value of the response. Represents the consistency response of frame A, mean(·) is used to calculate the mean, A is the current frame number, and a is any constant integer between 2 and A-1; The first consistency parameter is used to determine whether to perform online re-detection. If online re-detection is performed, the A-frame image is sampled through a sliding window to obtain the target sample to be classified. The steps of classifying the target sample to be classified, obtaining high-scoring positive target samples through a sample classifier based on the (A-1)th frame image, performing target consistency checks on the high-scoring positive samples using a template filter based on the (A-1)th frame image to obtain a second consistency parameter, and determining whether to update the initial target estimation result of the Ath frame image based on the second consistency parameter as the final target estimation result of the Ath frame image include: When the first consistency parameter is less than the threshold T rd At that time, perform online re-detection. If online re-detection is performed, the first L high-scoring target samples obtained by the sample classifier based on the A-1 frame image are selected and re-evaluated using a template filter. If the second consistency parameter of the re-detected high-scoring positive sample is greater than T, the re-detection process continues. rp Then, high-scoring positive samples are used to update the initial estimated target information.

7. The target tracking method based on a correlation filter according to claim 6, characterized in that, In the steps of determining whether to update the template filter and sample classifier based on the first or second consistency parameter, and completing online training and update learning based on the final target estimation result of the A-th frame image to obtain the real-time filter for the A-th frame image, When the first consistency parameter or the second consistency parameter is greater than T up At that time, the template filter and sample classifier are updated; The update strategies for real-time filters and template filters are as follows: In the formula, The model obtained after updating to frame A. The model is trained based on the final target estimation information in frame A, and η is the update learning rate. The sample classifier is updated, and its hyperplane h update strategy is as follows: In the formula, Let γ be the gradient of the loss function with respect to the hyperplane, and γ be the learning rate parameter.

8. A target tracking system based on a correlation filter, characterized in that, include: The data acquisition module is used to acquire the first frame image and related information of the target to be tracked; wherein, the related information of the target to be tracked specifically includes at least the location information and scale information of the target to be tracked; The initialization module is used to initialize the correlation filter based on the acquired first frame image and the relevant information of the target to be tracked, so as to obtain the initialized correlation filter; The tracking and updating module is used to acquire images frame by frame based on the obtained initial correlation filters, perform target tracking and correlation filter updates, and finally obtain the target tracking result. The steps for initializing the correlation filter include: Based on the acquired first frame image and information related to the target to be tracked, raw image data containing the target and some background information is acquired and features are extracted to obtain three feature images: histogram of directional gradients, color names, and grayscale values ​​of the first frame image. Based on these three feature images, the objective function of the real-time filter is minimized to obtain the initialized real-time filter. Based on the acquired first frame image and information related to the target to be tracked, raw image data containing only the target to be tracked is acquired. Based on this raw image data, the objective function of the template filter is minimized to obtain the initialized template filter. Based on the acquired first frame image and information related to the target to be tracked, positive and negative target samples of the same scale as the target to be tracked are acquired. By minimizing the objective function of the target sample classifier, the initialized target sample classifier is obtained. Thus, the initialization of the relevant filters is completed. The step of acquiring images frame by frame and performing target tracking and correlation filter updates based on the obtained initial correlation filter to finally obtain the target tracking result includes the following steps: The steps of acquiring the A-th frame image for target tracking and correlation filter updates include: Acquire the A-th frame image of the target to be tracked; Based on the target-to-be-tracked information in the A-th and A-1-th frames, the original image data containing the target and some background information in the A-th frame is obtained and its features are extracted. This yields three feature images: histogram of directional gradients, color naming, and grayscale, corresponding to the original image data. These feature images are then correlated with the real-time filter obtained from the A-1-th frame to obtain various feature responses. Based on these responses, a feature response weight matrix is ​​obtained through feature response evaluation. Adaptive weights are then derived from this matrix. Finally, adaptive fusion of these weights yields a multi-feature adaptive fusion response. Finally, the initial target estimation result for the A-th frame is obtained based on this multi-feature adaptive fusion response. Based on the target-to-be-tracked information in the A-th and A-1-th frames, the original image data containing only the target in the A-th frame is obtained and its features are extracted to obtain three feature images: histogram of orientation gradients, color name, and grayscale corresponding to the original image data containing only the target in the A-th frame. These three feature images are then correlated with a template filter obtained from the A-1-th frame to obtain a first consistency parameter. Based on the first consistency parameter, it is determined whether online re-detection should be performed. If online re-detection is performed, samples are collected from the A-th frame using a sliding window to obtain target samples to be classified. The target samples to be classified are then processed by a sample classifier obtained from the A-1-th frame to obtain high-scoring positive samples. The template filter obtained from the A-1-th frame is used to perform target consistency checks on the high-scoring positive samples to obtain a second consistency parameter. Based on the second consistency parameter, it is determined whether to update the initial target estimation result of the A-th frame as the final target estimation result of the A-th frame. Whether to update the template filter and sample classifier is determined based on the first consistency parameter or the second consistency parameter. If updated, the same steps as initialization are used to complete online training and update learning based on the A-th frame image and the final target estimation result of the A-th frame image to obtain the template filter and sample classifier of the A-th frame image. If not updated, the template filter and sample classifier of the A-th frame image are the template filter and sample classifier of the (A-1)-th frame image. The same steps as initialization are used to complete online training and update learning based on the A-th frame image and the final target estimation result of the A-th frame image to obtain the real-time filter of the A-th frame image.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the target tracking method based on a correlation filter as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the target tracking method based on the correlation filter as described in any one of claims 1 to 7.

Images