Multi-feature fusion boundary correlation filtering target tracking method and storage medium

Abstract

The invention discloses a boundary correlation filtering target tracking method based on multi-feature fusion, which comprises the following steps: in a target position calculation part, adopting two paths with different image features for parallel tracking, and in a tracking path based on color distribution features, utilizing the distribution difference between a background and a target area to obtain a target position; a background suppression model and an interference area detection model are established to improve the overall anti-interference capability; then, in the fusion stage of the multi-feature response, for the problem that the fusion weight is fixed and single, the mean value difference between the target and the background probability score graph is utilized to achieve self-adaptive calculation of the fusion weight; finally, in a target scale processing portion, boundary correlation filtering is used to calculate target scale changes in tracking. The problems that an existing method is limited in application scene, poor in resistance to complex backgrounds and similar interferences, single and fixed in multi-feature fusion weight, limited in scale search range during scale calculation, large in calculation amount and the like are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to a multi-feature fused boundary correlation filtering target tracking method and a storage medium. Background Art

[0002] At present, target tracking technology is widely used and target tracking methods are also diverse, but existing target tracking methods still have many problems, such as:

[0003] (I) The features of traditional correlation filter target tracking methods are usually relatively simple, so the types of targets they are applicable to are limited, and they have weak anti-interference capabilities against complex backgrounds. When encountering complex situations, these methods lack certain scene judgment and processing capabilities, which can easily cause target loss. For example: MOSSE (Minimum Output Sum of Squared Error) method, CSK (Circulant Structure of Tracking-by-detection with Kernels) method, KCF (Kernelized Correlation Filters) method, etc.;

[0004] (ii) Other more advanced methods use fusion features to represent targets, but in feature fusion, there is a lack of powerful background suppression mechanism and foreground enhancement mechanism, and the multi-feature fusion method is relatively simple. For example: CN (Color Name) method, Staple (real-time tracking based on supplementary learning) method, etc.

[0005] (III) Most target tracking methods tend to focus only on the position of the target, while ignoring the calculation of the target scale, or when calculating the scale change of the target, the accuracy is not high enough and the calculation cost is too high. For example: SAMF (Scale Adaptive Kernel Correlation Filter Tracker with Multiple Feature) method, DSST (Discriminative Scale Space Tracker) method, etc. Summary of the invention

[0006] To solve the above problems, the present invention provides a target tracking method with boundary correlation filtering based on multi-feature fusion. Firstly, a probability perception model based on color histogram features is introduced, and the distribution difference characteristics of the background area and the target area are used to establish a background suppression model and an interference area detection model to improve the overall anti-interference ability; then, in view of the problem that the multi-feature response fusion weight is fixed and single, the mean difference between the target and background probability score maps is used to realize the adaptive calculation of the fusion weight; finally, in terms of scale processing, boundary correlation filtering is used to calculate the scale change of the target in tracking, which improves the calculation ability of the target scale change while reducing the amount of calculation.

[0007] Specifically, the technical solution of the present invention includes the following steps:

[0008] S1, perform target calibration in the first frame of the video sequence.

[0009] S2, initialize the color histogram probability perception model. When the first frame image is input, the background suppression model, interference area detection model, and background estimation model are initialized based on the calibrated target information.

[0010] S3, initializing the correlation filter model. When the first frame image is input, the template feature, template filter, boundary template feature, and boundary template filter vector are initialized by the calibrated target information.

[0011] S4, calculate the color histogram response and adaptive fusion parameters. For a frame of image containing the target to be detected, first extract the search image block based on the target information (target position, target scale) of the previous frame, and calculate its target probability image, integral image, and color histogram response. Then perform mixed sampling detection on the integral image to obtain the interference area and the optimal area, and then calculate the adaptive fusion parameters. Finally, train the color histogram perception model for use in the next frame of image.

[0012] S5, calculate the fusion response and locate the target position. For a frame containing the target to be detected, the target information (target position, target scale) of the previous frame is used as a reference to extract the candidate region features, and the Gaussian response is calculated by the Gaussian kernel function, and then the relevant filter response is calculated. Then the fusion response is calculated and its peak position is located to obtain the target position of the frame image to be detected.

[0013] S6, calculating the target scale. For a frame of image containing the target to be detected, a boundary image block is extracted with the detected target position as the center, and then a boundary correlation filter is used to calculate the target frame offset to obtain the target scale of the frame image to be detected.

[0014] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the multi-feature fusion boundary correlation filtering target tracking method described in the present invention are implemented.

[0015] The working principle of the present invention is as follows:

[0016] like Figure 4 As shown, the present invention designs two tracking paths to track the target in the image to be detected. One path uses a correlation filtering method based on HOG features to calculate the correlation filtering response, and the other path establishes a color histogram probability perception model to calculate the color histogram response. Finally, the responses of the two paths are adaptively fused to locate the target position. In this way, the respective advantages of HOG features and color features can be fully utilized, so that the present method can cope with various complex tracking scenarios.

[0017] In the color histogram probability perception model, the present invention improves the modeling method, introduces the background suppression histogram to further strengthen the foreground, and on this basis adds the interference region detection model to further suppress the interference of similar objects. In this way, in practical applications, if the appearance of some interference regions is similar to the target, the probability distribution of these pixels will be significantly suppressed in the interference region detection model, thereby reducing the impact of interference on target tracking.

[0018] In terms of the fusion method of multi-feature responses, the present invention abandons the fixed weight fusion method in the traditional method and adopts the probability mean in the optimal area to calculate the fusion parameters. In this way, the fusion weight of each frame image can be adaptively adjusted, avoiding manual adjustment of parameters.

[0019] In the calculation of the scale of the target, the present invention uses boundary correlation filtering to calculate the four offsets of the target frame, so that only four sub-image blocks need to be extracted from each frame of the image to complete the estimation of the scale of the target. Therefore, compared with other calculation methods that require the extraction of many sub-image blocks, this method can significantly reduce the amount of calculation.

[0020] The beneficial effects of the present invention are as follows:

[0021] (1) The HOG feature and the color histogram feature are integrated for tracking, which solves the problem that the existing method has a single feature and limited applicable scenarios;

[0022] (ii) A probabilistic perception model based on color histogram features was introduced, and the background suppression model and interference region detection model were established respectively by utilizing the distribution difference characteristics of the background area and the target area, which solved the problem that the existing methods have poor resistance to complex backgrounds and similar interferences;

[0023] (3) By utilizing the mean difference between the target and background probability score maps, the adaptive calculation of the fusion weight is realized, which solves the problem of a single fixed weight for multi-feature fusion in existing methods;

[0024] (iv) Boundary correlation filtering is introduced to realize the calculation of target scale size, which solves the problems of low accuracy and large amount of calculation in existing methods when performing scale calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the mixed sampling described in the technical solution, where (a) is sparse sampling, (b) is dense sampling, and the four color areas represent the four sampling ranges of each sampling method.

[0026] Figure 2 It is a schematic diagram of the extraction process of the fusion response described in the technical solution, in which the two tracking paths calculate their own responses independently, and then fuse them through adaptive fusion parameters to obtain the fusion response, which is used as the basis for target positioning.

[0027] Figure 3 This is a flow chart of the adaptive scale calculation based on boundary correlation filtering described in the technical solution.

[0028] Figure 4 It is a specific flow chart of target tracking in the embodiment, and is also an overall flow chart of the method of the present invention.

[0029] Figure 5 Schematic diagram of tracking results of the present method and the comparative method on a video sequence in the embodiment, wherein the tracking frames of different methods are represented by rectangular frames of different colors.

[0030] Figure 6 Graphs showing the tracking curves of the present method and the comparative method in batch data simulation in the embodiment, wherein (a) is the overlap rate curve, and (b) is the accuracy curve. The tracking curves of different methods are represented by lines of different colors and line types. DETAILED DESCRIPTION

[0031] In order to make the technical means, creative features and objectives achieved by the present invention easy to understand, the technical solution of the present invention is further explained below in combination with one of the embodiments and specific implementation methods of a multi-feature fusion boundary correlation filtering target tracking method given by the present invention.

[0032] See also Figure 4 As shown, a multi-feature fusion boundary correlation filtering target tracking method includes the following steps:

[0033] S1, in the first frame image of the video sequence, the target is calibrated to obtain the target position and target scale.

[0034] S2, initialize the color histogram probability perception model, including:

[0035] S2.1, initialize the background suppression model, the expression is:

[0036]

[0037] Where I represents an expanded image block containing the target area, T represents the target area, and S represents the background area outside T. v ) * Indicates the probability that a pixel v with a RGB combination value of C belongs to the target. They represent the number of pixels whose RGB value combination is C in the T area of ​​I and the number of pixels whose RGB value combination is C in the S area. ψ is the improved background suppression histogram of the present invention, and its expression is:

[0038]

[0039] Where ξ is the background suppression coefficient, which is used to control the degree of background suppression. |T| and |S| are the target area and background area, respectively. It should be noted that not only the initialization stage of the first frame image, but also the training stage of each subsequent frame image can be obtained by P(v∈T|T,S,C v ) * The expression training obtains a color probability distribution lookup table for the next frame detection.

[0040] The working principle of the background suppression model designed by this method is: From the expression of ψ, we can know that is the difference histogram between the normalized background color histogram and the target color histogram. If the element in the difference histogram is positive and the larger it is, the greater the difference in the distribution of the pixel points in the two histograms is. It is the color component in the background S, and the corresponding element in ψ is less than 1, then P(v∈T|T,S,C v ) * The smaller the calculation result is; on the contrary, if the element in the difference histogram is negative and smaller, it means that the pixel is the color component in the target T, then the corresponding element in ψ is greater than 1, then P(v∈T|T,S,C v ) * The larger the calculation result is, the more it can suppress the background and highlight the target.

[0041] S2.2, initialize the interference area detection model, the expression is:

[0042]

[0043] Where G represents the potential interference area, P(v∈T|T,G,Cv ) * Indicates the probability that a pixel v with a RGB combination value of C belongs to the target. represents the number of pixels in image block I whose RGB values ​​in G are combined into C. w is the enhancement weight. In the present invention, w is set to the number of G. It should be noted that not only the initialization stage of the first frame image, but also the training stage of each subsequent frame image can be obtained by P(v∈T|T,G,C v ) * The expression training obtains a color probability distribution lookup table for the next frame detection.

[0044] The working principle of the interference region detection model designed by this method is: assuming that the target region T and a set of potential interference regions G have been obtained, then P(v∈T|T,G,C v ) * represents the probability that a pixel is the target only in the target area T and in many interference areas G. Therefore, if an interference area D similar to the target is actually detected, the probability value of the pixel in it being the target will be in P(v∈T|T,G,C v ) * In addition, this method sets the enhancement weight w to the number of G rather than a fixed value, so that the model can adaptively adjust its enhancement ability according to the actual interference detected in each frame.

[0045] S2.3, initialize the background estimation model, the expression is:

[0046]

[0047] In the formula, P(v∈S|C v ) represents the probability that a pixel v with a RGB combination value of C belongs to the background S. They represent the number of pixels whose RGB value combination is C in the T region of I and the number of pixels whose RGB value combination is C in the S region of I. It should be noted that not only the initialization stage of the first frame image, but also the training stage of each subsequent frame image can be calculated by P(v∈S|C v ) expression training to obtain a color probability distribution lookup table for the next frame detection.

[0048] Different from the background suppression model and the interference area detection model, the background estimation model does not participate in the calculation of the target position and is only used for the adaptive calculation of the subsequent multi-feature response fusion weights.

[0049] S3, initializing the relevant filtering model, including:

[0050] S3.1, initialize the template feature x, which is extracted from the target information of the first frame image. In the present invention, the image feature is the HOG (Histogram of Oriented Gradient) feature.

[0051] S3.2, initialize the template filter α, its formula is:

[0052]

[0053] In the formula, F and F -1 Respectively represent Fourier transform and inverse transform. y is the standard output matrix, λ is the regularization parameter, k is the single sample Gaussian response matrix, and its expression is:

[0054]

[0055] Where σ is the Gaussian kernel response bandwidth and |||| represents the matrix norm. Represents an element-wise dot product operation.

[0056] S3.3, initialize the boundary template feature. With the known target position as the center, extract the four boundary image blocks of the target, namely the left, right, top and bottom, and calculate their 42-channel fusion features (including 32-channel HOG features and 10-channel color features), and vectorize and stack the fusion features. The expression is:

[0057]

[0058] In the formula, j = {l, r, t, b} represents the left, right, upper and lower boundaries respectively, and x j Represents the 42-channel fused features of the boundary image. The right side of the equal sign is the one-dimensional vector stacking form of the fused features, where the left and right boundary images are row vector stacks, the upper and lower boundary images are column vector stacks, and nl is the number of feature vectors.

[0059] S3.4, initialize the boundary template filter vector, the expression is:

[0060]

[0061] In the formula, y j is the standard output vector of the boundary image, and λ is the regularization parameter. j is a single sample Gaussian response vector, expressed as:

[0062]

[0063] In the formula, length() represents the length of the vector.

[0064] It should be noted that not only in the initialization phase of the first frame image, but also in the training phase of each subsequent frame image, the expressions in S3.2 to S3.4 can be used to train the template features and template filters for use in the next frame detection.

[0065] S4, calculates the color histogram response and adaptive fusion parameters, including:

[0066] S4.1, for the frame image to be detected, calculate the target probability image of the candidate image block, which is calculated by the background suppression model and interference area detection model that have been trained for the previous frame image, and the expression is:

[0067] M=τ·M T +(1-τ)·M G

[0068] Where M is the final target probability image, M T is the target probability image calculated according to the background suppression model, M G is the target probability image calculated according to the interference area detection model. τ is the fusion weight.

[0069] The design basis of the calculation formula of M in this method is: in practical applications, if the appearance of some interference areas is close to the target, these pixels are mapped to M T The probability values ​​in may be too large, but they map to M G The probability value in is close to 0, because such points are close to the target appearance, but are not the target, and will be considered as interference points in the interference area detection of the previous frame, thereby suppressing its target probability distribution in the current frame. G It can restrict M to a certain extent. T ,This method combines the two and can simultaneously improve the background suppression ability and interference resistance ability of the color histogram detection model.

[0070] S4.2, calculate the integral image of the target probability image, the expression is:

[0071]

[0072] Where M is the target probability image of the frame to be detected, (m,n) represents the row and column index of the M element, and M int is the integral image, (i, j) represents M int The row and column indices of the elements. The above formula represents the accumulation operation on the elements of M.

[0073] S4.3, calculate the color histogram response, the expression is:

[0074]

[0075] In the formula, Respectively represent M int The upper left, lower right, upper right, and lower left sampling areas of |T| represent the target area of ​​the previous frame.

[0076] S4.4, for M int In order to improve the calculation accuracy of the interference area G, the present invention improves the sampling method and uses sparse and dense mixed sampling to extract the final integral vector, which is expressed as:

[0077]

[0078] In the formula, score o 、score i Respectively represent M int The sparse sampling vector and the dense sampling vector of |T1| and |T2| represent the target area and the compressed target area respectively. score is the final sampling vector, and the number of its elements is consistent with the area of ​​the sampling area. Figure 1 Schematic diagram of mixed sampling.

[0079] The theoretical basis for this method to use mixed sampling is: int The corresponding original image is cropped from the current frame with the target position in the previous frame as the center, while the actual position of the slower moving target in the current frame is still near the center of the image, whereas the faster moving target may drift to the edge of the image. Therefore, this method combines sparse sampling and dense sampling, and can universally take both cases into account, thereby improving the accuracy of sampling calculation.

[0080] S4.5, using the peak descending iteration method, calculate the best area T and a set of interference areas G. The iteration condition formula is:

[0081] ms>sf×ms'

[0082] In the formula, ms' is the score peak value of the first iteration, ms is the score peak value after the current iteration is completed, and sf is the set iteration threshold, whose value range is (0,1). When the above iteration conditions are met, the target probability value of the corresponding area is set to 0, and the coordinate information of the area is saved, and then M is recalculated. int and score until the iteration condition is not met. Finally, all the region information is combined, where the region obtained in the first iteration is the optimal region T, and all subsequent regions are interference regions G.

[0083] S4.6, calculate the multi-feature adaptive fusion parameters, the expression is:

[0084]

[0085] Wherein, v is the adaptive fusion weight improved by the present invention. S are the target probability image and the background probability image obtained previously, respectively, T is the optimal region obtained in S4.5, and th is the overfitting weight threshold. As can be seen from the above formula, the present invention sets v as the probability mean in the optimal region T, and in order to prevent v from overfitting, when υ exceeds th, it is taken as the negative exponent of e.

[0086] S4.7, train the color histogram perception model for the next frame of image. According to T and G calculated in S4.5, train the background suppression model, interference area detection model, and background estimation model. The formulas are P(v∈T|T,S,C v ) * ,P(v∈T|T,G,C v ) * ,P(v∈S|C v ).

[0087] S5, calculates the fusion response and locates the target position, including:

[0088] S5.1, calculate the correlation filter response, the expression is:

[0089]

[0090] In the formula, α is the template filter, is a two-sample Gaussian response matrix, which is expressed as:

[0091]

[0092] In the formula, x is the template feature trained in the previous frame image, and z is the candidate region feature.

[0093] S5.2, calculate the fusion response, the expression is:

[0094] res=vυres co +(1-υ)·res cf

[0095] Where υ is the adaptive fusion parameter calculated in S4, res is the final fusion response, and res co is the color histogram response, res cf is the correlation filter response.

[0096] Figure 2 The figure is a schematic diagram of the fusion response, in which the correlation filter tracking and color histogram detection paths independently calculate their responses, and finally the final response is obtained after fusion through adaptive fusion weights, which is used as the basis for locating the target. Figure 2It can be seen that after fusing the correlation filter response and the color histogram response, the response of the background area can be suppressed while the response of the potential target area is enhanced.

[0097] S5.3, locate the target position, the expression is:

[0098]

[0099] Where (x,y) t is the final target coordinate of this frame, (x, y) t-1 is the target coordinate of the previous frame, (x max ,y max ) is the peak position coordinate of res, W and H are the width and height of res respectively. csf is the scale factor of the previous frame, which means the ratio of the target size of the previous frame to the target size of the first frame.

[0100] S6, perform adaptive scaling based on boundary correlation filtering (see Figure 3 shown), including:

[0101] S6.1, taking the calculated target position as the center, extract the four boundary image blocks of the target, namely the upper, lower, left and right, and calculate the multi-channel fusion features of each image block. The number of feature channels is 42, including 32-channel HOG features and 10-channel color features.

[0102] S6.2, the 42-channel features of each image block are vectorized and stacked, and the expression is:

[0103]

[0104] Where j = {l, r, t, b} represents the left, right, top, and bottom boundaries respectively, and z l is the 42-channel fusion feature of the boundary image to be detected, and the right side of the equal sign is the one-dimensional row vector stacking form of the fusion feature.

[0105] S6.3, perform boundary correlation filtering and calculate the response vector of the boundary image block, which is expressed as:

[0106]

[0107] In the formula, α j is the boundary template filter vector, is a two-sample Gaussian response vector, expressed as:

[0108]

[0109] In the formula, is a single feature vector of boundary template features.

[0110] S6.4, calculate the target frame offset and obtain the target scale size, the expression is:

[0111]

[0112] In the formula, Δj represents the one-dimensional coordinate correction, x j,max Represents the response vector res j The peak index of .

[0113] The theoretical basis for this method to use boundary correlation filtering to calculate the target scale size is that the existing adaptive scale calculation methods have the problems of limited scale calculation range, difficulty in adapting to changes in the target aspect ratio, and high calculation cost. The boundary correlation filtering used in this method only needs to calculate the four sub-image blocks of the upper, lower, left and right for each frame of the image to complete the calculation of the target scale change, which significantly reduces the amount of calculation and is not affected by changes in the target aspect ratio.

[0114] At this point, the position of the target to be detected is calculated in step S5, and the scale of the target is calculated in step S6. Next, the target is framed using these two pieces of information to achieve target tracking.

[0115] Example

[0116] The specific embodiments provided for the present invention are as follows:

[0117] Given a video sequence that needs to be tracked, before the target tracking begins, the target is calibrated and initialized in the first frame of the video sequence. First, the target is calibrated to obtain the target position p1 and target scale s1 of the first frame, that is, Figure 4 Then, p1 and s1 extract the complete image block containing the background frame and initialize Figure 4 The “background suppression model”, “interference detection model” and “background estimation model” marked by the dotted box are used for subsequent frame images, and their calculation formulas are P(v∈T|T,S,C v ) * ,P(v∈T|T,G,C v ) * ,P(v∈S|C v ). Finally, the relevant filter model is initialized for use in subsequent frame images, that is, Figure 4 The calculation formula of "template feature" and "template filter" in S3 is α, x j , α j , represent template filter, boundary template feature, and boundary template filter vector respectively.

[0118] Target tracking begins, and the tracking process is as follows Figure 4 As shown, the specific implementation steps are as follows:

[0119] Input the tth frame of the image to be detected and calculate Figure 4 The "color histogram response" and "adaptive fusion parameters" in the image are as follows. First, the target position p of the t-1th frame image is t-1 , target size s t-1 The search box image block is extracted as the benchmark, and the target probability image M and the background probability image M are calculated by the pre-trained color histogram probability perception model. S , from S4.2 int The integral image is calculated by the expression of S4.3 res co The color histogram response is calculated by using the expression of int Perform mixed sampling detection to obtain the interference area G and the optimal area T, and calculate the adaptive fusion parameter by the expression of υ in S4.6. Finally, train the color histogram perception model, which is composed of T, G and P(v∈T|T,S,C v ) * ,P(v∈T|T,G,C v ) * ,P(v∈S|C v ) expression training updates the probability distribution lookup table for use in subsequent frame images.

[0120] Input the tth frame of the image to be detected, calculate the fusion response and locate the target position. First calculate Figure 4 The "correlation filter response" in the image is: the target position p of the t-1 frame t-1 , target size s t-1 Extract the candidate region feature z as the benchmark, and compare z with the template feature x according to S5.1 The expression for calculating the Gaussian response matrix is With the template filter α according to S5.1 res cf The relevant filter response is calculated by the expression of res in S5.2. Figure 4 The peak position of res is located by the "fusion response" in S5.3, and finally the target position p of this frame image is obtained by the expression in S5.3 t , that is, Figure 4 Target position for new frame.

[0121] In the tth frame image, Figure 4 The process of "Adaptive Scaling Based on Boundary Correlation Filtering" is as follows: Figure 3 As shown. First, we have obtained p t Extract the four boundary image blocks of the target as the center and follow S6.2 jThe expression calculates the feature vector group of each boundary image block, and then performs boundary correlation filtering, which is obtained by res in S6.3 j The expression of S6.4 is used to calculate the four boundary response vectors, and finally the peak position of each response vector is located. The offset of each of the four boundary boxes of the target is calculated by the expression of S6.4 to obtain the target scale size s of this frame image. t ,Right now Figure 4 Target scale for new frame.

[0122] The obtained p t 、s t , perform frame calibration on the target in this frame image to achieve target tracking.

[0123] The tracking results of the two video sequences in this group of embodiments are as follows: Figure 5 As shown in the figure, (a) and (b) are the tracking results of this method and the same type of comparison method on two video sequences, respectively. Among them, this method is represented by MFFT (Multi Feature Fusion Tracker), and its target tracking box is a red solid line. It can be seen that in the video sequences of groups (a) and (b), the appearance characteristics of the target are very similar to the background, and both have phenomena such as target rotation and target scale change. In the early stage when the target motion amplitude is small, all methods can track accurately, but in the middle and late stages of the video, most of the other methods except this method have tracking drift, while this method always maintains relatively stable tracking.

[0124] Furthermore, the proposed method and the comparison method were experimented on the entire OTB dataset, and the obtained success rate and accuracy curves are shown in the figure below. Figure 6 As shown in the figure, (a) is the overlap rate curve and (b) is the accuracy curve. It can be seen that in the simulation experiment of the batch data set, the success rate and accuracy curves of this method are both located at a relatively high position. The tracking effect of each method is quantified by indicators, and the performance indicator table obtained is shown in Table 1.

[0125] Table 1: Performance indicators of each method on the OTB dataset

[0126] Method Name MFFT DSST KCF KCFDP SAMF Staple AUC 0.631 0.553 0.510 0.562 0.570 0.591 precision 0.896 0.779 0.753 0.784 0.804 0.834 FPS 37.85 54.23 203.23 54.94 34.06 28.32

[0127] In Table 1, the AUC (Area Under Curve) indicator represents Figure 6 The sum of the areas under the curve of the curve (a). The larger the value, the higher the tracking success rate of the method. The precision index indicates Figure 6The accuracy value of the curve (b) when the horizontal axis is 20, the larger the value, the higher the tracking accuracy of the method; FPS (Frames Per Second) represents the simulation frame rate of the method, the larger the value, the faster the calculation processing speed of the method. It can be seen that the AUC index and precision index of this method are ranked first, and the FPS index is ranked fourth, indicating that compared with the traditional method, this method can improve the tracking success rate and accuracy to a certain extent without losing the tracking speed.

[0128] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

Claims

1. A multi-feature fusion boundary correlation filtering target tracking method, characterized in that: The following steps are involved: S1, perform target calibration in the first frame of the video sequence; S2, initialize the color histogram probability perception model; when the first frame image is input, the background suppression model, interference area detection model, and background estimation model are initialized according to the calibrated target information; S3, initializing the correlation filter model; when the first frame image is input, the template feature, template filter, boundary template feature, and boundary template filter vector are initialized according to the calibrated target information; S4, calculating the color histogram response and the adaptive fusion parameters; for a frame of image containing a target to be detected, firstly extracting the search image block based on the target position and target scale of the previous frame of image, and calculating its target probability image, integral image, and color histogram response; then performing mixed sampling detection on the integral image to obtain the interference area and the optimal area, and then calculating the adaptive fusion parameters; finally, training the background suppression model, the interference area detection model, and the background estimation model for use in the next frame of image; S5, calculate the fusion response and locate the target position; for a frame of image containing the target to be detected, the target position and target scale of the previous frame image are used as the reference to extract the candidate region features, and the Gaussian response is calculated by the Gaussian kernel function, and then the relevant filter response is calculated and then the fusion response is calculated and its peak position is located to obtain the target position of the frame image to be detected; S6, calculating the target scale; for a frame of image containing the target to be detected, extracting a boundary image block centered on the detected target position, and then using boundary correlation filtering to calculate the target frame offset to obtain the target scale of the frame image to be detected.

2. The multi-feature fusion boundary correlation filtering target tracking method according to claim 1 is characterized in that: The background suppression model in step S2 and step S4 is: Where I represents an expanded image block containing the target area, T represents the target area, and S represents the background area outside T; P(v∈T|T,S,C v ) * Indicates the probability that a pixel v with a RGB combination value of C belongs to the target; They represent the number of pixels with RGB value combination C in the T area and the number of pixels with RGB value combination C in the S area in I respectively; ψ is the improved background suppression histogram, which is expressed as: Where ξ is the background suppression coefficient, which is used to control the degree of background suppression; |T| and |S| are the target area and background area, respectively.

3. The multi-feature fusion boundary correlation filtering target tracking method according to claim 1 is characterized in that: The interference area detection model in step S2 and step S4 is: Where G represents the potential interference area, P(v∈T|T,G,C v ) * Indicates the probability that a pixel v with a RGB combination value of C belongs to the target; It represents the number of pixels whose RGB values ​​in G are combined into C in the image block I; w is the enhancement weight.

4. The multi-feature fusion boundary correlation filtering target tracking method according to claim 1 is characterized in that: The background estimation model in step S2 and step S4 is used for the adaptive calculation of the subsequent multi-feature response fusion weight, and its expression is: In the formula, P(v∈S|C v ) represents the probability that the pixel v with RGB combination value C belongs to the background S; They respectively represent the number of pixels whose RGB value combination is C in the T area of ​​I and the number of pixels whose RGB value combination is C in the S area.

5. The multi-feature fusion boundary correlation filtering target tracking method according to claim 1 is characterized in that: Step S3 specifically includes: S3.1, initialize the template feature matrix x, which is extracted from the target information of the first frame image, and the image feature is the oriented gradient histogram HOG feature; S3.2, initialize the template filter α, which is expressed as: In the formula, F and F -1 Respectively represent Fourier transform and inverse transform, y is the standard output matrix, λ is the regularization parameter, k is the single sample Gaussian response matrix, and its expression is: Where σ is the Gaussian kernel response bandwidth, |||| represents the matrix norm, Represents element-wise dot product operation; S3.3, initialize the boundary template feature, take the known target position as the center, extract the target's left, right, top and bottom four boundary image blocks, and calculate the 32-channel HOG feature and 10-channel color feature, a total of 42 channels of fusion features, and vectorize and stack the fusion features. The expression is: In the formula, j = {l, r, t, b} represents the left, right, upper and lower boundaries respectively, and x j Represents the 42-channel fusion features of the boundary image. The right side of the equal sign is the one-dimensional vector stacking form of the fusion features, where the left and right boundary images are row vector stacking, the upper and lower boundary images are column vector stacking, and nl is the number of feature vectors; S3.4, initialize the boundary template filter vector, which is expressed as: In the formula, y j is the standard output vector of the boundary image, λ is the regularization parameter, k j is a single sample Gaussian response vector, and its expression is: In the formula, length() represents the length of the vector.

6. The multi-feature fusion boundary correlation filtering target tracking method according to claim 1 is characterized in that: Step S4 specifically includes: S4.1, for the frame image to be detected, calculate the target probability image of the candidate image block, which is calculated by the background suppression model and interference area detection model that have been trained for the previous frame image, and its expression is: M=τ·M T +(1-τ)·M G Where M is the final target probability image, M T is the target probability image calculated according to the background suppression model, M G is the target probability image calculated according to the interference area detection model. τ is the fusion weight; S4.2, calculate the integral image of the target probability image, which is expressed as: Where M is the target probability image of the frame to be detected, (m,n) represents the row and column index of the M element, and M int is the integral image, (i, j) represents M int The row and column indices of the element; S4.3, calculate the color histogram response, which is expressed as: In the formula, Respectively represent M int The upper left, lower right, upper right, and lower left sampling areas, |T| represents the target area of ​​the previous frame; S4.4, for M int In order to improve the calculation accuracy of the interference area G, the present invention improves the sampling method and uses sparse and dense mixed sampling to extract the final integral vector, which is expressed as follows: In the formula, score o 、score i Respectively represent M int The sparse sampling vector and dense sampling vector of , |T1| and |T2| represent the target area and compressed target area respectively, score is the final sampling vector, and the number of its elements is consistent with the area of ​​the sampling area. S4.5, using the peak descending iteration method, calculate the best area T and a set of interference areas G; the iteration condition formula is: ms>sf×ms' Where ms' is the score peak value of the first iteration, ms is the score peak value after the current iteration is completed, and sf is the set iteration threshold, whose value range is (0,1); when the above iteration conditions are met, the target probability value of the corresponding area is set to 0, and the coordinate information of the area is saved, and then M is recalculated int and score until the iteration condition is not met; finally, all the regional information is combined, where the region obtained in the first iteration is the optimal region T, and all subsequent regions are interference regions G; S4.6, calculate the multi-feature adaptive fusion parameters, the expression is: Where υ is the adaptive fusion weight, M, M S are the target probability image and background probability image obtained previously, T is the best region obtained in step S4.5, and th is the overfitting weight threshold; S4.7, train the color histogram perception model for the next frame of image; according to T and G calculated in step S4.5, train the background suppression model, interference area detection model, and background estimation model, whose formulas are P(v∈T|T,S,C v ) * ,P(v∈T|T,G,C v ) * ,P(v∈S|C v ).

7. The multi-feature fusion boundary correlation filtering target tracking method according to claim 1 is characterized in that: Step S5 specifically includes: S5.1, calculate the correlation filter response, the expression is: In the formula, α is the template filter, is a two-sample Gaussian response matrix, which is expressed as: In the formula, x is the template feature trained in the previous frame image, and z is the candidate region feature; S5.2, calculate the fusion response, the expression is: res=υ·res co +(1-υ)·res cf Where υ is the adaptive fusion parameter calculated in S4, res is the final fusion response, and res co is the color histogram response, res cf is the correlation filter response; S5.3, locate the target position, the expression is: Where (x,y) t is the final target coordinate of this frame, (x, y) t-1 is the target coordinate of the previous frame, (x max ,y max ) is the peak position coordinate of res, W and H are the width and height of res respectively, and csf is the scale factor of the previous frame, which means the ratio of the target size of the previous frame to the target size of the first frame.

8. The multi-feature fusion boundary correlation filtering target tracking method according to claim 1 is characterized in that: Step S6 specifically includes: S6.1, taking the calculated target position as the center, extracting a total of four boundary image blocks, namely, the upper, lower, left, and right boundary blocks of the target, and calculating the multi-channel fusion features of each image block; wherein the number of feature channels is 42, including 32-channel HOG features and 10-channel color features; S6.2, the 42-channel features of each image block are vectorized and stacked, and the expression is: Where j = {l, r, t, b} represents the left, right, top, and bottom boundaries respectively; z l is the 42-channel fusion feature of the boundary image to be detected, and the right side of the equal sign is the one-dimensional row vector stacking form of the fusion feature; S6.3, perform boundary correlation filtering and calculate the response vector of the boundary image block, which is expressed as: In the formula, α j is the boundary template filter vector, is a two-sample Gaussian response vector, expressed as: In the formula, is a single feature vector of the boundary template feature; S6.4, calculate the target frame offset and obtain the target scale size, the expression is: In the formula, Δj represents the one-dimensional coordinate correction, x j,max Represents the response vector res j The peak index of .

9. The multi-feature fusion boundary correlation filtering target tracking method according to any one of claims 1 to 8, characterized in that: The background suppression model, interference region detection model, and background estimation model in steps S2 and S4 are not only used in the initialization stage of the first frame image, but each subsequent frame image is trained by these three models to obtain three color probability distribution lookup tables for use in the next frame image. The training formulas are P(v∈T|T,S,C v ) * ,P(v∈T|T,G,C v ) * ,P(v∈S|C v ).

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a multi-feature fusion boundary correlation filtering target tracking method as described in any one of claims 1-9 are implemented.

Images