A tracking method based on KCF target block and confidence template update mechanism

Abstract

The application relates to a tracking method based on a KCF target block and a confidence template updating mechanism and belongs to the technical field of target tracking. Firstly, the idea of block is introduced, and the final position of a target is determined by synchronously tracking two sub-blocks. Secondly, the original frame-by-frame updating template updating strategy of KCF is adjusted, the average correlation peak difference ratio (APCDR) of the two sub-blocks is respectively calculated, comparison is made with a given threshold, and only when the threshold is greater than the threshold, it is considered that no special situation occurs in the tracking process, and the sub-template is updated. The KCF algorithm is optimized by introducing the block idea and the confidence template updating strategy.

Description

TECHNICAL FIELD

[0001] The application relates to a tracking method based on KCF target block and confidence template updating mechanism and belongs to the technical field of target tracking. BACKGROUND

[0002] Video target tracking is one of the research hotspots in the field of computer vision. With the rapid improvement of computer processing capacity, video-based target tracking technology has developed rapidly, providing important support for intelligent monitoring, auxiliary driving, human-computer interaction and other applications. In recent years, many algorithms with excellent performance and speed have emerged in the field of target tracking technology. Among them, the related filtering algorithm is a relatively advanced algorithm and has attracted widespread attention and research. In the related filtering algorithm, although the DSST algorithm adds a scale transformation, it uses a single feature, introduces more noise, and leads to unstable tracking. The KCF algorithm uses a frame-by-frame model updating strategy. Therefore, when the target is blocked, tracking drift occurs. If the target is blocked for too long, tracking failure occurs. The SRDCF algorithm has good robustness but slow speed, which cannot meet the real-time requirement. The C-COT algorithm uses a neural network to extract features, greatly increasing the computational complexity and slowing down the speed. At present, most of the mainstream target tracking algorithms are based on deep learning algorithms, which have high requirements for computing resources and are difficult to achieve high-frame-rate tracking effects.

[0003] In view of the problems that the above algorithms cannot track after being blocked and cannot meet the real-time requirement, the number of target frames and the template confidence are improved in the application, the original target frame is divided into two blocks, and a judgment method, average correlation peak difference ratio, is proposed to judge whether template tracking is needed according to the blocking, so that the tracking can be faster and more stable.

[0004] In the target tracking algorithm, the traditional method is to use a single tracker to track and locate the entire target. The defect of this method is that the robustness to changes in the appearance of the target is poor. For example, when the target changes in shape, is blocked, rotates and the like, it is obvious that using a single target appearance to represent is difficult to cope with these changes. The probability that the appearance of the target does not change is extremely small, and in most cases, the local features of the target object remain unchanged, but the shape of the target changes constantly, such as a person crossing the road. For some target objects whose overall appearance changes dramatically, the local features remain unchanged, and in the tracking process, local blocking inevitably occurs. When the target is blocked, tracking drift occurs. If the target is blocked for too long, tracking failure occurs. Therefore, when the target is blocked, the template is contaminated, which may lead to the loss of the target. SUMMARY

[0005] The technical problem solved by the present application is that, aiming at the problems of existing occlusion and real-time deficiency, a tracking method based on KCF target block and confidence template updating mechanism is proposed based on a kernel correlation filter tracking framework.

[0006] The technical solution of the present application is: a tracking method based on KCF target block and confidence template updating mechanism, based on a kernel correlation filter tracking framework, using a histogram of oriented gradients (HOG) for feature extraction, block processing the target by using two sub-blocks of the same size as filter templates, and simultaneously calculating the APCDR values of the two sub-blocks, only when the APCDR value is greater than a given threshold, it is considered that the sub-block target tracked at this time is reliable, and template updating is performed. Compared with the correlation filter tracking method using a single target frame, more stable and fast tracking effect is achieved.

[0007] Specifically, the present application comprises the following steps:

[0008] Step 1: input the first frame;

[0009] A video includes a plurality of frames, each frame containing at least one target, wherein the target position on the first frame is known, and the target position on each remaining frame is unknown; the number of video frames is a positive integer greater than or equal to 1; the upper left corner of each frame in the video frame sequence is the coordinate origin (1, 1), and the width and height are Width and Height, respectively; a rectangular region (x0, y0, w0, h0) of the target to be tracked in the first frame is manually or automatically selected, i.e. the selected tracking target. Wherein (x0, y0) represents the upper left corner coordinates of the rectangular region, and w0, h0 represent the width and height of the rectangular region, respectively. The selected target of the first frame is also called the current frame tracking result (x1, y1, w1, h1) = (x0, y0, w0, h0), and the subscript represents the current frame number.

[0010] Step 2: initialize the target template;

[0011] Step 2.1: calculate the search window;

[0012] According to the rectangular region corresponding to the tracking result (x t-1 ,y t-1 ,w t-1 ,h t-1 ) of the last frame, i.e. the t-1 frame, the search window of the candidate target of the current frame, i.e. the t frame, can be calculated, in particular, the search window of the first frame is calculated according to (x0, y0, w0, h0). The center point of the search window is (x_s ty_s t , where x_s t = x t-1 + w t-1 / 2, y_s t = y t-1 + h t-1 / 2, width and height are w_s t = 1.5 x w t-1 + 0.5 x h t-1 , h_s t = 1.5 x h t-1 + 0.5 x w t-1 , respectively. To ensure the search range is within the video frame, the width and height of the search window are further modified according to the intersection of the search range and the current frame region. The distance between the boundary of the search window and the real target boundary is limited to be even, and the width and height of the search window are further modified.

[0013] The search window image can be normalized to form a standard search window according to the search window transformation factor, with width and height of w_sn t = w_s t x γ, h_sn t = h_s t x γ, and the width and height of the standard target window of the current frame are w_on t = w_sn t x 0.75 - h_sn t x 0.25, h_on t = h_sn t x 0.75 - w_sn t x 0.25.

[0014] After loading the video sequence, the center of the target is first determined, and then the target is divided into two equal-sized upper and lower blocks to determine the center points of the two frames, posl = (x_s t , y_s t + h0 / 4) and pos2 = (x_s t , y_s t - h0 / 4). The two sub-blocks are used as two templates for model updating.

[0015] Step 2.2: generate a standard Gaussian response map;

[0016] The standard Gaussian response map g is a two-dimensional matrix with width and height of w_g = w_sn t / cell, h_g = h_sn t / cell, and the matrix element value is the probability density function conforming to the two-dimensional Gaussian distribution N(0, 0, δ, δ, 0), which can be calculated according to The formula is calculated. Wherein, δ represents the standard deviation of two-dimensional Gaussian distribution, the calculation method is cell represents the size of each cell in the HOG feature extraction process, (i, j) represents the element coordinate position of the Gaussian response graph matrix, and the origin is located at the center point of the matrix. The frequency domain representation G of the standard Gaussian response graph can be obtained by Fourier transform.

[0017] Step 2.3: Extract the histogram of oriented gradient (HOG) feature;

[0018] Take cell as the HOG feature cell size parameter, 2x2 cells as the block size, and set the histogram group distance bin to 2π / 7. Extract the HOG feature f t in the current frame normalized search window, and the size is w_gxh_gx28. Smooth the feature f t using a cosine window with a size of w_gxh_g, and then perform Fourier transform to obtain the frequency domain representation F t of the HOG feature, which has the same size as f t .

[0019] Step 2.4: Calculate the correlation filter template of the HOG feature

[0020] Given the frequency domain representation F t of the normalized search window HOG feature and the frequency domain representation G of the standard Gaussian response graph, the frequency domain representation H t of the HOG feature correlation filter template can be calculated according to the formula H t =G / F t .

[0021] Step 3: Input the next frame and extract the feature;

[0022] According to the method of step 2.1, calculate the current frame search window Search(t), and according to step 2.3, extract the frequency domain representation F t of the current frame histogram of oriented gradient (HOG) feature. The response graph G t of the current frame HOG feature can be calculated according to the formula G t =F t ⊙H t-1 .

[0023] Step 4: Loop matrix and ridge regression classifier;

[0024] Using the kernel method based on Gaussian kernel function, calculate the kernel correlation matrix and classifier regression coefficient of all target samples and all training samples;

[0025] The circulant matrix of the mxn matrix X is obtained by regarding it as a block matrix, and the elements of each row of the matrix are obtained by shifting the previous element downward, and each column is obtained by shifting to the right. Therefore, the size of the circulant matrix is mn x mn, which is mainly to change the circulant matrix X into a square matrix, so that the eigenvalues and eigenvectors can be calculated. After the circulant matrix X is established, if it is determined that the block matrix at the (i, j) position has the maximum response, that is, the target frame is offset by i-1 units downward and j-1 units rightward relative to the previous target frame. Then the size of the label y is n x n. After combining it with the ridge regression algorithm, the following can be obtained:

[0026]

[0027] wherein, represents the conjugate form of x after DFT transformation, δ represents an all-1 vector, which is equivalent to the eigenvector of the unit matrix, and ⊙ represents matrix element point multiplication.

[0028] According to the property of DFT time domain convolution:

[0029]

[0030] And the time domain convolution is usually a circular convolution, that is, the original sequence is regarded as a period, and through verification, the following can be obtained:

[0031]

[0032] It can be found that Therefore, the following can be obtained:

[0033]

[0034] Using the above conclusion, w can be further converted as follows:

[0035]

[0036] At this point, the solution of the weight matrix w in the Fourier space becomes a simple point multiplication operation, and the operation complexity is greatly reduced.

[0037] Step 5: Positioning the target;

[0038] The function response value of all positions is calculated using the kernel correlation matrix and the classifier regression coefficient, the maximum response position is detected, and then the target position of the current frame is calculated. The result of the GL matrix element value represents the probability that the corresponding candidate target in the search window is the tracking result, and the candidate target corresponding to the maximum element value is the tracking result.

[0039] The number of candidate targets in the search window of the current frame is (w_sn t -w_on t ) x (h_snt -h_on t ). Let GL max , x_GL max and y_GL max denote the maximum element value and its corresponding horizontal and vertical coordinate position in the adaptive feature fusion result GL matrix, respectively. Then the current frame tracking result is (x t , y t , w t , h t ), where w t = w t-1 , h t = h t-1 , x t = x t-1 + (x_GL max - (w_sn t - w_on t ) / 2) / γ - w t / 2, y t = y t-1 + (y_GL max - (h_sn t - h_on t ) / 2) / γ - h t / 2, and γ is the transformation factor of the search window.

[0040] Step 6: Template update;

[0041] According to the position of the current frame tracking result (x t , y t , w t , h t ) and the method of Step 2.1, the search window Search'(t) is calculated. According to the method of Step 2.3, the frequency domain representation F t ' of the direction gradient histogram (HOG) feature in the range of Search'(t) is extracted. According to the method of Step 2.4, H t ' = G / F t ' is calculated. Let η be the update parameter. The update method of the current frame HOG feature correlation filter template H t is shown in the formula:

[0042] H t = (1-η)H t-1 +ηH t '

[0043] According to the position of the current frame tracking result (x t , y t , w t , h tWhen the target is accurately tracked, the resulting KCF response map will show a single, distinct peak. Mathematically, the overall distribution resembles a two-dimensional Gaussian distribution. Conversely, if the target encounters complex situations during tracking, the oscillations and peak characteristics of the response map reflect the confidence level of the tracking results. After accurately locating the target, the response map shows only a single peak with a smooth surrounding area. If the target is not accurately tracked, the response map exhibits violent oscillations, and its overall distribution differs significantly from the ideal Gaussian distribution. Based on this, a judgment method—Average Peak Difference (APK)—is proposed.

[0044] Correlation Difference Ratio, APCDR) is:

[0045]

[0046] in the formula This refers to the maximum response value. Refers to the minimum response value. The value of the response graph in coordinates (x, y).

[0047] The APCDR value is used to characterize the response map. Therefore, the APCDR value of two sub-blocks is calculated. Based on the comparison between the APCDR value and a threshold, if the value is less than or equal to the threshold, it is considered that a complex situation has occurred, and the template is not updated; otherwise, if the value is greater than the threshold, it is considered that no complex situation has occurred, and the template is updated.

[0048] Step 7: If the current frame is the last frame, the tracking ends; otherwise, proceed to Step 3.

[0049] The beneficial effects of this invention are as follows: This invention optimizes the KCF algorithm by introducing the concept of block partitioning and the confidence template update strategy, mainly in the following aspects:

[0050] 1. Addressing issues of occlusion and image blurring: By dividing the overall target into blocks, using two identical sub-blocks as filter templates, and simultaneously calculating the APCDR values ​​of both sub-blocks, the tracked sub-block target is considered reliable only when the APCDR value exceeds a given threshold, and template updates are then performed. This significantly enhances the tracking capability of the KCF algorithm and reduces the possibility of tracking drift.

[0051] 2. Addressing issues of occlusion and rapid movement: By using block-based tracking, the KCF algorithm's target search and rapid localization capabilities are significantly enhanced.

[0052] 3. Addressing the issue of similar backgrounds: After segmentation, the templates of the two sub-blocks are updated independently, which significantly improves the KCF algorithm's ability to distinguish between targets and backgrounds. Attached Figure Description

[0053] Figure 1 This is a diagram showing the selection of the tracking region in the first frame of this invention.

[0054] Figure 2 Yes Figure 1 The image after the selected target is divided into blocks;

[0055] Figure 3 This is a snapshot of the tracking results in an embodiment of the present invention;

[0056] Figure 4 This is a diagram showing the selection of the tracking region in the first frame of this invention.

[0057] Figure 5 Yes Figure 4 The image after the selected target is divided into blocks;

[0058] Figure 6 This is a diagram showing the tracking effect before occlusion in an embodiment of the present invention;

[0059] Figure 7 This is a diagram showing the tracking effect under improved occlusion in an embodiment of the present invention;

[0060] Figure 8 This is a diagram showing the improved tracking effect immediately after leaving the obstruction in an embodiment of the present invention.

[0061] Figure 9 This is a diagram showing the improved tracking effect after the device leaves the obstruction for a period of time in an embodiment of the present invention.

[0062] Figure 10 This is a flowchart of the steps of the present invention. Detailed Implementation

[0063] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0064] Example 1: According to the technical solution of the present invention, a Soccer video sequence is selected for tracking, which has eight challenging attributes: illumination change, scale change, occlusion, motion blur, fast motion, in-plane rotation, out-of-plane rotation, and background clutter.

[0065] like Figure 10 The specific steps are as follows:

[0066] Step 1: Input the first frame;

[0067] Soccer video is selected, where the width and height of the video frame are Width = 640, Height = 360. The rectangular region (x0, y0, w0, h0) of the first frame is selected as (302, 135, 67, 81), i.e. the selected tracking target is shown in the green rectangular frame. Wherein, (x0, y0) is the upper left corner coordinate of the rectangular region, w0, h0 is the width and height of the rectangular region. Figure 1

[0068] Step 2: initialize the target template;

[0069] Step 2.1: calculate the search window;

[0070] According to the tracking result (x t-1 , y t-1 , w t-1 , h t-1 ) of the last frame, i.e. the t-1 frame, the search window Search(t) of the candidate target of the current frame, i.e. the t frame, can be calculated. In particular, the search window of the first frame is calculated according to (x0, y0, w0, h0) : (302, 135, 67, 81). Taking the first frame as an example, the center point of the search window is (x_s t , y_s t ) : (336, 176), x_s t = x t-1 + w t-1 / 2 = 336, y_s t = y t-1 + h t-1 / 2 = 176, the width and height are w_s t = 1.5 × w t-1 + 0.5 × h t-1 = 141, h_s t = 1.5 × h t-1 + 0.5 × w t-1 = 155. In order to ensure that the search range is within the video frame range, the width and height of the search window are further corrected according to the intersection of the search range and the current frame region. The distance between the boundary of the search window and the boundary of the real target is limited to an even number, and the width and height of the search window are further corrected.

[0071] According to the search window transformation factor, the search window image can be standardized and transformed to form a standard search window, and the width and height of the standard search window are w_sn t = w_s t × γ = 141, h_sn t = h_s t × γ = 155, and the width and height of the standard target window of the current frame are w_on t = w_sn​t x 0.75 - h sn t x 0.25 = 67, h on t = h sn t x 0.75 - w sn t x 0.25 = 81.

[0072] After loading the video sequence, the center of the target is determined first, and then the target is divided into two blocks of equal size to determine the center points of the two frames, pos1 = (x_s t , y_s t + h0 / 4):(335, 195) and pos2 = (x_s t , y_s t - h0 / 4):(335, 154). The two sub-blocks are used as two templates for model updating. Figure 2 The figure after the target is divided into two sub-blocks, both represented by yellow frames. The large green frame is the overall target frame.

[0073] Step 2.2: Generate standard Gaussian response map;

[0074] The standard Gaussian response map g is a two-dimensional matrix, where cell = 4, its width and height are w_g = 41, h_g = 25, and its matrix element value is the probability density function of the two-dimensional Gaussian distribution N(0, 0, δ, δ, 0), which can be calculated according to The formula. Where δ represents the standard deviation of the two-dimensional Gaussian distribution, and the calculation method is cell represents the size of each cell in the HOG feature extraction process, (i, j) represents the element coordinate position of the Gaussian response map matrix, and the origin is at the center point of the matrix. The Fourier transform of the standard Gaussian response map can obtain its frequency domain representation G, which has the same size as g.

[0075] Step 2.3: Extract the Histogram of Oriented Gradient (HOG) feature;

[0076] Take cell as the HOG feature cell size parameter, 2x2 cells as the block size, and set the histogram group distance bin to 2π / 7. Extract the HOG feature f1 within the current frame standardized search window, which has a size of w_g x h_g x 28: (41 x 25 x 28). Smooth the feature f t with a cosine window of size w_g x h_g: (41 x 25), and then perform Fourier transform (Ft) to obtain the frequency domain representation F1 of the HOG feature, which has the same size as f1.

[0077] Step 2.4: Calculate the correlation filter template of the HOG feature

[0078] Given the frequency domain representation F1 of the HOG feature of the search window and the frequency domain representation G of the standard Gaussian response map, the frequency domain representation H1 of the HOG feature correlation filter template can be calculated according to the formula H1 = G / F1.

[0079] Step3: input the next frame and extract the feature;

[0080] The search window Search(2) of the current frame is calculated according to the method of Step2.1, and the frequency domain representation F2 of the direction gradient histogram (HOG) feature of the current frame is extracted according to Step2.3. The response map G2 of the HOG feature of the current frame can be calculated according to the formula G2 = F2⊙H1.

[0081] Step4: loop matrix and ridge regression classifier;

[0082] Using the kernel method based on the Gaussian kernel function, the kernel correlation matrix and the classifier regression coefficient of all target samples and all training samples are calculated;

[0083] The m×n matrix X is regarded as a block matrix, and the elements of each row of the matrix are obtained by shifting the previous element downward, and each column is obtained by shifting to the right. Therefore, the size of the circulant matrix is mn×mn, which is mainly to change the circulant matrix X into a square matrix, so that the eigenvalues and eigenvectors can be calculated. After the circulant matrix X is established, if it is determined that the response of the (i,j) block matrix is maximum, that is, the target frame is shifted downward by i-1 units and rightward by j-1 units relative to the previous target frame. Then the size of the label y is n×n. After combining it with the ridge regression algorithm, the following can be obtained:

[0084]

[0085] wherein, represents the conjugate form of x after DFT transformation, δ represents an all-1 vector, which is equivalent to the eigenvector of the unit matrix, and ⊙ represents the element-wise multiplication of the matrix.

[0086] According to the property of DFT time domain convolution:

[0087]

[0088] And the time domain convolution is usually circular convolution, that is, the original sequence is regarded as a period, and through verification, the following can be obtained:

[0089]

[0090] It can be found that Therefore, we can get:

[0091]

[0092] Using the above conclusion can continue to convert w for:

[0093]

[0094] So far, the solution of the weight matrix w in the Fourier space into a simple point multiplication operation, the operation complexity is greatly reduced.

[0095] Step5: positioning target;

[0096] Using the kernel correlation matrix and classifier regression coefficient calculation of all position function response value, the maximum response position is detected, and the target position of the current frame is calculated; the result of the GL matrix element value represents the probability of the corresponding search window candidate target being the tracking result, and the maximum element value corresponds to the candidate target, which is the tracking result;

[0097] The number of candidate targets in the search window of the current frame is (w_sn t -w_on t )×(h_sn t -h_on t )=75×75. Let GL max , x_GL max =42 and y_GL max =38 represent the maximum element value and its corresponding horizontal and vertical coordinate positions in the adaptive feature fusion result GL matrix, respectively, and the tracking result of the current frame is (x2, y2, w2, h2): (298, 137, 67, 81), wherein x2=298, y2=137, w2=67, h2=81, which is the transformation factor of the search window.

[0098] Step6: update target template;

[0099] According to the position of the tracking result (x2, y2, w2, h2) of the current frame: (298, 137, 67, 81) and the method of step 2.1, the search window Search'(2) is calculated, according to the method of step 2.3, the frequency domain representation F2' of the direction gradient histogram (HOG) feature in the range of Search'(2) is extracted, and according to the method of step 2.4, H2'=G / F2' is calculated. Let η=0.01 be the update parameter, and the update method of the current frame HOG feature correlation filter template H2 is shown in the formula:

[0100] H2=(1-0.01)×H1+0.01×H2'

[0101] According to the tracking result (x t ,y t ,w t ,h tWhen the target is accurately tracked, the obtained KCF response map presents a unique and obvious peak value, and the overall distribution is similar to a two-dimensional Gaussian distribution from a mathematical point of view. On the contrary, if the target encounters a complex situation in the tracking process, the response map is shaken and the peak value reflects the confidence of the tracking result. After accurate positioning of the target, the response map only has a sharp peak and the surrounding area is smooth. If the target cannot be accurately tracked, the response map is severely shaken, and the overall distribution is obviously different from the Gaussian distribution in the ideal case. On this basis, a judgment method, that is, an average peak correlation difference ratio (APCDR) is proposed, that is:

[0102]

[0103] In the formula, Max refers to the maximum response value, Min refers to the minimum response value, and R(x, y) refers to the value of the response map at the coordinates (x, y). The APCDR value is used to represent the situation of the response map. Therefore, the APCDR values of two sub-blocks are calculated, and according to the comparison result of the APCDR value and the threshold value, when the APCDR value is less than or equal to the threshold value, it is judged that a complex situation occurs, and the template is not updated; and when the APCDR value is greater than the threshold value, it is considered that a complex situation does not occur, and the template is updated. For the soccer video, a large number of tests show that when the threshold value is selected as 15, the tracking result is better.

[0104] Step 7: If the current frame is the last frame, the tracking ends; otherwise, go to Step 3.

[0105] Finally, the hardware experimental environment of the embodiment of the application is a computer configured with an Intel Core i7-10510U CPU, a main frequency of 2.3 GHz and a memory of 16 GB, and the success rate of the final tracking result reaches 81.9%, which is higher than 79.3% of the KCF. Partial tracking result screenshots are as shown in

[0106] Figure 3 , wherein the green box represents the tracking result of the KCF algorithm, and the two yellow boxes represent the tracking result of the application. It can be seen that the block processing has a good effect on the rapid movement of the target. Figure 3 Embodiment 2: According to the technical solution of the application, the Jogging video sequence is selected for tracking, which has eight challenge attributes of illumination change, scale change, occlusion, motion blur, rapid movement and background clutter.

[0107] Step 1: input the first frame;

[0108]

[0109] ​​Select the Jogging video, where the video frame width and height are Width = 352, Height = 288. Select the rectangular region (x0, y0, w0, h0) in the first frame: (180, 79, 37, 114), that is, the selected tracking target is shown in the green rectangular frame. Figure 4 (x0, y0) is the upper left corner coordinate of the rectangular region, and w0, h0: (37, 114) is the width and height of the rectangular region.

[0110] Step 2: Target frame division;

[0111] Step 2.1: Calculate the search window;

[0112] According to the tracking result (x t-1 ,y t-1 ,w t-1 ,h t-1 ) of the previous frame, i.e. t-1 frame, the search window Search(t) of the candidate target of the current frame, i.e. t frame, can be calculated. In particular, the search window of the first frame is calculated according to (x0, y0, w0, h0): (180, 79, 37, 114). Taking the first frame as an example, the center point of the search window is (x_s t ,y_s t ): (198, 136), x_s t =x t-1 +w t-1 / 2=198, y_s t =y t-1 +h t-1 / 2=136, the width and height are w_s t =1.5×w t-1 +0.5×h t-1 =170, h_s t =1.5×h t-1 +0.5×w t-1 =189. In order to ensure that the search range is within the video frame range, the width and height of the search window are further corrected according to the intersection of the search range and the current frame region. The distance between the boundary of the search window and the boundary of the real target is limited to an even number, and the width and height of the search window are further corrected.

[0113] According to the search window transformation factor, the search window image can be standardized and transformed to form a standard search window, and the width and height of the standard search window are w_sn t =w_s t ×γ=170, h_sn t =h_s t ×γ=189, and the width and height of the standard target window of the current frame are w_on t =w_snt x 0.75 - h sn t x 0.25 = 80, h on t = h sn t x 0.75 - w sn t x 0.25 = 99.

[0114] After loading the video sequence, the center of the target is determined first, and then the target is divided into two blocks of equal size to determine the center points of the two frames, pos1 = (x_s t , y_s t + h0 / 4): (198, 164) and pos2 = (x_s t , y_s t - h0 / 4): (198, 107). The two sub-blocks are used as two templates for model updating. Figure 5 The figure after the target is divided into two sub-blocks, both represented by yellow frames. The large blue frame is the overall target frame.

[0115] Step 2.2: Generate standard Gaussian response map;

[0116] The standard Gaussian response map g is a two-dimensional matrix, where cell = 4, its width and height are w_g = 23 and h_g = 35, and the matrix element value is the probability density function of the two-dimensional Gaussian distribution N(0, 0, δ, δ, 0), which can be calculated according to The formula. Where δ represents the standard deviation of the two-dimensional Gaussian distribution, and the calculation method is cell represents the size of each cell in the HOG feature extraction process, (i, j) represents the element coordinate position of the Gaussian response map matrix, and the origin is at the center point of the matrix. The Fourier transform of the standard Gaussian response map can obtain its frequency domain representation G, which has the same size as g.

[0117] Step 2.3: Extract the Histogram of Oriented Gradient (HOG) feature;

[0118] Take cell as the HOG feature cell size parameter, 2x2 cells as the block size, and set the histogram group distance bin to 2π / 7. Extract the HOG feature f1 within the current frame standardized search window, which has a size of w_g x h_g x 28: (23 x 35 x 28). Smooth the feature f t with a cosine window of size w_g x h_g: (23 x 35), and then perform Fourier transform (Ft) to obtain the frequency domain representation F1 of the HOG feature, which has the same size as f1.

[0119] Step 2.4: Calculate the correlation filter template of the HOG feature;

[0120] Given the frequency domain representation F1 of the HOG feature of the search window and the frequency domain representation G of the standard Gaussian response map, the frequency domain representation H1 of the HOG feature correlation filter template can be calculated according to the formula H1 = G / F1.

[0121] Step3: input the next frame and extract the feature;

[0122] The frequency domain representation F2 of the HOG feature of the current frame is extracted according to Step2.3, and the response map G2 of the HOG feature of the current frame can be calculated according to the formula G2 = F2⊙H1.

[0123] Step4: loop matrix and ridge regression classifier;

[0124] Using the kernel method based on the Gaussian kernel function, the kernel correlation matrix and the classifier regression coefficient of all target samples and all training samples are calculated;

[0125] The m×n matrix X is regarded as a block matrix, and the elements of each row of the matrix are obtained by shifting the previous element downward, and each column is obtained by shifting to the right. Therefore, the size of the circulant matrix is mn×mn, which is mainly to change the circulant matrix X into a square matrix, so that the eigenvalues and eigenvectors can be calculated. After the circulant matrix X is established, if it is determined that the response of the (i, j) block matrix is maximum, that is, the target frame is shifted downward by i-1 units and rightward by j-1 units relative to the previous target frame. Then the size of the label y is n×n. After combining with the ridge regression algorithm, the following can be obtained:

[0126]

[0127]

[0128] wherein, represents the conjugate form of x after DFT transformation, δ represents an all-1 vector, which is equivalent to the eigenvector of the unit matrix, and ⊙ represents the element-wise multiplication of the matrix.

[0129] According to the property of DFT time domain convolution:

[0130]

[0131] And the time domain convolution is usually circular convolution, that is, the original sequence is regarded as a period, and through verification, the following can be obtained:

[0132]

[0133] It can be found that Therefore, the following can be obtained:

[0134]

[0135] Using the above conclusions can continue to convert w for:

[0136]

[0137] So far, the solution of the weight matrix w in the Fourier space into a simple point multiplication operation, the operation complexity is greatly reduced.

[0138] Step5: positioning target;

[0139] Using the kernel correlation matrix and classifier regression coefficient calculation of all position function response value, the maximum response position is detected, and the target position of the current frame is calculated; the result of the GL matrix element value represents the probability of the corresponding search window candidate target is the tracking result, and the maximum element value corresponds to the candidate target is the tracking result;

[0140] The number of candidate targets in the search window of the current frame is (w_sn t -w_on t )×(h_sn t -h_on t )=75×75. Let GL max , x_GL max =42 and y_GL max =38 represent the maximum element value and its corresponding horizontal and vertical coordinate position in the adaptive feature fusion result GL matrix, then the tracking result of the current frame is (x2,y2,w2,h2):(180,83,37,114), wherein x2=180, y2=83, w2=37, h2=114, which is the transformation factor of the search window.

[0141] Step6: update target template;

[0142] According to the position of the current frame tracking result (x2,y2,w2,h2):(180,83,37,114) and the method of Step2.1, the search window Search'(2) is calculated, according to the method of Step2.3, the frequency domain representation F2' of the direction gradient histogram (HOG) feature in the range of Search'(2) is extracted, and according to the method of Step2.4, H2'=G / F2' is calculated. Let η=0.01 be the update parameter, and the current frame HOG feature correlation filter template H2 update method is shown in the formula:

[0143] H2=(1-0.01)×H1+0.01×H2'

[0144] According to the current frame tracking result (x t ,yt ,w t ,h t ) of the target is tracked accurately, the resulting KCF response map presents a unique and obvious peak value, and from a mathematical point of view, the overall distribution is similar to a two-dimensional Gaussian distribution. On the contrary, if the target encounters complex situations during tracking, the response map will be shaken and the peak value feature reflects the confidence of the tracking result. After accurate positioning of the target, the response map only presents a sharp peak, and the surrounding area is smooth. If the target cannot be accurately tracked, the response map will be shaken, and the overall distribution is obviously different from the Gaussian distribution in the ideal case. On this basis, a judgment method is proposed, that is, the average peak correlation difference ratio (APCDR) is:

[0145]

[0146] In the formula, Max refers to the maximum response value, Min refers to the minimum response value, and R(x, y) refers to the value of the response map at the coordinates (x, y).

[0147] The APCDR value is used to represent the situation of the response map. Therefore, the APCDR values of the two sub-blocks are calculated, and according to the comparison result of the APCDR value and the threshold value, when the APCDR value is less than or equal to the threshold value, it is judged that the complex situation occurs, and the template is not updated; and when the APCDR value is greater than the threshold value, it is considered that the complex situation does not occur, and the template is updated. For the Jogging video, after a large number of tests, it is found that when the threshold value is 25, the tracking result is better

[0148] Step 7: If the current frame is the last frame, the tracking ends; otherwise, go to Step 7.

[0149] Finally, the hardware experimental environment of the embodiment of the application is a computer configured with an Intel Core i7-10510U CPU, a main frequency of 2.3 GHz and a memory of 16 GB, and the success rate of the final tracking result reaches 88.5%, which is higher than the 74.8% of the KCF. Partial tracking result screenshots are shown in Fig. 3. Figures 6-9 In the figure, the green box represents the tracking result of the KCF algorithm, and the two yellow boxes represent the tracking result of the application. It can be seen that the blocking has a good processing effect on the target occlusion.

[0150] The specific embodiments of the application are described in detail above with reference to the accompanying drawings, but the application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the application.​​​

Claims

1.A tracking method based on KCF target block and confidence template update mechanism, characterized in that: Step1: input the first frame; Step2: initialize the target template; Step3: input the next frame and extract the features; Step4: loop matrix and ridge regression classifier; Step5: locate the target; Step6: template update; Step7: if the current frame is the last frame, the tracking ends, otherwise, go to Step3; the Step2 is specifically: Step2.1: calculate the search window; modify the width and height of the search window according to the intersection of the search range and the current frame region, limit the distance between the boundary of the search window and the boundary of the real target to be even, and further modify the width and height of the search window; after loading the video sequence, first determine the center of the target, and then divide the target into two blocks of equal size to determine the center points of the two boxes, namely: Then the two sub-blocks are used as two templates for model updating; Step2.2: generate a standard Gaussian response map; Step2.3: extract HOG features; Step2.4: calculate the HOG feature correlation filter template; the Step6 is specifically: the Step1 is specifically: the upper left corner of each frame in the video frame sequence is the coordinate origin (1, 1), and the width and height are Width and Height respectively, and the rectangular region (x0, y0, w0, h0) of the selected target in the first frame is selected, that is, the selected tracking target; wherein (x0, y0) represents the upper left corner coordinates of the rectangular region, w0 and h0 represent the width and height of the rectangular region respectively, the first frame selected target is also called the current frame tracking result (x1, y1, w1, h1) = (x0, y0, w0, h0), and the subscript represents the current frame number; the Step4 is specifically: using the kernel method based on the Gaussian kernel function, the kernel correlation matrix and the classifier regression coefficient of all target samples and all training samples are calculated; an m*n cyclic matrix X is established, the cyclic matrix is regarded as a block matrix, and the cyclic matrix size is mn*m n; if it is determined that the block matrix at the (i, j) position has the maximum response, that is, the target box is offset by i-1 units downward and j-1 units to the right relative to the previous target box, then the size of the label y is n*n, and the following is obtained after combining the ridge regression algorithm: According to the properties of DFT time domain convolution: The original sequence is regarded as a period, and the following is obtained by verification: Convert the weight matrix w to: At this point, the solution of the weight matrix w is completed. The Step5 is specifically: using the kernel correlation matrix and the classifier regression coefficient to calculate the function response value of all positions, detecting the maximum response position, and then solving the target position of the current frame; The result GL matrix element value of the feature represents the probability that the corresponding candidate target in the search window is the tracking result, so the candidate target corresponding to the maximum element value is the tracking result; wherein: ​ ​ ​ According to the last frame, i.e. t -1 frame tracking result The corresponding rectangular region calculates the current frame, i.e. t Frame candidate target search window ; The first frame search window is determined according to The center point of the search window is calculated as wherein , The width and height of the search window are respectively , ; ​ The search window image can be standardized and transformed into a standard search window according to a search window transformation factor, and the width and height of the standard search window are , , the width and height of the standard target window of the current frame are , ; ​ pos1 = (x_s t ,y_s t +h0 / 4) and pos2 = (x_s t ,y_s t -h0 / 4); ​ ​ Standard Gaussian response map g is a two-dimensional matrix with width and height of , The matrix element value is a probability density function conforming to a two-dimensional Gaussian distribution , calculated according to the formula , wherein δ represents the standard deviation of the two-dimensional Gaussian distribution: ； In the formula, indicates the size of each cell in the HOG feature extraction process , indicates the coordinate position of the element of the Gaussian response graph matrix, the origin is located at the center point of the matrix, and the frequency domain representation of the standard Gaussian response graph can be obtained by Fourier transform , which is the same size as g . ​ In HOG features f t are extracted in the current frame normalized search window with size w_g x h_g x 28, using a cosine window with size w_g x h_g to smooth f t , and then Fourier transform is performed to obtain the frequency domain representation F t of the HOG features, which has the same size as f t . ​ A known frequency domain representation F of the normalized search window HOG feature t and a frequency domain representation G of the standard Gaussian response map, then the frequency domain representation of the HOG feature correlation filter template is H t : H t = G / F t； ​ According to the position of the current frame tracking result (x t , y t , w t , h t ) and the method of Step 2.1, the search window Search'(t) is calculated; Extract the frequency domain representation F of the HOG features in the range of Search'(t) according to the method of Step 2.3 t '; H is calculated according to the method of Step 2.4 t ' = G / F t ' Let η be the update parameter, the current frame HOG feature related filter template H t The update method is shown in the formula: H t = (1 - η)H t - 1 + ηH t . 2.The tracking method based on KCF object patching and confidence template updating mechanism according to claim 1, wherein, ​ ​ 3.The tracking method based on KCF object patching and confidence template updating mechanism of claim 1, wherein, The Step3 is specifically: according to the method of Step2.1, calculating the current frame search window , according to Step2.3, extracting the frequency domain representation F of the current frame direction gradient histogram HOG feature t , the response map G of the current frame HOG feature t ; G t =F t ⊙H t-1 . 4.The tracking method based on KCF object patching and confidence template updating mechanism of claim 1, wherein, ​ ​ ​ ； ； ； ； wherein denotes the conjugate form of x after DFT transformation, δ denotes an all-one vector, which is equivalent to the eigenvector of an identity matrix, and denotes matrix element point multiplication. ​ ； ​ ； That is Thus, we have ； ​ ； ​ 5. The tracking method based on KCF object patching and confidence template updating mechanism according to claim 4, characterized in that, ​ ​ Let the number of candidate targets in the current frame search window be ; make , and These represent the adaptive feature fusion results. The maximum element value in the matrix and its corresponding x and y coordinates are used to determine the tracking result for the current frame. ; ​ w t = w t-1 , h t = h t-1 ; x t = x t-1 + (x_GL max - (w_sn t - w_on t ) / 2) / γ - w t / 2; y t = y t-1 + (y_GL max - h_sn t - h_on t ) / 2) / γ - h t / 2, is a transform factor of the search window.

Images