Tracking method based on multi-scale spatial constraint anti-occlusion

Abstract

This invention discloses a tracking method based on multi-scale spatial constraints to combat occlusion. It extracts HOG, CN, and grayscale features from candidate target regions and uses PCA to reduce the dimensionality of HOG and CN features, accelerating computation. A channel weight fusion method is employed, training filters separately for each feature layer. Adaptive fusion weights enhance the response of effective feature layers, addressing the problem of insufficient multi-feature response fusion. A stepped spatial constraint method is proposed to optimize the color space constraint model, preventing model errors from obscuring target information and limiting the effectiveness of the spatial domain constraint model. Adaptive learning rate and diffusion search methods reduce irrelevant information learned by the filters and improve tracking accuracy when the target is occluded. A tritree scale acceleration method is proposed, introducing scale filters and transforming the parallel structure of the scale filters into a tritree classification structure.

Description

Technical Field

[0001] This invention belongs to the field of target tracking technology, specifically relating to a tracking method based on multi-scale spatial constraints to resist occlusion. Background Technology

[0002] In recent decades, the field of computer vision has developed rapidly, and computers have acquired various image processing capabilities, including object detection, object recognition, object tracking, image segmentation, image super-resolution, and image fusion. Meanwhile, object tracking has been widely applied in video surveillance and security, autonomous driving technology, smart homes and the Internet of Things, medical image processing, and many other fields, playing a vital role in each. In-depth research into the principles and methods of object tracking technology is of great significance for promoting technological innovation and application in related fields.

[0003] Traditional correlation-filtering-based target tracking methods often employ feature fusion to address the issue of limited feature extraction methods. However, this approach only fuses different types of features at the response level, neglecting the consideration of individual feature layers. Regarding boundary effects, targets are not always square; even when spatial constraints are applied to target information, irrelevant information can still be used as positive samples during training, impacting the filter model. When the target is occluded, the information in the target region will not contain target information, leading to negative effects during filter learning. Furthermore, at multiple scales, existing methods always require calculating a large number of candidate region responses, resulting in slow processing speeds. Summary of the Invention

[0004] In view of this, the main objective of the present invention is to provide a tracking method based on multi-scale spatial constraints to resist occlusion.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0006] This invention provides a tracking method for resisting occlusion based on multi-scale spatial constraints, the method comprising:

[0007] Step 1: Input the first frame image and initialize the feature filter, scale filter, and color histogram model;

[0008] Step 2, enter the number t For each frame image, firstly, the gradient histogram (HOG) features of the candidate target region are extracted. Then, the HOG features are dimensionality-reduced using PCA to extract the fused gradient orientation histogram (FHOG) features of the candidate target region. Next, the color name (CN) features and grayscale features of the candidate target region at five layers are extracted. t >1;

[0009] Step 3, determine the first tThe response results of each channel in the histogram of gradients (HOG), fused histogram of gradient directions (FHOG), color name (CN), and grayscale features of the frame image are analyzed, and the second weight of the channel is determined; based on the first... t-1 The response result of each single channel in the frame image is used to determine the first channel weight; the final single channel weight is obtained based on the first channel weight and the second channel weight; the response is fused using the single channel weights to determine the first single channel weight. t The final feature response result of the frame image;

[0010] Step 4: Determine the first [characteristic] based on the maximum value of the final characteristic response result. t The target location in the frame image is predicted. Based on the predicted target location, the ternary tree scaling acceleration method is used to determine the [missing information - likely a specific value]. t Target size of the frame image;

[0011] Step 5: Extract basic samples by using the foreground and background and related information of the target's predicted position in the current frame, and process the samples using a step space constraint model.

[0012] Step 6: Determine the adaptive learning rate based on the second weight of the current frame channel obtained in Step 3, and determine the first weight using the learning rate. t+1 The search range for the frame image;

[0013] Step 7: Train and update the scaling filter model, correlation filter model, and color histogram model;

[0014] Step 8: Repeat steps 2 through 7 until the tracking ends.

[0015] In the above scheme, the initialization of the feature filter, scale filter, and color histogram model by inputting the first frame image specifically includes:

[0016] The target region of the first frame image has been manually labeled. Training samples are constructed by cyclically shifting the target region as the base sample.

[0017] The ridge regression method is used to obtain the initial filter model through the training samples. , ,in a This represents the fundamental vector corresponding to the sample. b This represents the standard response corresponding to the sample. This represents the filter model parameters to be solved. Represents the regularization coefficient;

[0018] Optimizing the filter model using kernel function method The optimized solution of the filter model is obtained as follows: In the formula, The representative sample is related to its own kernel in the calculation. I Represents the identity matrix;

[0019] An initial color histogram model is obtained based on the target region as the base sample;

[0020] An initial scale filter model is obtained based on the target region as the base sample.

[0021] In the above scheme, the input number t For each frame image, the gradient histogram (HOG) features of the candidate target region are extracted, including the following specific steps:

[0022] Convert the RGB image of the candidate region to a grayscale image using the following formula;

[0023] Perform Gamma correction on the image after grayscale conversion;

[0024] Using [-1, 0, 1] as the horizontal gradient operator, [-1, 0, 1] T The vertical gradient operator is used to determine the horizontal and vertical gradient vectors of the corrected image.

[0025] The magnitude of the gradient is determined based on the horizontal and vertical image gradient vectors. G ( x , y | with gradient direction θ ( x , y );

[0026] The gamma-corrected image is divided into small cell units (Cells). The gradient direction within each cell, ranging from 0° to 180°, is divided into nine parts, with each 20° direction represented by a directional region. Z1 to Z9 represent these nine directional regions. The gradients of these nine parts are weighted and statistically analyzed to obtain the gradient direction histogram of the Cell, which is a feature vector of length 9.

[0027] Several adjacent cells are combined into a block. The feature vectors of all cells in each block are concatenated to obtain the HOG feature of the current block. The gradient histogram within the block is normalized to obtain the normalized gradient magnitude.

[0028] The HOG features from all blocks are integrated to obtain 36-dimensional HOG features.

[0029] In the above scheme, the step of extracting the fused gradient orientation histogram (FHOG) feature of the candidate target region by dimensionality reduction of the HOG feature using the PCA method specifically includes:

[0030] Divide each 20° segment of the 360° into the same category, and determine each pixel point ( x , y The direction-insensitive feature corresponding to ) F 1. Direction-sensitive characteristics F 2;

[0031] Define pixel-level feature maps V Determine the gradient histogram for each pixel;

[0032] Determine the feature vector of each cell based on the size of the image;

[0033] The gradient energy of each cell is determined based on the eigenvector of each cell;

[0034] Gradient energy of each cell E Normalize the feature vector to obtain a value of . W × H And the direction-sensitive feature with a depth of 18 and a size of W × H And the direction-insensitive feature at depth 9 and size is W × H Furthermore, texture features with a depth of 4 ultimately form a size of W × H And it has FHOG features with a depth of 31.

[0035] In the above scheme, the step of further extracting the color name (CN) features of the candidate target region at the 5th layer specifically includes:

[0036] Convert the candidate target region image from RGB space to color space;

[0037] Feature dimensionality reduction is performed on candidate target region images converted to color space using the PCA method;

[0038] The feature vector matrix corresponding to the five largest feature values ​​in the dimensionality-reduced candidate target region image is extracted to form a 5-dimensional CN feature.

[0039] In the above scheme, the step of further extracting the grayscale features of the candidate target region specifically includes:

[0040] Convert the RGB image of the candidate target region to a grayscale image.

[0041] The converted grayscale image is normalized to control the pixel values ​​within the range of [-0.5, 0.5], thus obtaining the final grayscale features.

[0042] In the above scheme, the step according to the first t-1 The response result of each single channel in the frame image determines the first weight of the channel. Specifically, it includes:

[0043] Channel First Weight For this feature channel number t-1 The maximum peak value of the corresponding response during frame image detection.

[0044] In the formula, f d Indicates features from the previous frame. h d Represented as the first t-1 The filter coefficients of the frame.

[0045] In the above scheme, the determination of the first t The response results of each channel in the histogram of gradients (HOG), fused histogram of gradient directions (FHOG), color name (CN), and grayscale features of the frame image are analyzed, and the second weight of each channel is determined. Specifically, this includes: obtaining the first... t The feature response map of the frame is processed using a non-maximum suppression algorithm with a window size of 3 × 3 to obtain the value of the first peak. The value of the second peak ,according to Determine the second weight of the channel .

[0046] In the above scheme, the step of obtaining the final single-channel weight based on the first channel weight and the second channel weight, and then performing response fusion using the single-channel weight to determine the final response result, specifically includes: based on... Determine the single-channel weight In the formula, i Indicates that the channel number is the first i indivual, n Indicates the total number of feature channels;

[0047] according to Determine the final response result r es .

[0048] In the above scheme, the target prediction location is determined based on the maximum value of the final response result, and the ternary tree scaling acceleration method is used to determine the target prediction location. t The target scale of the frame image specifically includes

[0049] The final response result obtained r es The coordinates of the maximum value point ( x , y This is the predicted position of the target in the current frame;

[0050] Based on the predicted target location, the target scale size in the current frame is determined using a ternary tree scaling acceleration method.

[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0052] (1) To address the problem of insufficient fusion of multi-feature responses, a channel weight fusion method is adopted. For each feature layer, a filter is trained separately, and the response effect of the effective feature layer is made more prominent through adaptive fusion weights. In addition, HOG features, CN features and grayscale features of candidate regions are extracted, and PCA method is used to reduce the dimensionality of HOG features and CN features to accelerate the computational efficiency.

[0053] (2) To address the problem that the spatial constraint model has limited effect, a stepped spatial constraint method is proposed. This method optimizes the color space constraint model, effectively preserving the target area and preventing the constraint model from erroneously obscuring the target information when the color features are not obvious.

[0054] (3) To address the problem that the filter learns a lot of irrelevant information and the target is easily lost when the target is occluded, an anti-occlusion strategy is designed. By using an adaptive learning rate and a diffusion search method, the irrelevant information learned by the filter is reduced and the tracking accuracy when the target is occluded is improved.

[0055] (4) To address the problem of excessive computation in existing multi-scale methods, a tritree scale acceleration method is proposed. The scale filter is introduced into the algorithm of this invention, and the parallel structure of the scale filter is transformed into a tritree classification structure based on the scale filter to accelerate the computation efficiency. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and, together with their descriptions, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0057] Figure 1 This is the result of Bird1 sequence tracking.

[0058] Figure 2 This is the result of BlurOwl sequence tracking.

[0059] Figure 3 This is the result of Box sequence tracking.

[0060] Figure 4 This is the result of Panda sequence tracking.

[0061] Figure 5 This is the result of DragonBaby sequence tracking.

[0062] Figure 6This is the result of Human6 sequence tracking.

[0063] Figure 7 This is the result of Human7 sequence tracking.

[0064] Figure 8 This is the result of Ironman sequence tracing.

[0065] Figure 9 It is a curve showing the accuracy and success rate of each algorithm after summarizing the data. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0067] This invention provides a tracking method for resisting occlusion based on multi-scale spatial constraints, the implementation steps of which are as follows:

[0068] Step 1: Input the first frame image and initialize the feature filter, scale filter, and color histogram model.

[0069] Step 2, enter the number t ( t >1) For each frame image, first extract the gradient histogram (HOG) features of the candidate target region, and then use the PCA method to reduce the dimensionality and extract the fused gradient direction histogram (FHOG) features of the candidate target region; then extract the color name (CN) features and grayscale features of the 5 layers of the candidate target region.

[0070] Step 3: Calculate the response result of each channel of all features separately. Based on the response result of each single channel in the previous frame, obtain the first channel weight. Based on the response result of each single channel in the current image, obtain the second channel weight. Based on the first channel weight and the second channel weight, obtain the final single channel weight. Perform response fusion through the single channel weight to obtain the final feature response result of the t-th frame image.

[0071] Step 4: Determine the target prediction location based on the maximum value of the final feature response result. Based on the target prediction location, use the tritree scaling method to determine the target scale size of the current frame.

[0072] Step 5: Extract basic samples by using the foreground and background and related information of the target's predicted position in the current frame, and process the samples using a step space constraint model.

[0073] Step 6: Determine the adaptive learning rate based on the second weight of the current frame channel obtained in Step 3, and determine the first weight using the learning rate. t+1The search range for frame images.

[0074] Step 7: Train and update the scaling filter model, correlation filter model, and color histogram model.

[0075] Step 8: Repeat steps 2 through 7 until the tracking ends.

[0076] Step 1 initializes the feature filter, scaling filter, and color histogram model. This includes the following specific steps:

[0077] Step 1.1: The target region of the first frame image has been manually labeled. Using the target region as the base sample, training samples are constructed by cyclically shifting it. The specific operation method is as follows:

[0078] (1) Let the input image be I Size is i × j , represented as

[0079]

[0080] (2) Construct the permutation matrix P , Q This enables rapid image translation using a matrix. P , Q All of them are in the form of

[0081] P , Q The size is i × j

[0082] (3) Matrix P Responsible for vertical translation, matrix P Left multiplication I once, I Move down one row, matrix Q Responsible for left and right translation, matrix Q Right multiplication I once, I Shift one column to the right, the formula is expressed as follows:

[0083]

[0084] In the formula, l and r This indicates the number of vertical and horizontal translations. The base sample is constructed into a cyclic matrix by the above cyclic displacement operations and used as the training sample.

[0085] Step 1.2: Use the ridge regression method to initialize the filter model using the training samples obtained in Step 1.1. The specific operating method is as follows:

[0086] (1) Let the training sample matrix obtained in step 1.1 be . A , x For unknown vectors, Ax That is, the model's predicted value; let... b This represents the standard response corresponding to the sample, i.e., the actual observed data. The ideal relationship between training samples and standard responses is:

[0087]

[0088] (2) In the actual tracking process, there is an error between the model prediction and the observed data. In order to minimize the sum of squared errors, a correction vector is introduced. Δb The solution to the original equation is transformed into finding the correction vector. Δb The minimum value. At this point, for the equation The solution is equivalent to

[0089]

[0090] (3) Add a regularization term to the above equation to make it have a unique analytical solution and prevent overfitting. The objective function of ridge regression is then obtained.

[0091]

[0092] In the formula, Represents the regression function. a This represents the fundamental vector corresponding to the sample. b This represents the standard response corresponding to the sample. This represents the filter model parameters that we want to solve for. Let represent the regularization coefficient. Finding its minimum value and extending it to the complex domain yields...

[0093]

[0094] In the formula, H represents the conjugate transpose.

[0095] Fourier diagonalization Calculations were performed to obtain Substituting these values ​​into the above equation yields the filter model parameters. for

[0096]

[0097] Performing a Fourier transform on both sides of the above equation and simplifying it using the properties of diagonal matrices, we obtain...

[0098]

[0099] Step 1.3: Optimize the filter model using the kernel function method. The specific operation method is as follows:

[0100] (1) Parameters to be solved Transition to dual space ,have to

[0101]

[0102] At this point, the result variable is changed from Become , Represents the first dimension of the low-dimensional space N The mapping weights of each sample in the high-dimensional space. Represents the fundamental vector corresponding to the sample

[0103] (2) The objective function can be rewritten from the above equation as follows:

[0104]

[0105] In the formula, Represents training samples in high-dimensional space and The inner product of the two functions. The regression function becomes...

[0106]

[0107] The optimized solution of the filter model is calculated as follows:

[0108]

[0109] In the formula, The representative sample is related to its own kernel in the calculation. I Represents the identity matrix.

[0110] Step 1.4: Initialize the color histogram model using the target region as the base sample. The specific operation method is as follows:

[0111] (1) The foreground region of the target region is O The surrounding space is the background area. S Foreground area O With background area S Together they form the target candidate region I According to Bayesian statistical formulas, pixel points x The likelihood probability of belonging to the target is

[0112]

[0113] In the formula, b This represents the category intervals of the color histogram. b x express b pixels within the range xA combination of RGB values.

[0114] (2) Estimate the likelihood term for the likelihood probability using the following method:

[0115]

[0116] In the formula, express I middle O within the area b The number of pixels in the classification interval. express I middle S within the area b The number of pixels in the classification interval, | O | represents the area of ​​the foreground region,| S | Indicates the area of ​​the background region

[0117] (3) Based on the likelihood term estimation, the likelihood probability that pixel x in the target candidate region belongs to the target can be expressed as:

[0118]

[0119] In the formula, 0.5 represents the estimated response value, if the pixel point x RGB combination b x ,exist O and S None of them appeared, so the probability that they belong to the target is 0.5.

[0120] Step 1.5: Initialize the scale filter model using the target region as the base sample. The specific operation method is as follows:

[0121] Building a scale pyramid C s As a scale filter, it is used to calculate the target scale and scale pyramid. C s The formula is expressed as

[0122]

[0123] In the formula, N c This indicates the number of scales in the scale pyramid. N c The value is 33. Indicates the scale factor. The value is 1.02.

[0124] In step 2, enter the first... t ( t>1) For each frame image, first extract the Histogram of Gradients (HOG) features, then use PCA to reduce the dimensionality and extract the Fused Histogram of Gradient Orientation (FHOG) features; finally, extract the color name (CN) features and grayscale features from the five layers of the image. The specific steps include:

[0125] Step 2.1: Extract HOG features from candidate target regions. The specific operation method is as follows:

[0126] (1) Image grayscale conversion

[0127] Convert the RGB image of the candidate region to a grayscale image using the following formula.

[0128]

[0129] In the formula, I gray This represents the converted grayscale image. R c , G c , B c These represent the red, green, and blue components of the original image, respectively.

[0130] (2) Gamma correction

[0131] Gamma correction is applied to the grayscale converted image to normalize the pixel values. The correction formula is as follows:

[0132]

[0133] In the formula, γ g This represents the compression factor, typically taken as 0.5. x , y ) represents a pixel. H g ( x , y () represents the image of the correction result.

[0134] (3) Calculate the image gradient

[0135] Using [-1, 0, 1] as the horizontal gradient operator, [-1, 0, 1] T As a vertical gradient operator, the image gradient is calculated.

[0136]

[0137] In the formula, G x ( x , yThe horizontal gradient vector is represented by (). The vertical gradient of the image is calculated using the following formula.

[0138]

[0139] In the formula, G y ( x , y ) represents the gradient vector in the vertical direction.

[0140] Gradient magnitude | G ( x , y | with gradient direction θ ( x , y ) Calculate using the following two formulas respectively

[0141]

[0142]

[0143] (4) Constructing a gradient direction histogram

[0144] The gamma-corrected image is divided into small cellular units (also known as cells). The gradient direction within each cell, ranging from 0° to 180°, is divided into nine parts, with each 20° direction represented by a directional region. Z1 to Z9 represent the nine directional regions respectively. The gradients of the nine parts are weighted and statistically analyzed to obtain the gradient direction histogram of the cell, which is a feature vector of length 9.

[0145] (5) Combine Cell into Block

[0146] Several adjacent cells are merged into a single block. Within each block, the feature vectors of all its cells are concatenated to form the HOG feature of that block. The gradient histogram within the block is then normalized; the normalized gradient magnitude is...

[0147]

[0148] In the formula, g i This represents the initial value of the gradient. ε This represents a very small constant to prevent the denominator from being zero.

[0149] (6) Integrating HOG features

[0150] The HOG features from all blocks are integrated to obtain 36-dimensional HOG features.

[0151] Step 2.2: Perform PAC dimensionality reduction on the HOG features obtained in Step 2.1 to obtain FHOG features. The specific operation method is as follows:

[0152] (1) Considering the directional gradient, divide the 360° into 20° groups and calculate the gradient for each pixel. x , y The direction-insensitive feature corresponding to ) F 1. Direction-sensitive characteristics F 2. The calculation formula is as follows:

[0153]

[0154] In the formula, p This represents the class number, with a value of 18.

[0155] (2) Define pixel-level feature mapping V Calculate the gradient histogram for each pixel using the following formula:

[0156]

[0157] In the formula, b = {1, 2, …, p}

[0158] (3) Features of all pixels V polymerization

[0159] Let the image size be W × H Then the feature vector of each cell is

[0160]

[0161] In the formula, S hc Indicates cell size

[0162] The gradient energy of each cell is calculated using the following formula.

[0163]

[0164] In the formula, and It belongs to {-1, 1}. It represents a very small number that is not zero.

[0165] (4) Synthetic FHOG characteristics

[0166] Gradient energy of each cell E Normalize the feature vector to obtain a value of . W × HAnd the direction-sensitive feature with a depth of 18 and a size of W × H And the direction-insensitive feature at depth 9 and size is W × H Furthermore, texture features with a depth of 4 ultimately form a size of W × H And it has FHOG features with a depth of 31.

[0167] Step 2.3: Extract the color name features of the candidate target regions. The specific operation method is as follows:

[0168] (1) Convert the candidate target region image from RGB space to color space.

[0169] Let the transformation matrix be... L , represented as

[0170]

[0171] In the formula, i = 11, through matrix L The process of converting an image from RGB space to color space is represented as follows:

[0172]

[0173] In the formula, P cn Let be the feature matrix, representing the transformed 11-dimensional color features. i Color characteristics p i Represented as

[0174]

[0175] (2) Feature dimensionality reduction using PCA method

[0176] Feature matrix P cn covariance matrix P cov for

[0177]

[0178] Calculate using the following formula P cov eigenvalues and eigenvectors x

[0179]

[0180] In the formula, E Represents the identity matrix.

[0181] (3) Extract the feature vector matrix corresponding to the 5 largest feature values ​​to form a 5-dimensional CN feature.

[0182] Step 2.4: Extract the grayscale features of the candidate target regions. The specific operation method is as follows:

[0183] (1) Convert the RGB image of the candidate target region to a grayscale image using the following formula:

[0184]

[0185] In the formula, I gray This represents the converted image. R Indicates the magnitude of the red component. G Indicates the magnitude of the green component. B This indicates the magnitude of the blue component.

[0186] (2) Normalize the image to control the pixel values ​​within [-0.5, 0.5], expressed by the formula:

[0187]

[0188] In the formula, F gray This represents the final grayscale feature obtained.

[0189] In step 3, the response result is calculated separately for each channel of all features in the previous frame and the current image. Based on the response result of each single channel in the previous frame, the first channel weight is obtained; based on the response result of each single channel in the current image, the second channel weight is obtained; the final single-channel weight is obtained based on the first and second channel weights; and the response is fused using these single-channel weights to obtain the final response result. The specific steps include the following:

[0190] Step 3.1: Calculate the first weight of each feature channel in the previous frame image. The specific operating method is as follows:

[0191] Channel First Weight The maximum peak value of the response corresponding to the detection in the previous frame of this feature channel is calculated using the following formula.

[0192]

[0193] In the formula, f d Indicates features from the previous frame. h d These are represented as the filter coefficients from the previous frame. When the first frame is initialized, Set the value to 1.

[0194] Step 3.2: Calculate the second weight for each feature channel of the current frame image. The specific operating method is as follows:

[0195] The feature response map of the current frame is obtained. This response map is then processed using a non-maximum suppression algorithm with a window size of 3 × 3 to obtain the value of the first peak. The value of the second peak Set the second weight of the channel for To prevent the value from being too small, a threshold of 0.5 is set. Therefore, the second channel weight... It can be expressed as a formula

[0196]

[0197] Step 3.3: Calculate the single-channel weight based on the first channel weight and the second channel weight obtained in Steps 3.1 and 3.2. The specific operating method is as follows:

[0198] The following formula can be used to calculate it.

[0199]

[0200] In the formula, i Indicates that the channel number is the first i indivual, n Indicates the total number of feature channels.

[0201] Step 3.4, calculate the feature fusion response of the current frame. r es The specific operating method is as follows:

[0202] The feature fusion response of the current frame is

[0203]

[0204] In the formula, r es This indicates the final response of the current frame.

[0205] In step 4, the target prediction location is determined based on the maximum value of the final response result; based on the target prediction location, the target scale size of the current frame is determined using the ternary tree scaling acceleration method, as detailed below:

[0206] Step 4.1, determine the predicted location of the target. The specific operation method is as follows:

[0207] The fusion feature response obtained in step 3 r es The coordinates of the maximum value point (x , y This is the predicted position of the target in the current frame.

[0208] Step 4.2: Based on the predicted target location, use the ternary tree scaling method to determine the target scale in the current frame. The specific operation method is as follows:

[0209] (1) Constructing a scale filter

[0210] After determining the target location in the current frame, construct a scale pyramid. C s Construct multiple candidate target scales and calculate the target scale, scale pyramid. C s The formula is expressed as

[0211]

[0212] In the formula, N c This indicates the number of scales in the scale pyramid. N c The value is 33. Indicates the scale factor. The value is 1.02. Assume the target size is... W × H Then the scale pyramid is constructed in the first... i The target scale is .

[0213] (2) Obtain the target scale using a tritree classification structure.

[0214] Using a ternary tree classification structure, N c The value is 33, and the calculation is performed at the first level. i Characteristic responses at values ​​of 7, 17, and 27 R 7. R 17 , R 27 Take the value corresponding to the maximum response. n 1. Calculation at the second level n Pick n 1-4 n Characteristic response at 1+4 R n1-4 , R n1+4 Combine these two responses with R n1 Compare and take the response with the maximum value. n 2. Calculation at the third level n Pick n 2-2、 n 2-1n 2+1 n Response corresponding to 2+2 R n2-2 , R n2-1 , R n2+1 , R n2+2 These four responses and R n2 Compare and take the response with the maximum value. n 3, n = n The image patch scale corresponding to time 3 is the target scale of that frame.

[0215] In step 5, basic samples are extracted, and a ladder space constraint model is constructed using the foreground and background information and related data from the target prediction results. The specific steps are as follows:

[0216] Step 5.1: Construct training samples based on the target candidate regions as the base samples. The specific operation method is as follows:

[0217] (1) Given the target candidate region image I Size is i × j , represented as

[0218]

[0219] (2) Construct the permutation matrix P , Q This enables rapid image translation using a matrix. P , Q All of them are in the form of

[0220] P , Q The size is i × j

[0221] (3) Construct training samples according to the following formula

[0222]

[0223] In the formula, l and r This indicates the number of vertical and horizontal translations. The base sample is constructed into a cyclic matrix by the above cyclic displacement operations and used as the training sample.

[0224] Step 5.2, calculate the color space constraint model. The specific operation method is as follows:

[0225] Let the color space constraint model be mc , can be represented as

[0226]

[0227] In the formula, i Indicates the first in the training region i 1 pixel m c, i express i The model value corresponding to the point. The value of the point within the training region. i The color information of each pixel is represented as r i Coordinate information is represented as x i You can get i Joint probability at point p ( y i ) is defined as

[0228]

[0229] In the formula, p ( r i |m ci = j () indicates the possibility of appearance. p ( x i |m ci = j ) represents spatial possibility. p ( m ci = j ) represents the prior probabilities of the foreground and background.

[0230] Appearance possibilities p ( r i |m ci = j The value is calculated using the Bayesian criterion of the target foreground and background color models. For a given foreground region... O The surrounding space is the background area. S Foreground area O With background area S Together they form the target candidate region I According to Bayesian statistical formulas, the probability of an appearance can be calculated using the following formula.

[0231]

[0232] In the formula, bThis represents the category intervals of the color histogram. b x express b pixels within the range x A combination of RGB values.

[0233] Spatial Possibilities p ( x i |m ci = j The meaning is the positional reliability of each pixel, and the calculation method is as follows:

[0234]

[0235] In the formula, r This represents the distance from the current pixel to the center pixel. This represents the diagonal length of the corresponding bounding box. After obtaining the result, it is restricted to [0.5, 0.9]. Values ​​greater than 0.9 are set to 0.9, and values ​​less than 0.5 are set to 0.5.

[0236] Prior probabilities of foreground and background p ( m ci = j This refers to the ratio between the sizes of the regions extracted from the foreground and background histograms, calculated as follows:

[0237]

[0238] In the formula, S f Indicates the size of the foreground region. S b By representing the area of ​​the background region, we can obtain the final color space constraint model. m c

[0239] Step 5.3, Calculate the staircase space constraint model m l The specific operating method is as follows:

[0240] Construct a Hanning window of the same size as the model. The formula for the discrete coefficients of the Hanning window is:

[0241]

[0242] In the formula, the number of discrete points is N +1, the two one-dimensional Hanning window vectors can be determined by the model's width and height, represented as:

[0243]

[0244] A two-dimensional Hanning window can be obtained. h an2 for

[0245]

[0246] Next, the color space constraint model will be implemented. m c With the two-dimensional Hanning window h an2 By performing weighted fusion and linear stretching to restrict the value of each point in the model to the interval [0, 1], the stepped space constraint model is obtained. m l The formula is

[0247]

[0248] In the formula, i Represents the first in the model region i 1 pixel j This represents the number of pixels within the model region. m i In the step space constraint model, the first... i The value of each pixel.

[0249] In step 6, an adaptive learning rate is calculated based on the response of each channel in the current frame, and the candidate target region for the next frame image is determined using a diffusion search method based on the learning rate. The specific steps are as follows.

[0250] Step 6.1: Calculate the learning rate parameter for the feature response of the current frame. The specific operation method is as follows:

[0251] (1) Use the mean APCE as the APCE learning rate parameter The metric, assuming the current image is the [number]th [image]. t The APCE learning rate parameter can be expressed by the formula:

[0252]

[0253] (2) Adjust the second weight of the channel It is incorporated into the learning rate, making it a channel learning rate parameter. The formula is expressed as

[0254]

[0255] (3) Multiply the two learning rate parameters to obtain the final learning rate parameter. for

[0256]

[0257] Assume the base learning rate of the algorithm is set to 1. a The final adaptive learning rate is then obtained. for

[0258]

[0259] Train a filter separately for each feature channel, assuming that there are n One channel, then you will get n There are three adaptive learning rates, respectively. .

[0260] Step 6.2: Use diffusion search to determine the candidate target region for the next frame image. The specific operation method is as follows:

[0261] Assuming the original detection area S Size is w × h Set diffusion judgment threshold The adaptive learning rate is 0.3. Less than the diffusion judgment threshold At that time, based on the adaptive learning rate The size of the magnification factor adjusts the size of the detection area. It can be represented as

[0262]

[0263] When adaptive learning rate When the value is less than 0.3, the detection region size in the next frame is... When continuous n Adaptive learning rate during frame detection When it is less than 0.3, then the first... n The size of the frame detection region can be expressed by the formula:

[0264]

[0265] In the formula, w s and h s This represents the width and height of the expanded detection region, which is the size of the updated detection region for the next frame; if the adaptive learning rate of the next frame... If the value is greater than 0.3, the detection area size in the next frame will be restored to its original size.

[0266] In step 7, the scaling filter model, correlation filter model, and color histogram model are trained and updated. The specific processing steps are as follows:

[0267] Step 7.1: Construct training samples using the target candidate regions of the next frame obtained in Step 6 as the base samples. The specific steps are as follows.

[0268] (1) Given the target candidate region image I Size is i × j , represented as

[0269]

[0270] (2) Construct the permutation matrix P , Q This enables rapid image translation using a matrix. P , Q All of them are in the form of

[0271] P , Q The size is i × j

[0272] (3) Construct training samples according to the following formula

[0273]

[0274] In the formula, l and r This indicates the number of vertical and horizontal translations. The base sample is constructed into a cyclic matrix by the above cyclic displacement operations and used as the training sample.

[0275] Step 7.2: Construct a kernel matrix using the training samples obtained in Step 7.1 and the target candidate regions. The calculation method is as follows:

[0276]

[0277] In the formula, This represents the kernel matrix formed by the training samples and the target candidate regions. Indicates training samples a Samples from the area to be detected z The kernel-related calculations.

[0278] Substituting the above equation into the objective function expression The predicted response R(z) in the time domain can be obtained, expressed as:

[0279]

[0280] In the formula, This represents the kernel matrix formed by the training samples and the target candidate regions. This represents the correlation filter model coefficients of the previous frame image; the target location is... The position corresponding to the maximum value.

[0281] Step 7.3, for the color filter model x Scale filter model C、 coefficients of the correlation filter To update, the specific method is as follows:

[0282]

[0283]

[0284]

[0285] In the formula, η This represents the learning rate, set to 0.02. x t This represents the target sample vector of the current frame. x 1: t Indicates from the beginning to t The target sample vector learned at each step; α t Represents the correlation filter coefficients of the current frame. α 1: t This represents the correlation filter coefficients learned from the beginning up to time t; C t This represents the scale filter coefficients of the current frame. C 1: t This represents the scale filter coefficients learned from the beginning up to time t.

[0286] In step 8, repeat steps 2 through 7 until the tracking ends.

[0287] The effects of the present invention will be further described below with reference to simulation experiments.

[0288] Figures 1 to 8 The experimental results of the proposed algorithm and the comparison algorithms are presented on various sequences, where the proposed algorithm is denoted as SCAOMT (Spatial Constraints Against Occlusion Multiscale Tracking). The selected comparison algorithms are all correlation filter-based tracking algorithms, including KCF, SAMF, SRDCF, BACF, and LCT.

[0289] Human9 sequence results are as follows Figure 1As shown, the target in this sequence exhibits characteristics of varying illumination, scale, deformation, and rapid motion. In frame 80, the target moves forward and its scale decreases. Due to the larger target bounding box scale in the earlier stages of the LCT algorithm, it learned more background information, leading to LCT tracking drift. In frame 140, although the remaining five algorithms can track the target, KCF, SAMF, and BACF algorithms cannot adapt to the target scale changes, and the model information is also contaminated to some extent. By frame 395, the target has shrunk further, and only the algorithm of this invention can continue to maintain tracking accuracy; the other algorithms all experience some degree of drift.

[0290] Skiing sequence results are as follows: Figure 2 As shown, the targets in this sequence exhibit characteristics such as low resolution, in-plane rotation, out-of-plane rotation, and deformation. In frame 7, the skier occupies very few pixels and is rotating in the air; the KCF, LCT, SAMF, and BACF algorithms all experience some degree of drift. In frame 11, the skier in red clothing is moving rapidly through the air and is not easily distinguishable from the trees behind them; only the algorithm presented in this paper and the SRDCF algorithm continue to track them. By frame 25, only the algorithm presented in this paper continues to track them.

[0291] MotorRlling sequence results are as follows Figure 3 As shown, the target in this sequence exhibits characteristics such as background clutter, rapid movement, in-plane rotation, and motion blur. In frame 10, the motorcycle and driver continue to move upwards. The other algorithms treat the lights as part of the target. Because the algorithm of this invention uses a stepped space constraint model, it is more effective in learning the target region, resulting in more accurate tracking. In frame 26, the motorcycle and driver fall downwards, prompting all algorithms to re-track them. By frame 35, the motorcycle and driver continue to fall. Since the other comparison algorithms have learned a large amount of incorrect information, the correct information in a few frames cannot change the erroneous models of these algorithms. Therefore, only the algorithm of this invention can accurately track them.

[0292] The matrix sequence results are as follows Figure 4 As shown, the target in this sequence exhibits characteristics such as lighting changes, background clutter, rapid movement, and in-plane rotation. In frame 47, the scene brightens, and the character's head moves forward. The SRDCF algorithm fails to adapt to the rapid changes in features, drifting to the character's legs, and the other algorithms also exhibit some degree of drift. In frame 53, the character leans back, and the face brightens. The KCF, LCT, SAMF, and BACF algorithms all fail to track the target, and the tracking target of the algorithm in this invention shifts from the character's head to the ear. By frame 88, the algorithm in this invention continues to track the character's ear, and although it fails to adapt to the target's scale changes, it still maintains a certain level of accuracy.

[0293] Lemming sequence results are as follows Figure 5As shown, the target in this sequence exhibits characteristics such as occlusion, exceeding the field of view, scale changes, and out-of-plane rotation. In frame 370, the target appears after being occluded by a lighter; the SRDCF, KCF, LCT, and BACF algorithms all fail to follow the target's movement. In frame 920, the target grows larger and re-enters the target area; the KCF and LCT algorithms return to normal tracking, but fail to adapt to the target's scale. By frame 1003, while the KCF and SAMF algorithms continue tracking, their scale remains unadapted; only the algorithm of this invention maintains a good level in terms of both scale and accuracy.

[0294] KiteSurf sequence results are as follows: Figure 6 As shown, the target in this sequence exhibits characteristics of lighting changes, occlusion, in-plane rotation, and out-of-plane rotation. In frame 29, the surfer is dragged and rises rapidly; the KCF algorithm fails to track due to the small background area during detection, and the SRDCF and LCT algorithms experience some drift. In frame 34, the target moves to the upper left of the image; the SAMF and BACF algorithms treat the TV channel icon information as part of the target. By frame 51, the SAMF and BACF algorithms switch to tracking the TV channel icon, and all remaining algorithms except the one described in this invention fail to track it.

[0295] Human2 sequence results are as follows Figure 7 As shown, the target in this sequence exhibits characteristics of illumination changes, scale changes, and out-of-plane rotation. In frame 221, the target moves to the right, behind the chair, turns, and moves to the left. The BACF algorithm tracks the target as a chair, and the other algorithms continue tracking, but the KCF, LCT, and SAMF algorithms track too large a scale. In frame 382, ​​the target moves rapidly to the left and is partially occluded. The LCT algorithm loses track of the target, while the remaining algorithms continue tracking. By frame 703, the KCF, LCT, SAMF, and BACF algorithms have all failed to track. Although the SRDCF algorithm continues tracking, its scale is clearly inconsistent with the target size. Only the algorithm of this invention maintains a good level in terms of scale and accuracy.

[0296] Girl2 sequence results are as follows Figure 8 As shown, the target in this sequence exhibits characteristics such as occlusion, deformation, motion blur, and scale changes. In frame 107, the left side of the target is partially occluded by a pedestrian, causing the SRDCF and KCF algorithms to drift. In frame 113, the target is completely occluded, the KCF algorithm drifts significantly, while the other algorithms remain within a reasonable range. By frame 120, the target reappears. During the period the target was occluded, the other algorithms continued to learn error information. The algorithm of this invention maintains model accuracy by controlling the learning rate; therefore, only the algorithm of this invention can continue tracking, while all other algorithms fail to track.

[0297] Figure 9The images show the CLE and OR curves of the algorithm proposed in this invention and the comparison algorithm in batch simulations on the OTB100 dataset.

[0298] Table 1 shows the performance improvement of the proposed algorithm compared to other algorithms. Compared to KCF, SAMF, and BACF algorithms, the proposed algorithm shows a decrease in speed but an increase in accuracy, with AUC improved by 38.51%, 8.35%, and 13.83%, respectively, and precision improved by 63.64%, 9.76%, and 7.88%, respectively. Compared to SRDCF and LCT algorithms, it shows improvements in all aspects, with AUC improved by 7.94% and 5.55%, precision improved by 9.45% and 6.96%, and FPS improved by 434.35% and 53.12%, respectively.

[0299] Table 2 presents the quantitative metrics of CLE and OR obtained from simulations of the proposed algorithm and the comparative algorithms. The algorithm ranked first is indicated by bold and underline, and the second-ranked algorithm is also indicated by bold. In all batch simulation results of OTB100, the CLE and OR curves of the proposed algorithm are both at the top. In the AUC ranking, the proposed algorithm ranks first with a score of 0.856, and the LCT algorithm ranks second with a score of 0.811. In the precision ranking, the proposed algorithm ranks first with a score of 0.630, and the LCT algorithm ranks second with a score of 0.589.

[0300] Table 1 Performance improvement rate of the algorithm proposed in this invention

[0301]

[0302] Table 2 Performance metrics after summarizing data for each algorithm

[0303]

[0304] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A tracking method for resisting occlusion based on multi-scale spatial constraints, characterized in that, The method includes: Step 1: Input the first frame image and initialize the feature filter, scale filter, and color histogram model; Step 2, enter the number t For each frame image, firstly, the gradient histogram (HOG) features of the candidate target region are extracted. Then, the HOG features are dimensionality-reduced using PCA to extract the fused gradient orientation histogram (FHOG) features of the candidate target region. Next, the color name (CN) features and grayscale features of the candidate target region at five layers are extracted. t > 1; Step 3, determine the first t The response results of each channel in the histogram of gradients (HOG), fused histogram of gradient directions (FHOG), color name (CN), and grayscale features of the frame image are analyzed, and the second weight of each channel is determined. According to the first t-1 The response result of each single channel in the frame image determines the first weight of the channel. According to the first weight of the channel and channel second weight The final single-channel weights are obtained, and response fusion is performed using these single-channel weights to determine the [number of channels]. t The final feature response result of the frame image; Step 4: Determine the first [characteristic] based on the maximum value of the final characteristic response result. t The target location in the frame image is predicted. Based on the predicted target location, the ternary tree scaling acceleration method is used to determine the [missing information - likely a specific value]. t Target size of the frame image; Step 5: Extract basic samples by using the foreground and background and related information of the target's predicted location in the current frame, and process the samples using a step space constraint model; Step 6: Based on the second weight of the current frame channel obtained in Step 3. Determine the adaptive learning rate, and determine the first [unclear] using the learning rate. t+1 The search range for the frame image; Step 7: Train and update the scaling filter model, correlation filter model, and color histogram model; Step 8: Repeat steps 2 through 7 until the tracking ends.

2. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 1, characterized in that, The input first frame image is used to initialize the feature filter, scale filter, and color histogram model, specifically including: The target region of the first frame image has been manually labeled. Training samples are constructed by cyclically shifting the target region as the base sample. The ridge regression method is used to obtain the initial filter model through the training samples. , ,in a This represents the fundamental vector corresponding to the sample. b This represents the standard response corresponding to the sample. This represents the filter model parameters to be solved. Represents the regularization coefficient; Optimizing the filter model using kernel function method The optimized solution of the filter model is obtained as follows: In the formula, The representative sample is related to its own kernel in the calculation. I Represents the identity matrix; An initial color histogram model is obtained based on the target region as the base sample; An initial scale filter model is obtained based on the target region as the base sample.

3. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 1 or 2, characterized in that, The input number t For each frame image, the gradient histogram (HOG) features of the candidate target region are extracted, including the following specific steps: Convert the RGB image of the candidate region to a grayscale image; Perform Gamma correction on the image after grayscale conversion; Using [-1, 0, 1] as the horizontal gradient operator, [-1, 0, 1] T The vertical gradient operator is used to determine the horizontal and vertical gradient vectors of the corrected image. The magnitude of the gradient is determined based on the horizontal and vertical image gradient vectors. G ( x , y | with gradient direction θ ( x , y ); The gamma-corrected image is divided into small cell units. The gradient direction within each cell is divided into 9 parts from 0° to 180°. Each 20° direction is represented by a directional region. Z1 to Z9 represent the 9 directional regions respectively. The gradients of the 9 parts are weighted and statistically analyzed to obtain the gradient direction histogram of the cell, which is a feature vector of length 9. Several adjacent cells are merged into a block. The feature vectors of all cells in each block are concatenated to obtain the HOG feature of the current block. The gradient histogram within the block is normalized to obtain the normalized gradient magnitude. The HOG features from all blocks are integrated to obtain 36-dimensional HOG features.

4. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 3, characterized in that, The extraction of fused gradient orientation histogram (FHOG) features from candidate target regions using PCA for dimensionality reduction of HOG features specifically includes: Divide each 20° segment of the 360° into the same category, and determine each pixel point ( x , y The direction-insensitive feature corresponding to ) F 1. Direction-sensitive characteristics F 2; Define pixel-level feature mapping V Determine the gradient histogram for each pixel; Determine the feature vector of each cell based on the size of the image; The gradient energy of each cell is determined based on the eigenvector of each cell; Gradient energy of each cell E Normalize the feature vector to obtain a value of . W × H And the direction-sensitive feature with a depth of 18 and a size of W × H And the direction-insensitive feature at depth 9 and size is W × H Furthermore, texture features with a depth of 4 ultimately form a size of W × H And it has FHOG features with a depth of 31.

5. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 4, characterized in that, The extraction of color name (CN) features from the candidate target region at five layers specifically includes: Convert the candidate target region image from RGB space to color space; Feature dimensionality reduction is performed on candidate target region images converted to color space using the PCA method; The feature vector matrix corresponding to the five largest feature values ​​in the dimensionality-reduced candidate target region image is extracted to form a 5-dimensional CN feature.

6. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 5, characterized in that, The re-extraction of grayscale features of the candidate target region specifically includes: Convert the RGB image of the candidate target region to a grayscale image; The converted grayscale image is normalized to control the pixel values ​​within the range of [-0.5, 0.5], thus obtaining the final grayscale features.

7. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 6, characterized in that, According to the first t-1 The response result of each single channel in the frame image determines the first weight of the channel. Specifically, it includes: Channel First Weight For this single channel t-1 The maximum peak value of the corresponding response during frame image detection. In the formula, f d Indicates the features of the previous frame. h d Represented as the first t-1 The filter coefficients of the frame.

8. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 7, characterized in that, The determination of the first t The response results of each channel in the histogram of gradients (HOG), fused histogram of gradient directions (FHOG), color name (CN), and grayscale features of the frame image are analyzed, and the second weight of each channel is determined. Specifically, this includes: obtaining the first... t The feature response map of the frame is processed using a non-maximum suppression algorithm with a window size of 3 × 3 to obtain the value of the first peak. The value of the second peak ,according to Determine the second weight of the channel .

9. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 8, characterized in that, According to the first weight of the channel and channel second weight The final single-channel weights are obtained, and the responses are fused using these weights to determine the final response result. This process specifically includes: based on... Determine the single-channel weight In the formula, i Indicates that the channel number is the first i indivual, n Indicates the total number of feature channels; according to Determine the final response result r es .

10. The tracking method based on multi-scale spatial constraints to resist occlusion according to claim 9, characterized in that, Based on the maximum value in the final response result, the target prediction location is determined. Then, based on the target prediction location, the ternary tree scaling acceleration method is used to determine the [missing value]. t The target scale of the frame image specifically includes The final response result obtained r es The coordinates corresponding to the maximum value in ( x , y The predicted target position of the target in the current frame; Based on the predicted target location, the target scale size in the current frame is determined using a ternary tree scaling acceleration method.

Images