Object tracking method based on adaptive background perception and spatial attention mechanism

Abstract

The present invention discloses a target tracking method based on adaptive background perception and spatial attention mechanism, comprising the following steps: establishing a sample matrix corresponding to a video image sequence; preprocessing the image sample matrix through a spatial attention mechanism to establish a spatial feature enhanced image sample set; establishing an objective function of a filter model and training the filter model; optimizing the filter model using an adaptive spatial regularization strategy by applying adaptive weights to the filter; optimizing the filter model again using an adaptive background perception method and updating the objective function of the filter model; optimizing the objective function of the filter model and obtaining the position of the target in the current frame by solving the optimal filter parameters. The present invention improves the target discrimination capability of the tracking algorithm, so that the tracker can still cope with the target tracking requirements of complex scenes under the challenges of scale changes, complex backgrounds, changes in illumination, and motion blur.

Description

Technical Field

[0001] The present invention relates to the field of visual target tracking, and in particular to a target tracking method based on adaptive background perception and spatial attention mechanism. Background Art

[0002] Visual object tracking is one of the hot research areas in computer vision. Its main task is to obtain the position and scale of the target of interest in the previous frame of the video by designing tracking models and algorithms, using the feature information and relationships of the previous and next frames of the video, so as to predict the information in the subsequent frames and thus obtain the entire motion trajectory of the target.

[0003] With the rapid development of computer vision technology, target tracking algorithms have made significant progress and have been widely used in security inspection systems, traffic control, video surveillance and other application fields. Although target tracking research has made significant progress in both academia and industry, there are still many problems and challenges. Given that target tracking is a serialization problem, the target shape is clear in the initial frame, while the appearance of the target in subsequent frames is prone to change, which will bring time challenges to tracking, as well as challenges caused by environmental differences within a single frame. Therefore, real-world applications often face challenges from multiple unfavorable factors such as illumination changes, background changes, target occlusion, and motion blur. Therefore, designing a robust target tracker that can cope with complex challenges to handle tracking problems on lightweight portable devices such as gimbals and drones, and to improve the real-time and robustness requirements of visual tracking, is crucial to the future development of visual target tracking. Summary of the invention

[0004] Purpose of the invention: In view of the above problems, the purpose of the present invention is to provide a target tracking method based on adaptive background perception and spatial attention mechanism.

[0005] Technical solution: A target tracking method based on adaptive background perception and spatial attention mechanism of the present invention comprises the following steps:

[0006] Step 1: Collect video images, obtain the location and feature information of the tracked target, and establish the sample matrix A corresponding to the video image sequence i , i represents the number of video frames;

[0007] Step 2: Preprocess the image sample matrix A through the spatial attention mechanism i , adjust the attention weights of features at different positions of image samples, and establish a spatial feature enhanced image sample set Z i ;

[0008] Step 3: The feature enhanced image sample set Z obtained by preprocessing in step 2 i Cyclic sampling and establishment of the objective function of the filter model are performed to achieve target tracking by training the filter model;

[0009] Step 4, using the adaptive spatial regularization strategy to optimize the filter model, by applying adaptive weights to the filter, allocating feature resources of the sample target and background areas, and adjusting the dynamic relationship between the image sample and the ideal Gaussian template;

[0010] Step 5, the filter model is optimized again by an adaptive background perception method. By obtaining the background information of the image samples, the information in the background that is prone to tracking interference is screened to form a negative sample matrix. The filter is trained using this negative sample matrix, and the objective function solved by the filter model is updated.

[0011] Step 6, optimize the objective function of the filter model in step 5 through Fourier transform and alternating direction multiplier algorithm, and then update the tracking model, locate the target by solving the optimal filter parameters, and obtain the position of the target in the current frame.

[0012] Further, the step 2 comprises:

[0013] Step 201: transform the sample matrix A i Image sample x i Input to the spatial attention mechanism module, and calculate the sample x at any position a through the Gaussian function a and all other possible samples x at position b b The interaction relationship is expressed as:

[0014]

[0015] In the formula, x a T x b represents dot product similarity, T represents transposition operation; the h function value is proportional to the influence of the b position on the a position;

[0016] Step 202, calculate the image sample x again i The feature weight of each pixel in is expressed as:

[0017] g(x b )=Υ g x i

[0018] In the formula, Υ g represents the weight matrix;

[0019] Step 203: The spatial attention module performs spatial attention adjustment on each position of the sample matrix feature map to obtain a weighted sample feature Z a , the expression is:

[0020]

[0021] Where H represents the normalization parameter;

[0022] Use global average pooling to generate spatial information statistics, and transform the global spatial sample feature Z of each frame a Integrated into the spatial attention image sample matrix Z i .

[0023] Further, the step 3 comprises:

[0024] Step 301: The classical correlation filter tracking method collects sample matrix A based on the cyclic shift sampling. i , obtain the target circulant matrix A0, and complete the target tracking by implementing ridge regression. The objective function expression of the filter model is:

[0025]

[0026] Where y is the ideal Gaussian response matrix, i.e., the vectorized regression target of the image;

[0027] Step 302, using the spatial features obtained after preprocessing in step 2 to enhance the image sample Z i , and perform cyclic sampling on it. i The translation model around the target is obtained, and the circulant matrix Z0 is obtained. Then the objective function expression of the filter tracking model is:

[0028]

[0029] Further, the step 4 comprises:

[0030] Step 401, introduce the filter space prior information as a predefined spatial reference template, and the reference template is initially Gaussian shape distribution, establish the minimization relationship f(w)1 between the filter of the current frame image sample and the predefined spatial reference template, the expression is:

[0031]

[0032] Where λ1 and λ2 are regularization parameters; w is the correlation filter; w ′ is a predefined spatial reference template that changes with the target motion;

[0033] Step 402, using the adaptive spatial regularization strategy to act on the filter model, the objective function expression of the filter tracking model is:

[0034]

[0035] Further, the step 5 comprises:

[0036] Step 501, collect background interference information around the target in the image sample, obtain the filter response result matrix R related to the target sample in the current frame and the filter; select the area with higher filter response value around the target by adaptive background perception method, and record it as the local maximum value R of the response map l , then R l It can be calculated by the following formula:

[0037] R l =imregionalmax(R),l=1,2,3…

[0038] In the formula, l represents the position with higher response value in the current frame;

[0039] Step 502: Get the local maximum value R of the response graph l The three next largest values ​​R1, R2, and R3, which are second only to the maximum response value of the target center, are extended to each subsequent frame of the image sample to form a set of background interference samples, and the background perception negative sample Z is used. j express;

[0040] Step 503: Use background-aware negative samples to train the filter so that the filter has a higher response value at the target sample and a response value close to zero at the other background samples. The response value f(w)2 of the background-aware negative sample is calculated as:

[0041]

[0042] Where λ3 represents the regularization coefficient;

[0043] Step 504, after adding the adaptive background perception module, the filter model objective function is updated, and the filter tracking model objective function based on the adaptive background perception and spatial attention mechanism is summarized as follows:

[0044]

[0045] Step 505: Circulate the target sample matrix Z0 with the background-aware negative sample Z j Merge into a circulant matrix Q representing sample features, and transform the response matrix y into Keep the corresponding constraints of the filter on the target and background samples, so that the target sample is close to the ideal regression target y and the background perception negative sample is close to 0, then the objective function expression of the filter tracking model is modified as follows:

[0046]

[0047] In the formula,

[0048] Further, the step 6 comprises:

[0049] Step 601: transform the function F p (w, Q) is minimized in the Fourier domain, and the partial derivative of the correlation filter w is obtained, and the expression is:

[0050]

[0051] Where I represents the unit matrix;

[0052] Step 602, using the diagonalization property and reversibility of the circulant matrix, the circulant matrix is ​​expressed as a corresponding eigenvector, and the simplified filter expression is:

[0053]

[0054] Where F is the discrete Fourier matrix;

[0055] Step 603: Using the circulant matrix convolution theorem, the filter solution is converted into the Fourier domain to obtain the optimal filter solution. The expression is:

[0056]

[0057] Step 604, use the previous frame filter The correlation calculation is performed with the sample features of the current frame image to obtain the corresponding result map, and the corresponding position with the maximum response value is taken as the target tracking result of the current frame.

[0058] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0059] 1. The present invention improves the filter's ability to distinguish target features of image samples through a spatial attention mechanism, focusing on sample features that are beneficial to target tracking in spatial position, and then introduces a spatial regularization constraint strategy to dynamically update the filter according to the changes in the current frame filter and the predefined spatial constraint template, thereby enhancing the filter's ability to capture spatial information; an adaptive background perception method is proposed to filter background information that is easily interfered with filter training, thereby improving the tracking algorithm's ability to distinguish targets, so that the tracker can still cope with the target tracking needs of complex scenes under the challenges of scale changes, complex backgrounds, lighting changes, and motion blur;

[0060] 2. The present invention considers enhancing the spatial characteristics of samples, improving the sample quality and spatial representation ability of target samples, and provides a more reasonable background and space perception strategy, which effectively improves the robustness of the filter;

[0061] 3. The present invention improves the target tracking performance, reduces the risk of target drift in complex scenarios, and can maintain good real-time performance. It can be used for target tracking carried by drones and gimbals. In practical applications, it can provide support for intelligent video surveillance systems and promote the safe and efficient operation of intelligent prevention and control systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the process structure of the present invention;

[0063] Figure 2 Schematic diagram of the spatial attention mechanism module structure;

[0064] Figure 3 It is a schematic diagram of the structure of the adaptive background perception correlation filter model;

[0065] Figure 4 The accuracy and success rate of the method in this embodiment and other methods on the OTB100 dataset are as follows: (a) accuracy graph, (b) success rate graph;

[0066] Figure 5 The success rates of the method in this embodiment and other methods on 8 attribute sequences on OTB100: (a) scale change, (b) complex background, (c) illumination change, (d) deformation, (e) motion blur, (f) fast motion, (g) in-plane rotation, and (h) out-of-plane rotation;

[0067] Figure 6 The visual tracking results of the method of this embodiment and other methods on some sequences of the OTB100 dataset: (a) Bird1, (b) Couple, (c) DragonBaby, and (d) Matrix. DETAILED DESCRIPTION

[0068] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0069] like Figure 1 , a target tracking method based on adaptive background perception and spatial attention mechanism described in this embodiment includes the following steps:

[0070] Step 1: Collect video images, obtain the location and feature information of the tracked target, and establish the sample matrix A corresponding to the video image sequence i , i represents the number of video frames.

[0071] Video images are collected through portable devices to establish image sample sets for real-time visual tracking. The initialization target is framed in the first frame of the image sample set to obtain the position and characteristics of the tracked target and establish the sample matrix A corresponding to each frame of the video image sequence.i .

[0072] Step 2: Preprocess the image sample matrix A through the spatial attention mechanism i , adjust the attention weights of features at different positions of image samples, and establish a spatial feature enhanced image sample set Z i .

[0073] Step 201: transform the sample matrix A i Image sample x i Input to the spatial attention mechanism module, and calculate the sample x at any position a through the Gaussian function a and all remaining possible samples x at position b b The interaction relationship is expressed as:

[0074]

[0075] In the formula, x a T x b represents dot product similarity, T represents transpose operation; the h function value is proportional to the influence of b position on a position.

[0076] Step 202, calculate the image sample x again i The feature weight of each pixel in is expressed as:

[0077] g(x b )=Υ g x i

[0078] In the formula, Υ g represents the weight matrix;

[0079] Step 203: The spatial attention module performs spatial attention adjustment on each position of the sample matrix feature map to obtain a weighted sample feature Z a , the expression is:

[0080]

[0081] Where H represents the normalization parameter;

[0082] Use global average pooling to generate spatial information statistics, and transform the global spatial sample feature Z of each frame a Integrated into the spatial attention image sample matrix Z i .

[0083] Figure 2 In the example, i is 1. The image samples of the current frame are divided into 9 positions. The feature similarity between x1 and the samples at the other 8 positions is calculated, and then the feature weights at each position are calculated. The two parts are then operated to obtain the sample features.

[0084] Step 3: The feature enhanced image sample set Z obtained by preprocessing in step 2 i Cyclic sampling is performed, and the objective function of the filter model is established, and target tracking is achieved by training the filter model.

[0085] Step 301: The classical correlation filter tracking method collects sample matrix A based on the cyclic shift sampling. i , obtain the target circulant matrix A0, and complete the target tracking by implementing ridge regression. The objective function expression of the filter model is:

[0086]

[0087] Where y is the ideal Gaussian response matrix, i.e., the vectorized regression target of the image;

[0088] Step 302, using the spatial features obtained after preprocessing in step 2 to enhance the image sample Z i , and perform cyclic sampling on it. i The translation model around the target is obtained, and the circulant matrix Z0 is obtained. Then the objective function expression of the filter tracking model is:

[0089]

[0090] Step 4, using the adaptive spatial regularization strategy to optimize the filter model, by applying adaptive weights to the filter, allocating feature resources of the sample target and background areas, and adjusting the dynamic relationship between the image sample and the ideal Gaussian template;

[0091] Step 401, introduce the filter space prior information as a predefined spatial reference template, and the reference template is initially Gaussian shape distribution, establish the minimization relationship f(w)1 between the filter of the current frame image sample and the predefined spatial reference template, the expression is:

[0092]

[0093] Where λ1 and λ2 are regularization parameters; w is the correlation filter; w ′ is a predefined spatial reference template that changes with the target motion;

[0094] Step 402, using the adaptive spatial regularization strategy to act on the filter model, the objective function expression of the filter tracking model is:

[0095]

[0096] Step 5, the filter model is optimized again through the adaptive background perception method. By obtaining the background information of the image sample, the information in the background that is easy to cause tracking interference is screened to form a negative sample matrix. The filter is trained using this negative sample matrix, and the objective function solved by the filter model is updated, such as Figure 3 shown.

[0097] Step 501, collect background interference information around the target in the image sample, obtain the filter response result matrix R related to the target sample in the current frame and the filter; use the adaptive background perception method to filter the area with higher filter response value around the target, and record it as the local maximum value R of the response map l , then R l It can be calculated by the following formula:

[0098] R l =imregionalmax(R),l=1,2,3…

[0099] In the formula, l represents the position with higher response value in the current frame;

[0100] Step 502: Get the local maximum value R of the response graph l The three next largest values ​​R1, R2, and R3, which are second only to the maximum response value of the target center, are extended to each subsequent frame of the image sample to form a set of background interference samples, and the background perception negative sample Z is used. j express;

[0101] Step 503: Use background-aware negative samples to train the filter so that the filter has a higher response value at the target sample and the response values ​​at the other background samples are close to zero. The response value f(w)2 of the background-aware negative sample is calculated as:

[0102]

[0103] Where λ3 represents the regularization coefficient;

[0104] Step 504, after adding the adaptive background perception module, the filter model objective function is updated, and the filter tracking model objective function based on the adaptive background perception and spatial attention mechanism is summarized as follows:

[0105]

[0106]

[0107] Step 505: Circulate the target sample matrix Z0 with the background-aware negative sample Z j Merge into a circulant matrix Q representing sample features, and transform the response matrix y into Keep the corresponding constraints of the filter on the target and background samples, so that the target sample is close to the ideal regression target y and the background perception negative sample is close to 0, then the objective function expression of the filter tracking model is modified as follows:

[0108]

[0109] In the formula,

[0110] Step 6, optimize the objective function of the filter model in step 5 through Fourier transform and alternating direction multiplier algorithm, and then update the tracking model, locate the target by solving the optimal filter parameters, and obtain the position of the target in the current frame.

[0111] Step 601: transform the function F p (w, Q) is minimized in the Fourier domain, and the partial derivative of the correlation filter w is obtained, and the expression is:

[0112]

[0113] Where I represents the unit matrix;

[0114] Step 602, using the diagonalization property and reversibility of the circulant matrix, the circulant matrix is ​​expressed as a corresponding eigenvector, and the simplified filter expression is:

[0115]

[0116] Where F is the discrete Fourier matrix;

[0117] Step 603: Using the circulant matrix convolution theorem, the filter solution is converted into the Fourier domain to obtain the optimal filter solution. The expression is:

[0118]

[0119] Step 604, use the previous frame filter The correlation calculation is performed with the sample features of the current frame image to obtain the corresponding result map, and the corresponding position with the maximum response value is taken as the target tracking result of the current frame.

[0120] To verify the effect of the target tracking method described in this embodiment, this method was tested on the OTB100 benchmark dataset. In the experiment, the method of this embodiment was compared with 9 mainstream correlation filtering methods, including ASRCF, ECO, C-COT, TADT, DeepSRDC, BACF, SRDCF, STAPLE, and KCF.

[0121] The accuracy and success rate results of this embodiment and other methods on the OTB100 dataset are as follows: Figure 4 (a) and Figure 4 As shown in (b), the experimental results show that the target tracking method in this embodiment achieves a relatively excellent accuracy of 91.4% and achieves the best tracking success rate of 69.5%. Compared with the benchmark methods SRDCF and BACF, the accuracy is improved by 15.8% and 12% and the success rate is improved by 16.2% and 13% respectively.

[0122] The tracking success rates of this embodiment and other methods in different attribute scene sequences of the OTB100 dataset are as follows: Figure 5 (a) to Figure 5 As shown in (h), the experimental results show that the target tracking method in this embodiment achieves the highest success rate and can better cope with eight tracking challenges such as scale change, complex background, illumination change, and motion blur.

[0123] The visualization results of this embodiment and other methods on different attribute scene sequences of the OTB100 dataset are as follows: Figure 6 (a) to Figure 6 As shown in (d), the experimental results show that other mainstream tracking methods are more likely to experience loss and drift, or to return briefly after deviation, while the target tracking method in this embodiment can always maintain the sensitivity of capturing the target during the tracking process.

Claims

1. A target tracking method based on adaptive background perception and spatial attention mechanism, characterized in that: The steps include: Step 1: Collect video images, obtain the location and feature information of the tracked target, and establish the sample matrix A corresponding to the video image sequence i , i represents the number of video frames; Step 2: Preprocess the image sample matrix A through the spatial attention mechanism i , adjust the attention weights of features at different positions of image samples, and establish a spatial feature enhanced image sample set Z i ; Step 3: The feature enhanced image sample set Z obtained by preprocessing in step 2 i Cycle sampling, establish the objective function of the filter model, and achieve target tracking by training the filter model; the specific contents are as follows: Enhance the image sample Z using the spatial features obtained after preprocessing in step 2 i , and perform cyclic sampling on it. i The translation model around the target is obtained, and the circulant matrix Z0 is obtained. Then the objective function expression of the filter tracking model is: Where y is the ideal Gaussian response matrix, i.e., the vectorized regression target of the image; λ1 is the regularization parameter; w is the correlation filter; Step 4, using the adaptive spatial regularization strategy to optimize the filter model, by applying adaptive weights to the filter, allocating feature resources of the sample target and background areas, and adjusting the dynamic relationship between the image sample and the ideal Gaussian template; Step 5, the filter model is optimized again by an adaptive background perception method. By obtaining the background information of the image samples, the information in the background that is prone to tracking interference is screened to form a negative sample matrix. The filter is trained using this negative sample matrix, and the objective function solved by the filter model is updated. Step 6, optimize the objective function of the filter model in step 5 through Fourier transform and alternating direction multiplier algorithm, and then update the tracking model, locate the target by solving the optimal filter parameters, and obtain the position of the target in the current frame.

2. The target tracking method according to claim 1, characterized in that: The step 2 comprises: Step 201: transform the sample matrix A i Image sample x i Input to the spatial attention mechanism module, and calculate the sample x at any position a through the Gaussian function a and all other possible samples x at position b b The interaction relationship is expressed as: In the formula, x a T x b represents dot product similarity, T represents transposition operation; the h function value is proportional to the influence of the b position on the a position; Step 202, calculate the image sample x again i The feature weight of each pixel in is expressed as: g(x b )=Y g x i In the formula, Υ g represents the weight matrix; Step 203: The spatial attention module performs spatial attention adjustment on each position of the sample matrix feature map to obtain a weighted sample feature Z a , the expression is: Where H represents the normalization parameter; Use global average pooling to generate spatial information statistics, and transform the global spatial sample feature Z of each frame a Integrate into the spatial attention image sample matrix Z i .

3. The target tracking method according to claim 1, characterized in that: The step 4 comprises: Step 401, introduce the filter space prior information as a predefined spatial reference template, and the reference template is initially Gaussian shape distribution, establish the minimization relationship f(w)1 between the filter of the current frame image sample and the predefined spatial reference template, the expression is: Where λ1 and λ2 are regularization parameters; w is the correlation filter; w′ is a predefined spatial reference template that changes with the target motion; Step 402, using the adaptive spatial regularization strategy to act on the filter model, the objective function expression of the filter tracking model is:

4. The target tracking method according to claim 3, characterized in that: The step 5 comprises: Step 501, collect background interference information around the target in the image sample, obtain the filter response result matrix R related to the target sample in the current frame and the filter; use the adaptive background perception method to filter the area with higher filter response value around the target, and record it as the local maximum value R of the response map l , then R l It can be calculated by the following formula: R l =imregionalmax(R),l=1,2,3… In the formula, l represents the position with higher response value in the current frame; Step 502: Get the local maximum value R of the response graph l The three next largest values ​​R1, R2, and R3, which are second only to the maximum response value of the target center, are extended to each subsequent frame of the image sample to form a set of background interference samples, and the background perception negative sample Z is used. j express; Step 503: Use background-aware negative samples to train the filter so that the filter has a higher response value at the target sample and a response value close to zero at the other background samples. The response value f(w)2 of the background-aware negative sample is calculated as: Where λ3 represents the regularization coefficient; Step 504, after adding the adaptive background perception module, the filter model objective function is updated, and the filter tracking model objective function based on the adaptive background perception and spatial attention mechanism is summarized as follows: Step 505: Circulate the target sample matrix Z0 with the background-aware negative sample Z j Merge into a circulant matrix Q representing sample features, and transform the response matrix y into Keep the corresponding constraints of the filter on the target and background samples, so that the target sample is close to the ideal regression target y and the background perception negative sample is close to 0, then the objective function expression of the filter tracking model is modified as follows: In the formula, 5. The target tracking method according to claim 4, characterized in that: The step 6 comprises: Step 601: transform the function F p (w, Q) is minimized in the Fourier domain, and the partial derivative of the correlation filter w is obtained, and the expression is: Where I represents the unit matrix; Step 602, using the diagonalization property and reversibility of the circulant matrix, the circulant matrix is ​​expressed as a corresponding eigenvector, and the simplified filter expression is: Where F is the discrete Fourier matrix; Step 603: Using the circulant matrix convolution theorem, the filter solution is converted into the Fourier domain to obtain the optimal filter solution. The expression is: Step 604, use the previous frame filter The correlation calculation is performed with the sample features of the current frame image to obtain the corresponding result map, and the corresponding position with the maximum response value is taken as the target tracking result of the current frame.

Images