A Siamese network target tracking method based on multiple attention

Abstract

The present invention claims protection for a Siamese Multi-attention Network target tracking method based on multiple attentions (SiamMAN), which belongs to the field of computer vision technology. It mainly includes the following steps: first, in order to more efficiently utilize target feature information to deal with the problem of complex background interference, a multiple attention module is designed to optimize the features, wherein the channel attention branch is used to give higher weights to more discriminative channels, and the position attention branch makes full use of the target's position information; secondly, in order to better utilize shallow features, a feature fusion method is designed in the multiple attention module, which uses the residual learning method to fuse the shallow features and the features optimized by the attention branch, and then fuses the two attention features to further enhance the feature representation; finally, the Focal-EIoU loss is used as the regression loss function to guide the tracker to generate a more accurate tracking frame.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular to a visual target tracking method. Background Art

[0002] Video object tracking is a key research topic in computer vision. Given the object's size and position in the initial frame of a video sequence, it is necessary to predict the object's size and position in subsequent frames. In recent years, with the continuous development of object tracking algorithms, the theory of object tracking has become increasingly sophisticated. Object tracking technology is now widely used in video surveillance, pedestrian tracking, intelligent transportation, and modern military applications. However, in complex and ever-changing real-world scenarios, moving objects are often affected by various factors such as occlusion, scale changes, background clutter, and illumination variations. Accurately tracking any object remains a challenging task. Therefore, the research on object tracking technology remains of great research and practical value.

[0003] At present, target tracking methods can be roughly divided into two categories: one is based on correlation filtering, and the other is based on deep learning. The target tracking method based on correlation filtering mainly locates the target by designing a filter template to perform correlation operations with the target candidate area, and finding the maximum value in the response map, which has the advantage of fast speed. However, due to the limitations of the cosine window and the search area, this method will cause the filter template to learn too little information, which makes it easy to cause tracking drift in scenes with large-scale deformation and complex background interference. The target tracking method based on deep learning can provide more discriminative features, making the tracker more robust. Among them, the target tracking algorithm based on the Siamese network transforms the tracking problem into a similarity matching problem, and uses an end-to-end training method to simplify the tracking problem, thereby achieving higher tracking accuracy.

[0004] However, the following problems still exist in the target tracking method based on the twin network: 1) In complex and changeable real-life scenarios, complex background interference is a very difficult problem in the tracking field, and the existing trackers for complex background challenges mostly use deep backbone networks to extract features. Although more features are extracted, a large number of parameters are introduced, which increases the computational overhead and leads to slow tracking; 2) In addition, other trackers for complex background challenges enhance the network's anti-interference ability by increasing training data, but this increases the time cost, increases training time and does not improve the tracking effect much. In order to alleviate the above shortcomings, the present invention proposes a twin network target tracking method based on multiple attentions for the complex background challenges in real-life scenarios to improve tracking performance.

[0005] CN111192292A, a target tracking method and related equipment based on attention mechanism and twin network, by inputting the target template and search area into a preset target tracking model, and outputting the target tracking information of the target template in the search area through the preset target tracking model; the target tracking model includes a twin network, and a channel attention module and / or a spatial attention module is added to the twin network. The target tracking method and related equipment provided in this embodiment, due to the addition of a channel attention module and / or a spatial attention module to the residual network, significantly improve the average expected overlap rate and robustness of the twin tracking algorithm, and improve the robustness of motion changes, camera motion, occlusion, and size change attributes in tracking. Therefore, when using the target tracking method provided in this embodiment for target tracking prediction, more accurate results can be obtained.

[0006] The invention with publication number CN111192292A uses SiamRPN++ as the base network, adopts ResNet-50 as the feature extraction network, and adds channel attention and / or spatial attention modules to the twin network. The bounding box regression and classification branches are used to obtain the bounding box regression results and feature classification results. Although both are target tracking methods based on twin networks and both use the attention mechanism to improve tracking performance, the present invention and the invention with publication number CN111192292A differ in the following aspects:

[0007] (1) Feature Extraction Network Selection: The invention with publication number CN111192292A uses the deep ResNet-50 as the feature extraction network, while the present invention uses the relatively shallow GoogLeNet as the feature extraction network. Therefore, the present invention has fewer parameters and faster tracking speed.

[0008] (2) Position of the attention module: The invention with publication number CN111192292A first adds a channel attention module and then a spatial attention module to each residual learning unit of the residual network, while the present invention designs a multiple attention module, which consists of a channel attention branch and a position attention branch. The features extracted by the feature extraction network will enter the two attention branches in parallel and then be fused. In this way, the channel information and position information can be fully utilized, the interference of redundant information can be reduced, and the tracking performance can be improved.

[0009] (3) Target prediction method: The invention with publication number CN111192292A adopts the Anchor-based method to predict the location of the target, while the present invention adopts the Anchor-free method, which distinguishes the foreground and background through the classification branch, and calculates the distance between each point and the target center point through the center branch to suppress the situation where some predicted points are far from the center point, and then obtains the tracking frame through the regression branch. The present invention uses the Focal-EIoU loss as the regression loss function to guide the tracker to generate a more accurate tracking frame. Summary of the Invention

[0010] The present invention aims to solve the above problems of the prior art. A target tracking method based on a twin network with multiple attentions is proposed. The technical solution of the present invention is as follows:

[0011] A Siamese network target tracking method based on multiple attentions includes the following steps:

[0012] Step 1: Select the first frame of the video as the template image, select the subsequent frames of the video as the search images, and then perform preprocessing operations on the template image and the search image respectively;

[0013] Step 2: Input the preprocessed template image and search image into the template branch and search branch of the twin network respectively, perform feature extraction through the GoogLeNet feature extraction backbone network, and obtain the feature map of the template image and the feature map of the search image;

[0014] Step 3: The template image features and search image features are input into a multi-attention module consisting of a parallel channel attention branch and a position attention branch. The channel attention branch assigns higher weights to more discriminative feature channels, and the position attention branch makes full use of the target's position information to further refine the features. The target information is propagated from the template image features to the search image features through the graph attention mechanism, resulting in a feature response graph.

[0015] Step 4: Input the feature response map into the classification-regression subnetwork. The regression branch replaces the IoU loss by introducing the Focal-EIoU loss, thereby guiding the tracker to generate a more accurate tracking frame. The resulting tracking frame is the final tracking result.

[0016] Furthermore, the step 1: selecting the first frame image of the video as the template image, selecting the subsequent frame images of the video as the search image, and then performing preprocessing operations on the template image and the search image respectively, specifically includes the following steps:

[0017] A1. Template image preprocessing: Select the first frame of the video and calibrate the target area using a rectangular box. The center of the rectangular box represents the target center. Expand the four sides of the target rectangular box by p pixels. If the rectangular box exceeds the image boundary, the excess part is filled with the image pixel mean. Finally, scale the cropped target image to 127×127 pixels.

[0018] A2. Search image preprocessing: Select subsequent frames of the video and demarcate the target area using a rectangular box. The center of the rectangular box represents the target center. Expand the four sides of the target rectangular box by p pixels. If the rectangular box exceeds the image boundary, the excess part is filled with the image pixel mean. Finally, scale the cropped target image to 287×287 pixels.

[0019] Furthermore, the step 2: inputting the preprocessed template image and search image into the template branch and search branch of the twin network respectively, performing feature extraction through the GoogLeNet feature extraction backbone network to obtain the features of the template image and the features of the search image, specifically includes the following steps:

[0020] B1. Obtain a 127×127 template image z according to step A1, and obtain a 287×287 search image x according to step A2.

[0021] B2. Input the template image z into the template branch of the twin network and extract the template image features through the GoogLeNet (Inception v3) feature extraction backbone network

[0022] B3. Input the search image x into the search branch of the twin network and extract the search image features through the GoogLeNet (Inception v3) feature extraction backbone network

[0023] Furthermore, step 3: inputting the features of the template image and the search image into a multi-attention module consisting of a parallel channel attention branch and a position attention branch, wherein the channel attention branch assigns higher weights to feature channels with more discriminative properties, and the position attention branch makes full use of the position information of the target to further refine the features; propagating the target information from the template image features to the search image features through the graph attention mechanism to obtain a feature response graph, specifically includes the following steps:

[0024] C1, template image features obtained according to step B2 It is input into the multi-attention module, which mainly includes the channel attention branch and the position attention branch, to obtain two different attention features. The two attention features are fused by pixel-by-pixel multiplication operation, and then the feature representation is further enhanced by two 3×3 convolution operations to finally obtain the refined template image features.

[0025] C2, search image features obtained according to step B3 It is input into the multi-attention module, which mainly includes the channel attention branch and the position attention branch, to obtain two different attention features. The two attention features are fused by pixel-by-pixel multiplication operation, and then the feature representation is further enhanced by two 3×3 convolution operations to finally obtain the refined search image features.

[0026] C3, the refined template image features and search image features The target information is propagated from the template image features to the search image features through the graph attention mechanism to obtain a feature response graph F fin .

[0027] Furthermore, in step C3, the refined template image features and search image features The target information is propagated from the template image features to the search image features through the graph attention mechanism. The specific steps are:

[0028] D1. Template image features Each 1×1×C grid in is considered as a node, where C is the number of feature channels, and a node set V containing all nodes is obtained. z ;

[0029] D2. Search image features Each 1×1×C grid in is considered as a node, where C is the number of feature channels, and a node set V containing all nodes is obtained. x ;

[0030] D3. Construct a complete bipartite graph G = (V, E), where V = V z ∪V x , The two subgraphs of G are and

[0031] D4. The more similar a location in the search image is to a local location in the template image, the more likely it is to be the foreground, and more target information should be transferred there. First, linearly transform each node separately and calculate the inner product of the two to calculate the correlation score. The correlation score is the degree of similarity between the two nodes. Finally, a response graph is generated. The formula is:

[0032]

[0033] Among them, e i,j Represents node i∈V z and node j∈V z The correlation score between x and W z is the linear transformation matrix, and are the eigenvectors of nodes i and j respectively.

[0034] Furthermore, the multiple attention module comprises the following specific steps:

[0035] E1, input the unoptimized feature F into the channel attention branch, give higher weights to more discriminative channels, and then use residual learning to perform pixel multiplication operation with the unoptimized feature to obtain feature F1. The formula is:

[0036]

[0037] Among them, F1 represents the feature after optimization by the channel attention branch in the multi-attention module, and F represents the feature before optimization. represents pixel multiplication operation, σ represents Sigmoid activation function, F SENet Represents the features obtained after the channel attention mechanism;

[0038] E2, input the unoptimized feature F into the position attention branch, and then use residual learning to perform pixel multiplication operation with the unoptimized feature to obtain the feature F2. The formula is:

[0039]

[0040] Among them, F2 represents the feature after optimization by the position attention branch in the multi-attention module, and F represents the feature before optimization. represents pixel multiplication operation, σ represents Sigmoid activation function, F CA Represents the features obtained after the position attention mechanism;

[0041] E3, feature fusion is performed on the feature F1 obtained by step E1 and the feature F2 obtained by step E2. First, the two features are pixel-multiplied, and then two 3×3 convolution operations are performed to further enhance the feature representation to obtain the optimized feature F output , the formula is:

[0042]

[0043] Among them, F output represents the optimized features, F 3×3 represents a 3×3 convolution operation, Represents a pixel multiplication operation.

[0044] Furthermore, step 4 inputs the feature response map into the classification-regression subnetwork. The regression branch replaces the IoU loss by introducing the Focal-EIoU loss, thereby guiding the tracker to generate a more accurate tracking frame. The resulting tracking frame is the final tracking result. Specifically, the following steps are included:

[0045] F1. According to step C3, the obtained feature response map is convolved and input into the classification branch, centrality branch and regression branch respectively;

[0046] F2, the classification branch uses conventional cross entropy loss for classification tasks, and obtains the classification loss L cls ;

[0047] F3, the centrality branch is parallel to the classification branch and is used to remove abnormal data to obtain the centrality loss L cen ;

[0048] F4, the regression branch uses Focal-EIoU loss for regression tasks, and obtains the regression loss L reg ;

[0049] F5. Obtain classification loss L according to step F2. cls , according to step F3, we get the center loss L cen , according to step F4, we get the regression loss L reg , the formula for calculating the final total loss function is:

[0050] L=L cls +λ1L cen +λ2L reg (5)

[0051] Among them, L represents the total loss function, λ1 represents the hyperparameter in the centrality loss function, and λ2 represents the hyperparameter in the regression loss function.

[0052] Furthermore, the Focal-EIoU loss and the loss function of the regression branch are:

[0053]

[0054] Among them, L EIOU represents EIoU loss, IOU represents the intersection-over-union ratio of two anchor boxes, b represents the center point of the anchor box, and b gt represents the center point of the true value box, w represents the width of the anchor box, w gt represents the width of the true value box, h represents the height of the anchor box, h gt Indicates the height of the true value box, w c Indicates the width of the minimum bounding box, h c Indicates the height of the minimum bounding box;

[0055] L reg =IoU γ L EIoU (7)

[0056] Among them, L reg represents the regression loss calculated by Focal-EIoU loss, and γ is a hyperparameter.

[0057] The advantages and beneficial effects of the present invention are as follows:

[0058] 1. This paper addresses the complex background interference commonly encountered in target tracking by designing a Siamese network target tracking method based on multiple attentions. This method improves tracking performance at the feature level by designing a multiple attention module. Compared with the most advanced trackers (SiamFC++, SiamCAR, and SiamGAT), this method demonstrates superior tracking performance on common target tracking datasets.

[0059] 2. Feature optimization networks are a method for improving the tracking performance of trackers. Therefore, this paper designs a multi-attention module, including a channel attention branch and a position attention branch, which strengthens the network's learning ability for feature selection and reduces the burden of redundant information on the network. The channel attention branch assigns higher weights to more discriminative feature channels, while the position attention branch fully utilizes the target's position information. A feature fusion module is then designed to fuse these two different features, further enhancing the feature representation and obtaining a more robust feature representation, thereby effectively improving the tracking accuracy of the tracker.

[0060] 3. The target tracking task includes classification and regression branches. The classification branch is used to determine positive and negative samples, and the regression branch is used to determine the bounding box of the target. Currently, most target tracking methods often use IoU loss as the regression loss function. However, when the two tracking frames do not intersect, the use of IoU loss cannot well reflect the distance between the two frames. The EIoU paper proposes a Focal-EIoU loss, which calculates the aspect ratio of the tracking frame. Therefore, the present invention introduces the Focal-EIoU loss on the regression branch to guide the tracker to generate a more accurate regression frame, thereby further improving the tracking performance of the tracker. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a general framework diagram of a target tracking method based on a multiple attention twin network according to a preferred embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the multiple attention module designed in the present invention;

[0063] Figure 3 This is a diagram of the tracking effect of the present invention on the MotorRolling, Board and Soccer video sequences in the OTB100 dataset. DETAILED DESCRIPTION

[0064] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0065] The technical solution of the present invention to solve the above technical problems is:

[0066] The present invention is based on the SiamGAT target tracking framework, as detailed in Dongyan Guo, Yanyan Shao, Ying Cui, Zhenhua Wang, Liyan Zhang, and Chunhua Shen. Graph attention tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 9543-9552, 2021. First, a tracking framework is built using SiamGAT. A multi-attention module is then designed to optimize features extracted by the backbone network. Finally, the Focal-EIoU loss is introduced to guide the tracker to obtain a more accurate regression frame, thereby improving tracking accuracy.

[0067] As attached Figure 1As shown in the figure, a Siamese network target tracking method based on multiple attentions includes the following steps:

[0068] 1. As attached Figure 1 As shown, the first frame image of the video is selected as the template image, and the subsequent frame images of the video are selected as the search images. Then, the template image and the search image are preprocessed, specifically including:

[0069] 1.1 Template image preprocessing: Select the first frame of the video, calibrate the target area with a rectangular box, and expand the four sides of the target rectangular box by p pixels. If the rectangular box exceeds the image boundary, the excess part is filled with the image pixel mean. Finally, scale the cropped target image to 127×127 pixels.

[0070] 1.2 Search image preprocessing: Select the subsequent frames of the video, calibrate the target area with a rectangular box, and expand the four sides of the target rectangular box by p pixels. If the rectangular box exceeds the image boundary, the excess part is filled with the image pixel mean. Finally, scale the cropped target image to 287×287 pixels.

[0071] 2. As attached Figure 1 As shown in the figure, the preprocessed template image and search image are input into the template branch and search branch of the twin network respectively. The GoogLeNet feature extraction backbone network is used to extract features, and the features of the template image and the search image are obtained, including:

[0072] 2.1 According to step 1.1, obtain a 127×127 template image z, and according to step 1.2, obtain a 287×287 search image x;

[0073] 2.2 Input the template image z into the template branch of the twin network, and extract the template image features through the GoogLeNet (Inception v3) feature extraction backbone network

[0074] 2.3 Input the search image x into the search branch of the twin network, and extract the search image features through the GoogLeNet (Inception v3) feature extraction backbone network

[0075] 3. As attached Figure 1As shown in the figure, the features of the template image and the search image are input into a multi-attention module consisting of a parallel channel attention branch and a position attention branch. The channel attention branch gives higher weights to feature channels with more discriminative properties, and the position attention branch makes full use of the target's position information to further refine the features. The target information is propagated from the template image features to the search image features through the graph attention mechanism, and a feature response graph is obtained, which specifically includes:

[0076] 3.1 Template image features obtained according to step 2.2 It is input into the multi-attention module, which mainly includes the channel attention branch and the position attention branch, to obtain two different attention features. The two attention features are fused by pixel-by-pixel multiplication operation, and then the feature representation is further enhanced by two 3×3 convolution operations to finally obtain the refined template image features.

[0077] 3.2 Search image features obtained according to step 2.3 It is input into the multi-attention module, which mainly includes the channel attention branch and the position attention branch, to obtain two different attention features. The two attention features are fused by pixel-by-pixel multiplication operation, and then the feature representation is further enhanced by two 3×3 convolution operations to finally obtain the refined search image features.

[0078] 3.3 The refined template image features and search image features The target information is propagated from the template image features to the search image features through the graph attention mechanism to obtain a feature response graph F fin .

[0079] 4. The refined template image features and search image features The target information is propagated from the template image features to the search image features through the graph attention mechanism. The specific steps are:

[0080] 4.1 Template Image Features Each 1×1×C grid in is considered as a node, where C is the number of feature channels, and a node set V containing all nodes is obtained. z ;

[0081] 4.2 Searching for Image Features Each 1×1×C grid in is considered as a node, where C is the number of feature channels, and a node set V containing all nodes is obtained. x ;

[0082] 4.3 Construct a complete bipartite graph G = (V, E), where V = V z ∪V x , The two subgraphs of G are and

[0083] 4.4 Because the more similar a location in the search image is to a local location in the template image, the more likely it is to be the foreground, and more target information should be transferred there. First, linearly transform each node separately and calculate the inner product of the two to calculate the correlation score. The correlation score is the degree of similarity between the two nodes. Finally, a response graph is generated. The formula is:

[0084]

[0085] Among them, e i,j Represents node i∈V x and node j∈V z The correlation score between x and W z is the linear transformation matrix, and are the eigenvectors of nodes i and j respectively.

[0086] 5. As attached Figure 2 As shown in Figure 2, the multi-attention module contains a channel attention branch and a position attention branch, specifically including:

[0087] 5.1 Input the unoptimized feature F into the channel attention branch, assign higher weights to more discriminative channels, and then use residual learning to perform pixel multiplication with the unoptimized feature to avoid redundant information and suppress background noise, thereby obtaining the feature F1. The formula is:

[0088]

[0089] Among them, F1 represents the feature after optimization by the channel attention branch in the multi-attention module, and F represents the feature before optimization. represents pixel multiplication operation, σ represents Sigmoid activation function, F SENet Represents the features obtained after the channel attention mechanism.

[0090] 5.2 Input the unoptimized feature F into the position attention branch, make full use of the target's position information, and then use residual learning to perform pixel multiplication with the unoptimized feature to avoid redundant information and suppress background noise, thereby obtaining the feature F2. The formula is:

[0091]

[0092] Among them, F2 represents the feature after optimization by the position attention branch in the multi-attention module, and F represents the feature before optimization. represents pixel multiplication operation, σ represents Sigmoid activation function, F CA Represents the features obtained after the position attention mechanism.

[0093] 5.3 The feature F1 obtained by step 5.1 and the feature F2 obtained by step 5.2 are fused. First, the two features are multiplied pixel by pixel, and then two 3×3 convolution operations are performed to further enhance the feature representation to obtain the optimized feature F output , the formula is:

[0094]

[0095] Among them, F output represents the optimized features, F 3×3 represents a 3×3 convolution operation, Represents a pixel multiplication operation.

[0096] 6. As attached Figure 1 As shown in the figure, the feature response map is input into the classification-regression subnetwork. The regression branch replaces the IoU loss by introducing the Focal-EIoU loss, thereby guiding the tracker to generate a more accurate tracking frame. The resulting tracking frame is the final tracking result, which specifically includes:

[0097] 6.1 According to step 3.3, the obtained feature response map is convolved and input into the classification branch, centrality branch and regression branch respectively;

[0098] 6.2 The classification branch uses conventional cross entropy loss for classification tasks, and obtains the classification loss L cls ;

[0099] 6.3 The center branch is parallel to the classification branch to remove abnormal data and obtain the center loss L cen ;

[0100] 6.4 Regression branch uses Focal-EIoU loss for regression task and obtains regression loss L reg , the regression loss is calculated as:

[0101]

[0102] L reg =IoU γ L EIoU (6)

[0103] Among them, L EIOUrepresents EIoU loss, IOU represents the intersection-over-union ratio of two anchor boxes, b represents the center point of the anchor box, and b gt represents the center point of the true value box, w represents the width of the anchor box, w gt represents the width of the true value box, h represents the height of the anchor box, h gt Indicates the height of the true value box, w c Indicates the width of the minimum bounding box, h c Indicates the height of the smallest bounding box, L reg represents the regression loss calculated by Focal-EIoU loss, and γ is a hyperparameter.

[0104] 6.5 Obtain the classification loss L according to step 6.2 cls , according to step 6.3, we get the center loss L cen , according to step 6.4, we get the regression loss L reg , the formula for calculating the final total loss function is:

[0105] L=L cls +λ1L cen +λ2L reg (7)

[0106] Among them, L represents the total loss function, λ1 represents the hyperparameter in the centrality loss function, and λ2 represents the hyperparameter in the regression loss function.

[0107] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0108] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0109] The above embodiments should be understood as merely illustrating the present invention and not as limiting the scope of protection of the present invention. After reading the contents of the present invention, technicians may make various changes or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A target tracking method based on a Siamese network with multiple attentions, characterized in that: The following steps are involved: Step 1: Select the first frame of the video as the template image, select the subsequent frames of the video as the search images, and then perform preprocessing operations on the template image and the search image respectively; Step 2: Input the preprocessed template image and search image into the template branch and search branch of the twin network respectively, perform feature extraction through the GoogLeNet feature extraction backbone network, and obtain the feature map of the template image and the feature map of the search image; Step 3: The template image features and search image features are input into a multi-attention module consisting of a parallel channel attention branch and a position attention branch. The channel attention branch gives higher weights to more discriminative feature channels, and the position attention branch makes full use of the target's position information to further refine the features. The target information is propagated from the template image features to the search image features through the graph attention mechanism to obtain a feature response graph; Step 4: Input the feature response map into the classification-regression subnetwork. The regression branch replaces the IoU loss by introducing the Focal-EIoU loss, thereby guiding the tracker to generate a more accurate tracking frame. The resulting tracking frame is the final tracking result. The multiple attention module specifically comprises the following steps: E1, input the unoptimized feature F into the channel attention branch, give higher weights to more discriminative channels, and then use residual learning to perform pixel multiplication operation with the unoptimized feature to obtain feature F1. The formula is: Among them, F1 represents the feature after optimization by the channel attention branch in the multi-attention module, and F represents the feature before optimization. represents pixel multiplication operation, σ represents Sigmoid activation function, F SENet Represents the features obtained after the channel attention mechanism; E2, input the unoptimized feature F into the position attention branch, and then use residual learning to perform pixel multiplication operation with the unoptimized feature to obtain the feature F2. The formula is: Among them, F2 represents the feature after optimization by the position attention branch in the multi-attention module, and F represents the feature before optimization. represents pixel multiplication operation, σ represents Sigmoid activation function, F CA Represents the features obtained after the position attention mechanism; E3, feature fusion is performed on the feature F1 obtained by step E1 and the feature F2 obtained by step E2. First, the two features are pixel-multiplied, and then two 3×3 convolution operations are performed to further enhance the feature representation to obtain the optimized feature F output , the formula is: Among them, F output represents the optimized features, F 3×3 represents a 3×3 convolution operation, Represents a pixel multiplication operation.

2. A Siamese network target tracking method based on multiple attentions according to claim 1, characterized in that: The step 1: selecting the first frame image of the video as the template image, selecting the subsequent frame images of the video as the search images, and then performing preprocessing operations on the template image and the search image respectively, specifically includes the following steps: A1. Template image preprocessing: Select the first frame of the video and calibrate the target area using a rectangular box. The center of the rectangular box represents the target center. Expand the four sides of the target rectangular box by p pixels. If the rectangular box exceeds the image boundary, the excess part is filled with the image pixel mean. Finally, scale the cropped target image to 127×127 pixels. A2. Search image preprocessing: Select subsequent frames of the video and demarcate the target area using a rectangular box. The center of the rectangular box represents the target center. Expand the four sides of the target rectangular box by p pixels. If the rectangular box exceeds the image boundary, the excess part is filled with the image pixel mean. Finally, scale the cropped target image to 287×287 pixels.

3. A target tracking method based on a Siamese network with multiple attentions according to claim 2, characterized in that: Step 2: Input the preprocessed template image and search image into the template branch and search branch of the twin network respectively, perform feature extraction through the GoogLeNet feature extraction backbone network, and obtain the features of the template image and the features of the search image, specifically including the following steps: B1. Obtain a 127×127 template image z according to step A1, and obtain a 287×287 search image x according to step A2. B2. Input the template image z into the template branch of the twin network and extract the template image features through the GoogLeNet feature extraction backbone network B3. Input the search image x into the search branch of the twin network and extract the search image features through the GoogLeNet feature extraction backbone network 4. A Siamese network target tracking method based on multiple attentions according to claim 3, characterized in that: Step 3: Inputting the features of the template image and the search image into a multi-attention module composed of a parallel channel attention branch and a position attention branch, wherein the channel attention branch assigns higher weights to more discriminative feature channels, and the position attention branch makes full use of the target's position information to further refine the features; The target information is propagated from the template image features to the search image features through the graph attention mechanism to obtain a feature response graph, which specifically includes the following steps: C1, template image features obtained according to step B2 It is input into the multi-attention module, which includes the channel attention branch and the position attention branch, to obtain two different attention features. The two attention features are fused by pixel-by-pixel multiplication operation, and then the feature representation is further enhanced by two 3×3 convolution operations to finally obtain the refined template image features. C2, search image features obtained according to step B3 It is input into the multi-attention module, which includes the channel attention branch and the position attention branch, to obtain two different attention features. The two attention features are fused by pixel-by-pixel multiplication operation, and then the feature representation is further enhanced by two 3×3 convolution operations to finally obtain the refined search image features. C3, the refined template image features and search image features The target information is propagated from the template image features to the search image features through the graph attention mechanism to obtain a feature response graph F fin .

5. A Siamese network target tracking method based on multiple attentions according to claim 4, characterized in that: In step C3, the refined template image features and search image features The target information is propagated from the template image features to the search image features through the graph attention mechanism. The specific steps are: D1. Template image features Each 1×1×C grid in is considered as a node, where C is the number of feature channels, and a node set V containing all nodes is obtained. z ; D2. Search image features Each 1×1×C grid in is considered as a node, where C is the number of feature channels, and a node set V containing all nodes is obtained. x ; D3. Construct a complete bipartite graph G = (V, E), where V = V z ∪V x , The two subgraphs of G are and D4. The more similar the position in the search image is to the local position of the template image, the more likely it is to be the foreground, and more target information should be transferred there. First, linearly transform the nodes separately and calculate the inner product of the two to calculate the correlation score. The correlation score is the similarity between the two nodes. Finally, a response graph is generated. The formula is: Among them, e i,j Represents node i∈V x and node j∈V z The correlation score between x and W z is the linear transformation matrix, and are the eigenvectors of nodes \ and j respectively.

6. A Siamese network target tracking method based on multiple attentions according to claim 1, characterized in that: Step 4 inputs the feature response map into the classification-regression subnetwork. The regression branch replaces the IoU loss by introducing the Focal-EIoU loss, thereby guiding the tracker to generate a more accurate tracking frame. The resulting tracking frame is the final tracking result. Specifically, the following steps are included: F1. According to step C3, the obtained feature response map is convolved and input into the classification branch, centrality branch and regression branch respectively; F2, the classification branch uses conventional cross entropy loss for classification tasks, and obtains the classification loss L cls ; F3, the centrality branch is parallel to the classification branch and is used to remove abnormal data to obtain the centrality loss L cen ; F4, the regression branch uses Focal-EIoU loss for regression tasks, and obtains the regression loss L reg ; F5. Obtain classification loss L according to step F2. cls , according to step F3, we get the center loss L cen , according to step F4, we get the regression loss L reg , the formula for calculating the final total loss function is: L=L cls +λ1L cen +λ2L reg (5) Among them, L represents the total loss function, λ1 represents the hyperparameter in the centrality loss function, and λ2 represents the hyperparameter in the regression loss function.

7. A Siamese network target tracking method based on multiple attentions according to claim 6, characterized in that: The Focal-EIoU loss and the loss function of the regression branch are: Among them, L EIOU represents EIoU loss, IOU represents the intersection-over-union ratio of two anchor boxes, b represents the center point of the anchor box, and b gt represents the center point of the true value box, w represents the width of the anchor box, w gt represents the width of the true value box, h represents the height of the anchor box, h gt Indicates the height of the true value box, w c Indicates the width of the minimum bounding box, h c Indicates the height of the minimum bounding box; L reg =IoU γ L EIoU (7) Among them, L reg represents the regression loss calculated by Focal-EIoU loss, and γ is a hyperparameter.

Images