Target tracking method based on double attention mechanism

Abstract

The application discloses a target tracking method based on a double attention mechanism, comprising the following steps: constructing a backbone network model, wherein the backbone network model comprises a detection branch and a template branch; for a plurality of sets of template frames and search frames, the template branch is used to extract deep features of the template frames, and the detection branch is used to extract deep features of the search frames; the deep features extracted in step 1 are selectively enhanced through a deformable attention mechanism; the features enhanced in step 2 are subjected to weight redistribution and hierarchical cross fusion through sparse attention, and finally, fused features are obtained; the fused features obtained in step 3 are subjected to position and scale prediction of a target through a multilayer perception machine; and the multilayer perception machine outputs target position information of each detection frame, thereby completing tracking. The application can reduce attention weight on a background, increase attention weight on potential targets in a search area, and enhance robustness of the network, thereby finally overcoming interference of the background on a tracker.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a target tracking method based on a double attention mechanism. BACKGROUND

[0002] Target tracking is an extremely important part of computer vision, and its main task is to locate the target in the subsequent frames according to the target position given in the initial frame in a given continuous video sequence. Target tracking needs to solve two main problems: 1. mining features related to the object of interest; 2. establishing the correspondence between frames and frames. Research on target tracking is of great help to the fields of video analysis, scene understanding, etc. Target tracking has great application value in real life, such as intelligent monitoring, human-computer interaction, automatic driving and medical diagnosis, etc.

[0003] The tracker based on the twin network has great advantages in tracking accuracy, and since they are usually high-speed end-to-end networks, they convert the target tracking problem into a matching problem of the template frame and the search frame. SiamRPN draws on the successful region candidate network in the target detection field, and divides the target tracking task into two sub-tasks of classification and regression. SiamRPN++ uses ResNet50 as a feature extraction network and a deep cross-correlation operation to solve the problem of shallow twin network and center preference. The tracker based on Transformer is popular due to the excellent modeling ability of the attention mechanism between features and the global. TransT uses self-attention to enhance the features of the template frame and the search frame, and cross-attention to model the spatial dependency of the template frame and the search frame. Stark introduces spatiotemporal information into the tracker based on Transformer.

[0004] During the tracking process, the target is disturbed by the background, the tracker occupies a large proportion of secondary information (background), and is easily distracted by the background, which to some extent weakens the target and makes the distinction between foreground and background blurred, that is, the tracker lacks attention to the most relevant information in the search area, thereby reducing the tracking accuracy. Although the existing deep trackers based on twin networks and based on Transformer have good performance in tracking accuracy, we notice that most trackers still cannot solve the tracking problem of the target in the complex scene disturbed by the background. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide a target tracking method based on a double attention mechanism, which can reduce the attention weight to the background, increase the attention weight to the potential target in the search area, and enhance the robustness of the network, and finally overcome the interference of the background to the tracker.

[0006] To solve the above technical problems, the present application adopts the following technical solutions:

[0007] A target tracking method based on a double attention mechanism includes the following steps:

[0008] Step 1, a backbone network model is constructed, the backbone network model includes a detection branch and a template branch, for a given several groups of template frames and search frames, the template branch is used to extract deep features of the template frames, and the detection branch is used to extract deep features of the search frames;

[0009] Step 2, the deep features of the template frames and the search frames extracted in step 1 are selectively enhanced through deformable attention;

[0010] Step 3, the features of the template frames and the search frames enhanced in step 2 are subjected to weight redistribution and hierarchical cross fusion through sparse attention, and finally fusion features are obtained;

[0011] Step 4, the fusion features obtained in step 3 are predicted through a multi-layer perception machine to obtain the position and scale of the target;

[0012] Step 5, the multi-layer perception machine outputs the target position information of each detection frame, and the tracking is completed.

[0013] Further, the detection branch and the template branch each include 4 convolution layers, the input of a subsequent convolution layer is the output of a previous convolution layer, and the input image is subjected to feature extraction through the multi-layer convolution layers to capture more abundant features.

[0014] Further, the method for selectively enhancing the features in step 2 is:

[0015] A uniform grid is generated for the input feature map, and then reference points are generated according to the grid size;

[0016] A lightweight subnetwork is constructed, the feature map is linearly projected to query marked as q=xW q , and then fed to the lightweight subnetwork θ offset (·) to generate the offset Δp=θ offset (q) of the reference points;

[0017] The deformation points are obtained according to the reference points and the offsets thereof, the deformation points are subjected to bilinear interpolation to sample features from the feature map, then the sampled features are fed to the key and value projections to obtain deformable key and value, and finally the deformable key and value are subjected to multi-head attention to obtain enhanced feature maps.

[0018] Further, the method for generating the reference points in step 2 is:

[0019] Given an input feature x 2 R H×W×C , a uniform grid consisting of points is generated as reference; specifically, downsample the input feature map size by a factor r, H G = H / r, W G = W / r, where H denotes the height of the input feature map, W denotes the width of the input feature map, and the values of the reference points are linearly spaced two-dimensional coordinates {(0, 0), …, (H G -1, W G -1)}, which are then normalized to the range [-1, 1] according to the grid size H G x W G , where (-1, -1) denotes the top-left corner of the grid and (1, 1) denotes the bottom-right corner of the grid, thus obtaining the coordinates of the reference points.

[0020] Further, the lightweight subnetwork includes two convolutional modules with nonlinear activation, in which the input features are first captured by a 5x5 depth convolutional layer to capture local features, and then a GELU activation and a 1x1 convolutional layer are used to obtain the offset of the reference points.

[0021] Further, the method for obtaining a feature enhancement map according to the deformation points is as follows:

[0022] The features are sampled at the positions of the deformation points as key and value to obtain a projection matrix:

[0023] q = xW q ,

[0024] with Δp = θ offset (q),

[0025] In the formula, respectively denote the deformed key and value, W k denotes a linear mapping, W v denotes a linear mapping, and φ(·;·) is a sampling function; wherein the sampling function φ(·;·) is set as:

[0026]

[0027] After obtaining the deformed key and value, multi-head attention is performed on q, k, and v, and a relative position offset R is used to obtain an enhanced feature map, and the output formula of the attention head is:

[0028]

[0029] In the formula, m represents the number of attention heads, z represents the output, sigma represents the softmax operation, T represents the transpose operation, and d represents the dimension.

[0030] Further, the working mechanism of the sparse attention module in step 3 is:

[0031] First, the similarity matrix of query and key is calculated for the enhanced features obtained in step 2, and then the K largest elements in each row of the similarity matrix are normalized by using the softmax function, and the other elements are set to 0. Finally, the similarity matrix and the value are multiplied to obtain the final fusion result; wherein the softmax function is:

[0032] SMHA = SoftMax(TopK(QK T ))V T ;

[0033] In the formula, TopK represents that only the K largest values in each row of QK T are normalized, and the rest are set to 0.

[0034] Further, in step 3, the deformable attention in step 2 and the sparse attention in step 3 are combined to form a feature fusion network of the tracker, wherein the feature fusion network is described as:

[0035] F ZD m = DFA (F ZS m-1 , F XS m-1 , F XS m-1 );

[0036] F ZS m = SFA (F ZD m , F XD m-1 , F XD m-1 );

[0037] F XD m = DFA (F XS M , F ZS m-1 , F ZS m-1 );

[0038] F XS m = SFA (F XD m , FZD m ,F ZD m );

[0039] where m represents the current feature fusion layer, m-1 represents the last layer, ZD represents the DFA module of the template branch based on deformable attention, ZS represents the SFA module of the template branch based on sparse attention, XD represents the DFA module of the search branch based on deformable attention, and XS represents the SFA module of the search branch based on sparse attention.

[0040] Further, the multi-layer perception in step 4 includes two branches: a regression branch and a classification branch; the prediction of the feature vector in response to the label is selected as a positive sample, and the rest is a negative sample, and the multi-layer perception is trained; wherein all samples contribute to the classification loss, and only the positive samples contribute to the regression loss; the classification loss and the regression loss are sampled as the loss function of the multi-layer perception to train the multi-layer perception; finally, the fusion feature vector obtained through the feature fusion layer of several layers is input into the trained multi-layer perception to obtain the position and scale information of the target.

[0041] Further, the standard binary cross-entropy loss is used as the classification loss, which is defined as:

[0042] ζ cls =-∑ j [y j log(p(y j ))+(1-y j )log(1-p(y j ))];

[0043] where y j represents the true label of the jth sample, y j = 1 represents foreground, y j = 0 represents background, and p(y j ) represents the probability that the predicted label of the jth sample is the true label;

[0044] For the regression loss, L1 norm loss ζ1 and generalized intersection over union loss ζ GIoU are adopted, which are specifically:

[0045]

[0046] In the formula, represents only the points belonging to the target in the sample, b j represents the jth regression frame of the algorithm result, represents the regression frame of Ground-truth, and l GIOU loss and the commonly used l1 loss are adopted, λG and λ1 is a hyperparameter.

[0047] Compared with the prior art, the present application has the beneficial effects that: the present application proposes a feature fusion network based on deformable attention and sparse attention, which ensures that the network can focus on the most concerned information through deformable attention after extracting the deep features of the template frame and the search frame, and fuses the enhanced features of the template frame and the search frame through sparse attention, reduces the attention weight to the background, and relatively increases the attention weight to the potential target in the search area. Through the two operations, the robustness of the network is enhanced, and finally the interference of the background to the tracker is overcome. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is the overall framework diagram of the target tracking method based on the double attention mechanism of the embodiment of the present application;

[0049] Figure 2 is the deformable attention network structure diagram of the embodiment of the present application;

[0050] Figure 3 is the sparse attention network structure diagram of the embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0052] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0053] The present application will be further described below with reference to specific embodiments, but not as a limitation of the present application.

[0054] The embodiment of the present application provides a target tracking method based on a double attention mechanism, and the main idea is that: the network focuses on the most concerned information through deformable attention; and the features of the template frame and the search frame after enhancement are fused through sparse attention, the attention weight to the background is reduced, and the attention weight to the potential target in the search area is relatively increased. Through the two operations, the robustness of the network is enhanced, and finally the interference of the background to the tracker is overcome. As shown in Figure 1 The embodiment of the present application specifically includes the following steps:

[0055] Step 1, a backbone network model is constructed, the backbone network model includes two branches, which are a detection branch and a template branch, as shown in Figure 1As shown. For a given number of template frames and search frames, the template branch extracts deep features from the template frames, and the detection branch extracts deep features from the search frames; the specific process of feature extraction of the template frames and the search frames in step 1 is as follows:

[0056] The template frame and the search frame feature extraction process are completely consistent, and the template frame is taken as an example for description. The template frame The first convolutional module includes a convolutional layer with 64 7x7 convolutional kernels and a 3x3 max pooling layer, both with a stride of 2. The second convolutional module includes a convolutional layer with 64 1x1 convolutional kernels, a convolutional layer with 64 3x3 convolutional kernels, and a convolutional layer with 256 1x1 convolutional kernels, all with a stride of 1. The third convolutional module includes a convolutional layer with 128 1x1 convolutional kernels, a convolutional layer with 128 3x3 convolutional kernels, and a convolutional layer with 512 1x1 convolutional kernels, all with a stride of 2. The fourth convolutional module includes a convolutional layer with 256 1x1 convolutional kernels, a convolutional layer with 256 3x3 convolutional kernels, and a convolutional layer with 1024 1x1 convolutional kernels, all with a stride of 2. The output of the previous convolutional module is the input of the subsequent convolutional module, and each subsequent convolutional module can output more effective information than the previous convolutional module, so that the template frame and the search frame deep features obtained are more rich, providing more robust features for subsequent feature fusion and target positioning.

[0057] In step 2, the deep features of the template frames and the search frames extracted in step 1 are selectively enhanced by deformable attention;

[0058] In this step, a deformable attention mechanism is proposed to effectively model the relationship between tokens under the guidance of important regions in the feature map. These focus regions are determined by multiple groups of deformation sampling points, which are learned from the query by an offset network. This embodiment uses bilinear interpolation to sample features from the feature map, and then feeds the sampled features to the key and value projections to obtain deformable keys and values. Finally, a standard multi-head attention is applied to obtain enhanced feature maps. In addition, the positions of the deformation points provide a stronger relative position bias to facilitate the learning of deformable attention. As Figure 2The input feature map is shown as A uniform grid of points is generated as reference. Specifically, the grid size is down-sampled from the input feature map size by a factor r, H G = H / r, W G = W / r, where H denotes the height of the input feature map, W denotes the width of the input feature map, and the values of the reference points are linearly spaced two-dimensional coordinates {(0, 0), …, (H G -1, W G -1)}. Then they are normalized to the range [-1, 1] according to the grid shape H G x W G , where (-1, -1) denotes the top-left corner and (1, 1) denotes the bottom-right corner, so that the reference points are obtained. In order to obtain the offset of each reference point, the feature map is linearly projected to query marked as q = xW q , and then fed into a lightweight subnetwork θ offset (·) to generate the offset Δp = θ offset (q). The subnetwork designed using the query feature is used to generate the offset and output the offset value of the reference point respectively. Considering that each reference point covers a local s x s area (s is the maximum value of the offset), the generation network should also have the perception of local features to learn reasonable offsets. Therefore, the lightweight subnetwork is set to two convolution modules with nonlinear activation in this embodiment. In the lightweight subnetwork, the input feature is first captured by a 5 x 5 deep convolution to capture local features; then, GELU activation and 1 x 1 convolution are used to obtain 2D offset. It is also worth noting that the bias in the 1 x 1 convolution is reduced to mitigate the forced offset of all positions.

[0059] After obtaining the reference points and their corresponding offsets, the two are added to obtain the deformation points. The feature is sampled at the position of the deformation point as key and value to obtain the projection matrix:

[0060]

[0061]

[0062] wherein, denote the deformed key, value respectively, W k denotes a linear mapping, W v denotes a linear mapping. Specifically, the sampling function φ(·;·) is set to bilinear interpolation in this embodiment to make it differentiable:

[0063]

[0064] In the formula, (r x , r y) represents an integer pixel point on the feature map, (p x , p y ) represents the point to be interpolated, and x represents the entire feature map. g(a, b) = max(0, 1- |a-b|) and (r x , r y ) index all positions. Since g is only non-zero on the 4 integral points closest to (p x , p y ), it simplifies formula (3) to a weighted average of 4 positions. After obtaining the deformed key and value, multi-head attention is performed on q, k, and v, and a relative position offset R is adopted, where the output formula of the attention head is:

[0065]

[0066] In the formula, m represents the number of attention heads, z represents the output, sigma represents the softmax operation, T represents the transpose operation, and d represents the dimension.

[0067] In order to promote the diversity of deformation points, the embodiment follows a similar example in multi-head self-attention and divides the feature channels into G groups. Features from each group use a shared lightweight subnetwork to generate a corresponding offset, respectively. In practice, the number of heads M of the attention module is set to be a multiple of the size of the offset group G to ensure that multiple attention heads are allocated to a group of deformed key and value.

[0068] Step 3, the enhanced template frame and search frame features in step 2 are respectively subjected to hierarchical cross fusion of weight reassignment by sparse attention, and finally the fusion features are obtained;

[0069] After processing in step 2, each pixel value of each attention feature is calculated from all pixel values of the input feature, which makes the foreground edge region blurred. For this purpose, the embodiment also proposes a sparse attention method, in which each pixel value of the attention feature is determined only by the K most similar pixel values, which makes the foreground more concentrated and the foreground edge region more resolution. As shown in Figure 3 , first, the similarity matrix of query and key is calculated for the enhanced feature map obtained in step 2, then only the K largest elements of each row of the similarity matrix are normalized using the softmax function, and the other elements are set to 0; finally, the similarity matrix is multiplied by the value to obtain the final result; wherein the softmax function is:

[0070] SMHA = SoftMax(TopK(QK T ))V T ; (5)

[0071] where TopK denotes normalizing only the K largest values of each row of QK T and setting the rest to 0.

[0072] Figure 3 The right side shows the difference between the two attention normalizations. The original attention amplifies the relatively small similar weights, which makes the output features vulnerable to noise and background interference. However, sparse attention can significantly alleviate this problem.

[0073] In order to reduce the attention weight on the background and relatively increase the attention weight on the potential target in the search area, the embodiment combines the deformable attention of step 2 and the sparse attention of step 3 to form a feature fusion network of the tracker. The feature fusion network first focuses the deformable attention on the most attention information, and then uses the sparse attention to reduce the attention on the background. The two together overcome the influence of the target background on the tracking. Specifically, the feature fusion network can be described by the following formula:

[0074] F ZD M = DFA(F ZS M-1 , F XS m-1 , F XS m-1 ); (6)

[0075] F ZS m = SFA(F ZD m , F XD m-1 , F XD m-1 ); (7)

[0076] F XD m = DFA(F XS m , F ZS m-1 , F ZS m-1 ); (8)

[0077] F XS m = SFA(F XD m , F ZD m , F ZD m ); (9)

[0078] In the formula, m represents the current feature fusion layer, m-1 represents the previous layer, ZD represents the DFA module of the template branch based on deformable attention, ZS represents the SFA module of the template branch based on sparse attention, XD represents the DFA module of the search branch based on deformable attention, and XS represents the SFA module of the search branch based on sparse attention.

[0079] The feature fusion network is used to perform feature fusion on the input feature map to obtain fused features.

[0080] Step 4: The fused features obtained in step 3 are input into a multi-layer perceptron to predict the position and scale of the target.

[0081] The fused feature vector is obtained through several layers of feature fusion layers and is input into a prediction head (multi-layer perceptron) to obtain the position and scale information of the target. The multi-layer perceptron includes two branches: a regression branch and a classification branch. In this embodiment, the prediction of the feature vector in response to the label is selected as a positive sample, and the rest is a negative sample, and the multi-layer perceptron is trained. During the training process, all samples contribute to the classification loss, and only the positive samples contribute to the regression loss. This embodiment uses a standard binary cross-entropy loss as the classification loss, which is defined as:

[0082]

[0083] where y j represents the true label of the jth sample, y j = 1 represents foreground, and y j = 0 represents background. p(y j ) represents the probability that the predicted label of the jth sample is the true label.

[0084] For the regression loss, this embodiment adopts the L1 norm loss ζ1 and the generalized intersection over union loss ζ GIoU , which is specifically:

[0085]

[0086] In the formula, b represents only the points in the sample that belong to the target, b j represents the jth regression box of the algorithm result, represents the regression box of Ground-truth, and the l GIoU loss and the commonly used l1 loss are adopted, λ G and λ1 are hyperparameters.

[0087] The classification loss and the regression loss are taken as the loss function of the multilayer perceptron, and the multilayer perceptron is trained to obtain a trained multilayer perceptron. Then, the fusion feature vector obtained through the feature fusion layer is input into the trained multilayer perceptron, so that the position and scale information of the target can be obtained.

[0088] Step 5: The multilayer perceptron outputs the position information of the target in each detection frame, and the tracking is completed.

[0089] The embodiment also provides a quantitative comparison table of the method of the embodiment and other advanced algorithms in a dataset OTB100 (OTB2015), and the table is specifically shown in Table 1 (the crossed-out part is ranked first, and the underlined part is ranked second). The algorithm proposed in the embodiment achieves the second-ranked result in the tracking environment of scale change and occlusion, achieves the first-ranked result in the tracking environment of fast motion, cluttered background and motion blur, and the difference between the second-ranked scale change and occlusion and the first-ranked result is not large, which proves the effectiveness of the method proposed in the embodiment.

[0090] Table 1: Success rates of 10 trackers in different tracking challenges of OTB dataset

[0091]

[0092]

[0093] The above merely describes the preferred embodiments of the present application, but does not limit the implementation manners and protection scope of the present application. It should be noted by those skilled in the art that any equivalent replacement and obvious changes made according to the contents of the present application should be included in the protection scope of the present application.

Claims

1. A target tracking method based on a dual-attention mechanism, characterized in that, The method comprises the following steps: Step 1, constructing a backbone network model, wherein the backbone network model comprises a detection branch and a template branch, and for a plurality of sets of template frames and search frames, the template frames are extracted by the template branch and the search frames are extracted by the detection branch; Step 2, the deep features of the template frames and the search frames extracted in step 1 are selectively enhanced by deformable attention; Step 3, the features of the template frames and the search frames enhanced in step 2 are subjected to weight redistribution and hierarchical cross fusion by sparse attention, and finally the fusion features are obtained; Step 4, the fusion features obtained in step 3 are subjected to multi-layer perception prediction to obtain the position and scale of the target; Step 5, the multi-layer perception outputs the target position information of each detection frame, and the tracking is completed; The method for selectively enhancing the features in step 2 is as follows: A uniform grid is generated for the input feature map, and then reference points are generated according to the grid size; A lightweight subnetwork is constructed, and the feature map is linearly projected to the query label as and then fed into the lightweight subnetwork to generate the offset of the reference point ; Deformation points are obtained according to the reference points and their offsets, the deformation points are sampled from the feature map by bilinear interpolation, and then the sampled features are fed to the key and value projection to obtain deformable key and value, and finally multi-head attention is performed on the deformable key and value to obtain enhanced feature maps; The method for obtaining feature enhancement maps according to the deformation points is as follows: The features are sampled at the positions of the deformation points as key and value, and a projection matrix is obtained: ； ； In the formula, p is a reference point, , respectively represent the transformed key and value, represents a linear mapping, represents a linear mapping, is a sampling function; wherein the sampling function is set as: In the formula, (x, y) represents an integer pixel point on the feature map, r x , r y (x, y) represents a point to be interpolated, p x , p y (x, y) represents a point to be interpolated, x represents the entire feature map; ; After obtaining the deformed key and value, multi-head attention is performed on q, k and v, and relative position offset R is used to obtain the enhanced feature map, and the output formula of the attention head is as follows: ； In the formula, m represents the number of attention heads, z represents output, denotes a softmax operation, T represents a transpose operation, and d represents a dimension.

2. The target tracking method based on a dual attention mechanism according to claim 1, characterized in that, Both the detection branch and the template branch comprise four convolutional layers, and the input of the last convolutional layer is the output of the previous convolutional layer, so that more rich features can be captured by the multi-layer convolutional layers.

3. The target tracking method based on a dual attention mechanism according to claim 1, characterized in that, The method for generating reference points in step 2 is as follows: Given input features , a uniform grid consisting of points is generated as reference; specifically, a factor r is down-sampled from the input feature map size, , where, H represents the height of the input feature map ，W represents the width of the input feature map ， The values of the reference points are linearly spaced two-dimensional coordinates { (0, 0),…,( H G -1, W G -1)} and then normalized to the range H G W G [1, 1] according to the grid size 1, 1] where 1, 1) represents the top-left corner of the grid and (1, 1) represents the bottom-right corner of the grid, so the coordinates of the reference points are obtained.

4. The target tracking method based on a dual attention mechanism according to claim 1, characterized in that, The lightweight subnetwork comprises two convolutional modules with nonlinear activation, in the lightweight subnetwork, the input features are first captured by a 5x5 deep convolutional layer, and then GELU activation and a 1x1 convolutional layer are used to obtain the offset of the reference points.

5. The target tracking method based on a dual attention mechanism according to claim 1, characterized in that, The working mechanism of the sparse attention module in step 3 is as follows: Firstly, the similarity matrix of query and key is calculated for the enhanced features obtained in step 2, then the K largest elements of each row of the similarity matrix are normalized by using the softmax function, and the other elements are set to 0; finally, the similarity matrix and the value are multiplied to obtain the final fusion result; wherein the softmax function is as follows: ； In the formula, TopK represents normalizing only K maximum values of each row of QK T and setting the rest to 0.

6. The target tracking method based on a dual attention mechanism according to claim 1, characterized in that, In step 3, before obtaining the fusion features, the deformable attention in step 2 and the sparse attention in step 3 are combined to form a feature fusion network of the tracker, wherein the feature fusion network is described as follows: ； ； ； ； Wherein, m represents the current feature fusion layer, m-1 represents the last layer, ZD represents the DFA module of the template branch based on deformable attention, ZS represents the SFA module of the template branch based on sparse attention, XD represents the DFA module of the search branch based on deformable attention, and XS represents the SFA module of the search branch based on sparse attention.

7. The target tracking method based on a dual attention mechanism according to claim 1, characterized in that, The multi-layer perception in step 4 includes two branches: a regression branch and a classification branch; the prediction of the feature vector in response to the label is selected as a positive sample, and the rest is a negative sample, and the multi-layer perception is trained; wherein, all samples contribute to the classification loss, and only the positive samples contribute to the regression loss; the classification loss and the regression loss are sampled as the loss function of the multi-layer perception to train the multi-layer perception; finally, the fusion feature vector obtained through the feature fusion layer of several layers is input into the trained multi-layer perception to obtain the position and scale information of the target.

8. The target tracking method based on a dual attention mechanism according to claim 7, characterized in that, The standard binary cross-entropy loss is used as the classification loss, which is defined as: ； wherein, denotes the true label of the j-th sample, = 1 denotes foreground, = 0 denotes background, denotes the probability that the predicted label of the j-th sample is the true label; For the regression loss, the Huber loss is adopted the Huber loss and the generalized intersection over union loss , in particular: ； where, represents the points in the sample that belong to the target, b j represents the first regression box of the algorithm result, represents the regression box of Ground-truth, which adopts loss and the commonly used loss, and is a hyperparameter.

Images