Target tracking method and system based on attention gating network

By introducing attention gating network and rectangular box optimization network in target tracking, the problem of low robustness in complex scenarios in the prior art is solved, and effective treatment of similar interferers and target partial occlusion is achieved, and tracking is improved Accuracy and robustness.

CN119027455APending Publication Date: 2024-11-26FUDAN UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411253962.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing segmentation tracking algorithms are difficult to deal with similar distractors, significant appearance and background changes, and partial occlusion of targets in complex scenarios, and are less robust.

Method used

Using the target tracking method based on attention-gating network, templates and search area features are extracted through the backbone network, sparse gating weights are generated, and combined with cross-correlation networks and segmentation networks, the position feature output of similar objects is suppressed, and further optimized through rectangular boxes. The network alleviates the problem of partial occlusion of targets.

Benefits of technology

The robustness and accuracy of the target tracking algorithm in complex scenarios is improved, and effective processing of similar interferers, significant appearance and background changes is achieved, ensuring accurate tracking of the target.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027455A_ABST
    Figure CN119027455A_ABST
Patent Text Reader

Abstract

The invention relates to a target tracking method and system based on an attention gating network, and the method comprises the steps: obtaining a target tracking video sequence, carrying out the image sampling, and constructing an image pair containing a target template image and a search region image; template features and search area features are extracted from the target template image and the search area image by adopting a backbone network; respectively adopting a cross-correlation network and an attention gating network to generate a position feature of the target and a sparse gating weight according to the template feature and the search area feature; and performing dot product operation on the gating weight and the position feature, inputting the gating weight and the position feature into a segmentation network, generating a segmentation mask of the target, outputting a target rectangular frame according to a maximum segmentation region, and displaying the target rectangular frame at a corresponding position of the target tracking video sequence. Compared with the prior art, the method has the advantages that the robustness of a tracking algorithm to the problems that the target is partially shielded, the target has the remarkable appearance and the background changes is effectively improved, and accurate and robust target tracking in a complex scene is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target tracking technology, and in particular to a target tracking method and system based on an attention gated network. Background Art

[0002] With the rapid development of hardware devices such as visual sensors, graphics processors, and computers, as well as the leap forward in deep learning technology, breakthroughs have been made in the field of visual target tracking, and target tracking technology has become increasingly mature. Unlike the requirement for limiting target category information in target detection tasks, target tracking tasks aim to continuously track the position and scale information of targets in subsequent video frames given only the initial rectangular frame of any target of interest. With the development and popularization of target tracking technology, visual target tracking has been widely used in military, industrial, and civilian fields, mainly including military reconnaissance, drone tracking, missile guidance, video surveillance, visual navigation, medical diagnosis, autonomous driving, intelligent transportation, human-computer interaction, sports competition, and other fields. Therefore, visual target tracking technology has important practical and research significance.

[0003] In recent decades, target tracking technology has made great progress. Among them, the tracking algorithm based on discriminative segmentation applies the target segmentation network to the field of target tracking, breaking the limitations of the tracking algorithm based on rectangular boxes, realizing pixel-level segmentation and tracking of targets, and obtaining excellent tracking performance on multiple tracking datasets. However, this type of algorithm still faces challenging problems that need to be solved in actual complex scenes, such as similar interference objects, significant appearance and background changes, partial or severe occlusion, etc. Therefore, the existing segmentation tracking algorithm still has a lot of room for improvement. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a target tracking method and system based on an attention gated network that achieves robustness against similar interferers and alleviates the problem of partial occlusion of the target.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A target tracking method based on an attention gated network comprises the following steps:

[0007] Obtain target tracking video sequences, perform image sampling, and construct image pairs including target template images and search area images;

[0008] The backbone network is used to extract template features and search area features from the target template image and search area image in the image pair respectively;

[0009] The cross-correlation network is used to generate the target's position features based on the template features and the search area features. The attention gating network is used to generate sparse gating weights based on the template features and the search area features. The gating weights are dot-producted with the position features to obtain the processed position features.

[0010] The processed position features are input into the segmentation network to generate a segmentation mask of the target, and a target rectangular frame is output according to the largest segmentation area, and the segmentation mask and the target rectangular frame are displayed at corresponding positions of the target tracking video sequence.

[0011] Furthermore, the attention gating network includes multiple convolution modules and multiple cascaded multi-head attention modules, the multiple convolution modules are used to perform dimensionality reduction and transformation processing on template features and search area features, and map them to the same feature space; the multiple cascaded multi-head attention modules are used to fuse semantic information step by step for the results processed by multiple convolution modules, and finally generate sparse gating weights through Sigmoid operation.

[0012] Furthermore, the multi-head attention module divides the input features into multiple groups of subspaces based on the feature dimension, and independently calculates the attention of each group of subspaces, and finally fuses them;

[0013] The calculation expression corresponding to the multi-head attention module is:

[0014] MH Att(Q,K,V)=Concat(h1,...,h n )W O

[0015] h i =Att(q i , k i , v i )

[0016] Where MHAtt represents the multi-head attention mechanism, Q, K, and V represent the query, key, and value in the attention, respectively, n represents the number of input features grouped, and W O represents the parameter matrix of the output linear layer, i represents the i-th feature group, and Att represents the single-head scale dot product attention mechanism.

[0017] Furthermore, the cross-correlation network first performs dimensionality reduction and transformation processing on the template features and the search area features through multiple convolution modules, and then calculates the similarity score of each position between the template features and the search area features through pixel-by-pixel cross-correlation operation, and divides them into a foreground similarity map and a background similarity map according to the mask of the template; then, a TopK operation is used to generate the final single-channel foreground similarity score and background similarity score, and a Softmax calculation is performed on the foreground similarity score and the background similarity score to obtain a target position similarity map; a DCF module is used to locate the center position of the target, thereby generating a position map; the foreground similarity map, the target position similarity map and the position map are cascaded and input into the convolution module to obtain the position features of the target.

[0018] Furthermore, the expression of the pixel-by-pixel cross-correlation operation is:

[0019]

[0020] In the formula, S i,j is the similarity score of each position between the template feature and the search area feature, i, j represents the feature position, c represents the feature dimension, and F t and F r Represent the template features and search area features respectively;

[0021] The expression of the TopK operation is:

[0022]

[0023] In the formula, S j is the similarity score of position j, m and k represent the first m maximum values ​​and the kth maximum value respectively;

[0024] The calculation expression of the position map is:

[0025]

[0026] Where LM is the position map, point (p x ,p y ) represents the predicted target center position, x and y are the horizontal and vertical coordinates of the input image respectively.

[0027] Furthermore, the segmentation network includes a plurality of upsampling modules, and the mask of the target is refined step by step through the plurality of upsampling modules;

[0028] The processing process of each upsampling module includes: using a convolutional layer to reduce the dimension of the processed position features and the template features extracted by the backbone network, and transforming them to the same feature space, and then performing an element-by-element summation operation; using a double-resolution upsampling layer to improve the resolution of the summation operation result.

[0029] Furthermore, the loss function in the segmentation network training process introduces an edge loss term, and the calculation expression of the edge loss term is:

[0030]

[0031] In the formula, is the edge loss term, N represents the total number of pixels in the input image, FG and BG represent the foreground pixel and background pixel sets respectively, EG represents the edge pixel set, that is, the set of points with a target edge thickness of 1 pixel in the real mask, and y i and p i represent the true label and predicted mask respectively.

[0032] Furthermore, the method uses an IoUNet-based rectangular frame optimization network to optimize the output target rectangular frame based on the segmentation network output result. The processing process of the IoUNet-based rectangular frame optimization network includes:

[0033] In the first frame of the target tracking video sequence, the backbone network is used to extract the target template feature x0, thereby initializing the IoU modulation vector; according to the rectangular box B0 output by the segmentation network, a set of proposed rectangular boxes B are generated by random jittering. m , and extract the corresponding search feature x for the proposed rectangle t ; According to the IoU modulation vector and the proposed rectangular box B m and search feature x t The constructed search vector maximizes the IoU value of the rectangular box to obtain the optimized target rectangular box.

[0034] Furthermore, the calculation expression of the IoU value of the maximized rectangular box is:

[0035]

[0036] In the formula, IoU(B m ) is the optimized target rectangular box, v(x0,B0) is the IoU modulation vector generated based on the target template feature x0 and the rectangular box B0 output by the segmentation network, v(x t ,B m ) is based on the proposed rectangle B m and search feature x t The constructed search vector, For IoU prediction network.

[0037] The present invention also provides a target tracking system based on an attention gated network, comprising a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the method described above.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] (1) Existing segmentation and tracking algorithms only use the cross-correlation mechanism for feature fusion. The cross-correlation mechanism is a local linear matching mechanism that is difficult to capture the rich semantic information of the target. Therefore, this type of tracking algorithm has low robustness in actual complex scenes and is difficult to handle challenging problems such as similar interference objects, significant appearance or background changes, and partial occlusion in complex scenes.

[0040] According to the template features and search area features proposed by the backbone network, the present invention uses an attention gating network to effectively capture the global semantic information of the target, thereby generating sparse gating weights; a cross-correlation network is used to generate the position features of the target, and the gating weights are subjected to a dot product operation with the position features to suppress the position feature output of similar objects, thereby achieving robustness to similar interference objects;

[0041] After the position features calculated by gated weights are input into the segmentation network, a rectangular box optimization network is used to further optimize the output accuracy of the target tracking rectangular box to alleviate the problem of partial occlusion of the target. Overall, it can effectively improve the robustness of the tracking algorithm to partial occlusion of the target, significant appearance of the target and background changes, and achieve accurate and robust target tracking in complex scenes.

[0042] (2) The rectangular box optimization network based on IoUNet introduced in the present invention generates a set of rectangular box proposals through random jittering on the basis of the segmentation results, and extracts the corresponding search features; and combines the original output of the segmentation network to obtain the optimized target rectangular box by maximizing the IoU value of the rectangular box, so that the output rectangular box not only contains the local information of the target, but also alleviates the problem of partial occlusion of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a flow chart of a target tracking method based on an attention gated network provided in an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the effect of a target tracking method based on an attention gating network provided in an embodiment of the present invention on a data set. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0046] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0048] Example 1

[0049] This embodiment provides a target tracking method based on an attention gating network. The method uses a gating network based on mutual attention to generate sparse gating weights to control the position feature output of the cross-correlation network, thereby suppressing the feature weights of similar interferers and achieving robust segmentation and tracking. Figure 1 As shown in the figure, the method mainly includes an attention gating network, a cross-correlation network, a segmentation network and a rectangular box optimization network, including the following steps:

[0050] S1: Obtain the target tracking video sequence, perform image sampling, and construct an image pair including the target template image and the search area image;

[0051] S2: Use the backbone network to extract template features and search area features from the target template image and search area image in the image pair respectively;

[0052] S3: Use the cross-correlation network to generate the target's position features based on the template features and the search area features; use the attention gating network to generate sparse gating weights based on the template features and the search area features; perform dot product operation on the gating weights and the position features to obtain the processed position features;

[0053] S4: Input the processed position features into the segmentation network, generate a segmentation mask of the target, and output the target rectangular box according to the largest segmentation area, and display the segmentation mask and the target rectangular box at corresponding positions in the target tracking video sequence.

[0054] This is equivalent to the following: in the tracking process, the backbone network is first used to extract template features and search area features; then the cross-correlation network is used to generate the position similarity features of the target; then the attention gating network is used to generate sparse gating weights, and the gating weights are dot-producted with the position features to suppress the position feature output of similar objects; finally, the gated position features are input into the segmentation network to generate an accurate mask of the target, and the rectangular frame of the target is output based on the largest segmented area. In order to alleviate the problem of partial occlusion of the target, a rectangular frame optimization network is used to further optimize the output accuracy of the target tracking rectangular frame.

[0055] In addition, optionally, an online target tracking system based on an attention-gated network is built through the design of an interactive interface, which achieves real-time and accurate tracking in complex scenes.

[0056] The specific contents of the above method are described below.

[0057] 1. Attention Gating Network

[0058] The attention gating network includes multiple convolution modules and multiple cascaded multi-head attention modules. Multiple convolution modules are used to reduce the dimension and transform the template features and search area features, and map them to the same feature space; multiple cascaded multi-head attention modules are used to fuse the semantic information of the results processed by multiple convolution modules step by step, and finally generate sparse gating weights through Sigmoid operation.

[0059] The multi-head attention module is good at capturing global semantic information because it can focus on global information and enhance the target of interest. This module mainly contains linear layers, multi-head attention modules and feedforward networks. First, define the single-head attention formula as follows:

[0060]

[0061] Where Att represents the single-head scale dot product attention mechanism. Q, K, and V represent the query, key, and value in the attention, respectively. k Represents the scale factor, whose value is the dimension of key K. K T represents the transpose of the matrix K.

[0062] Based on formula (1), a multi-head attention mechanism is defined as shown in formula (2) and formula (3). Different from single-head attention, the multi-head attention mechanism divides the input features into multiple groups of subspaces based on the feature dimension, and then independently calculates the attention of each group of subspaces to enhance the representation ability of the input features. In the MHCA module, three linear layers are first used to transform the input features into the query Q, key K and value V in the attention mechanism; then the multi-head attention mechanism defined in formula (2) is used to fuse the semantic information of the target; then the semantic features are output through the feedforward network and residual connection; finally, the semantic information of the target is fused step by step through three multi-head attention modules, and the final gating weight is output through the Sigmoid layer.

[0063] The calculation expression corresponding to the multi-head attention module is:

[0064] MH Att(Q,K,V)=Concat(h1,...,h n )W O (2)

[0065] h i =Att(q i , k i , v i ) (3)

[0066] Where MHAtt represents the multi-head attention mechanism, Q, K, and V represent the query, key, and value in the attention, respectively, n represents the number of input features grouped, and W O represents the parameter matrix of the output linear layer, i represents the i-th feature group, and Att represents the single-head scale dot product attention mechanism.

[0067] In this embodiment, the attention gating network mainly includes two convolution modules and three multi-head attention modules. First, the weight-sharing 1x1 and 3x3 convolution modules are used to reduce the dimension and transform the target template features and search area features extracted by the backbone network to improve the computational efficiency and map the two features to the same feature space; then three cascaded multi-head mutual attention modules are used to fuse rich semantic information step by step; finally, sparse gating weights are generated through Sigmoid operation to suppress the position feature output of similar objects.

[0068] 2. Cross-correlation network

[0069] The cross-correlation network aims to extract the position distribution information of the target to supplement the target two-dimensional position distribution information lost in the attention mechanism. In this network, the template features and search area features are first reduced in dimension and transformed through multiple convolution modules, and then the similarity score of each position between the template features and the search area features is calculated through pixel-by-pixel cross-correlation operations, and divided into foreground similarity maps and background similarity maps according to the mask of the template; then the final single-channel foreground similarity score and background similarity score are generated through TopK operations, and the foreground similarity score and background similarity score are Softmax calculated to obtain the target position similarity map; the DCF module is used to locate the center position of the target, thereby generating a position map; the foreground similarity map, the target position similarity map and the position map are cascaded and input into the convolution module to obtain the position features of the target.

[0070] The expression of pixel-by-pixel cross-correlation operation is:

[0071]

[0072] In the formula, S i,j is the similarity score of each position between the template feature and the search area feature, i, j represents the feature position, c represents the feature dimension, and F t and F r Represent the template features and search area features respectively;

[0073] The expression of TopK operation is:

[0074]

[0075] In the formula, S j is the similarity score of position j, m and k represent the first m maximum values ​​and the kth maximum value respectively;

[0076] The calculation expression of the position map is:

[0077]

[0078] Where LM is the position map, point (p x ,p y ) represents the predicted target center position, x and y are the horizontal and vertical coordinates of the input image respectively.

[0079] Specifically, in this embodiment, two weight-sharing 1x1 and 3x3 convolution modules are first used to transform and reduce the template and search area features extracted by the backbone network to improve the computational efficiency; then, the pixel-by-pixel cross-correlation operation defined in formula (4) is used to calculate the similarity score of each position between the template feature and the search area feature, and the similarity is divided into foreground and background according to the mask of the template; then, the TopK operation defined in formula (5) is used to generate the final single-channel foreground similarity score S1 j and background similarity score S0 j , in S1 j and S0 j A Softmax calculation is performed between the foreground and the foreground to obtain a target position similarity map. In order to enhance the robustness of the algorithm, the DCF (Discriminative Correlation Filters) module is further used to roughly locate the center position of the target, and the position map LM is generated according to the predicted center position using formula (6). Finally, the foreground similarity map, the position similarity map, and the position map are concatenated and input into a two-layer 3x3 convolution module to generate the target position feature.

[0080] 3. Segmentation Network

[0081] The segmentation network aims to generate an accurate mask of the target to achieve pixel-level segmentation and tracking. The network specifically includes multiple upsampling modules, which gradually refine the target mask.

[0082] The processing process of each upsampling module includes: using a convolutional layer to reduce the dimension of the processed position features and the template features extracted by the backbone network, and transforming them to the same feature space, and then performing an element-by-element summation operation; using a double-resolution upsampling layer to improve the resolution of the summation operation result.

[0083] Usually, the segmentation network uses the Binary Cross-Entropy (BCE) loss function to train the segmentation model. However, the BCE loss function considers all pixels to be equally important, and the edge of the target accounts for a small proportion, so the model trained using the BCE loss function ignores the edge information of the target, which ultimately leads to inaccurate edge segmentation. Therefore, this scheme introduces an additional edge loss term, as shown in formula (7), to implicitly guide the segmentation model to pay attention to the edge information of the target, thereby maintaining the integrity of the edge segmentation for subsequent accurate target scale estimation.

[0084]

[0085] In the formula, is the edge loss term, N represents the total number of pixels in the input image, FG and BG represent the foreground pixel and background pixel sets respectively, EG represents the edge pixel set, that is, the set of points with a target edge thickness of 1 pixel in the real mask, and y i and p i represent the true label and predicted mask respectively.

[0086] In this embodiment, the segmentation network mainly includes three up-sampling modules (UM) to refine the target mask step by step. Each up-sampling module includes two groups of 3x3 convolutional layers and a double resolution up-sampling layer (DUP). In each up-sampling module, two groups of convolutional layers are first used to reduce the dimension and transform the position features processed by the attention gated network and the template features extracted by the backbone network, respectively, and the two features are transformed into the same feature space to realize the element-by-element summation operation of the two features; then the DUP layer is used to improve the feature resolution. Finally, three such up-sampling UM modules are used to output the final high-resolution target mask.

[0087] In addition, the segmentation template samples are updated online according to the results of segmentation tracking to improve the robustness of the segmentation network to changes in the target's salient appearance and background.

[0088] 4. Rectangular frame optimization network

[0089] Segmentation-based tracking algorithms have difficulty dealing with the problem of partially occluded targets, because the tracking task ultimately requires the output of a rectangular box of the target, and the general segmentation-based tracking algorithm generates a rectangular box of the target based on the largest segmentation area. Once the target is partially occluded, the target will be divided into multiple areas, which makes the output rectangular box contain only local information of the target, and as the error accumulates, it will eventually lead to tracking failure. In order to alleviate this problem, preferably, this solution introduces a rectangular box optimization network based on IoUNet to further optimize the output target rectangular box based on the segmentation result.

[0090] The rectangular box optimization network based on IoUNet uses the backbone network to extract the target template feature x0 in the first frame of the target tracking video sequence and initializes the IoU (Intersection over Union) modulation vector; then, according to the rectangular box B0 output by the segmentation module, a set of rectangular box proposals B is generated by random jitter. m , and extract the corresponding search feature x for the proposed rectangle t Finally, the optimized target rectangular box is obtained by maximizing the IoU value of the rectangular box through formula (8).

[0091]

[0092] In the formula, IoU(B m ) is the optimized target rectangular box, v(x0,B0) is the IoU modulation vector generated based on the target template feature x0 and the rectangular box B0 output by the segmentation network, v(x t ,B m ) is based on the proposed rectangle B m and search feature x t The constructed search vector, For IoU prediction network.

[0093] 5. Target Tracking System

[0094] The present solution also proposes a target tracking system based on an attention gating network, comprising a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the above-mentioned target tracking system method based on an attention gating network.

[0095] Preferably, the target tracking system also designs an interactive interface, allowing the mouse to select any target and continuously and accurately track the target in subsequent frames. This system has strong scalability and is applicable to all tracking algorithms based on rectangular box initialization.

[0096] The following uses the VOT2016

[13] and VOT2018[6] test tracking datasets as examples to illustrate the use of the solution of the present invention.

[0097] The target tracking method based on the attention gating network in the present invention is represented by AGST-BR (Attention-basedGating network for Segmentation Tracking).

[0098] This method mainly includes three steps:

[0099] 1) Collect video sequences from public datasets on the Internet (including Youtube-VOS

[14] , GOT-10K[8], and LaSOT

[15] datasets) and randomly sample the entire video sequence to generate training image pairs for training the segmentation and tracking model.

[0100] 2) Through the backbone network, the collected image pairs are extracted and trained to obtain a target tracking model based on the attention gating network. The model generates gating weights through the attention network to suppress the output of the similar object position features of the cross-correlation network, and further optimizes the output accuracy of the final tracking rectangle through the rectangle optimization network.

[0101] 3) In the designed online target tracking system, an interactive interface is used to initialize any target of interest by clicking and dragging the mouse, and real-time and accurate tracking is achieved in subsequent frames.

[0102] The specific implementation steps of the above segmentation tracking method and system are as follows:

[0103] 1. Building training image pairs for segmentation and tracking models

[0104] 1.1) Download the training video sequences of Youtube-VOS, GOT-10K and LaSOT datasets through Baidu Cloud or Google Cloud;

[0105] 1.2) For the Youtube-VOS segmentation dataset, the real annotated masks are directly used to train the segmentation and tracking model; for the GOT-10K and LaSOT tracking datasets, the box-to-mask method is used to pre-generate pseudo-annotated masks to train the segmentation and tracking model;

[0106] 1.3) Randomly sample the video sequence number, and perform two random image samplings on all the sequence frames in the sampled video. The maximum video frame interval between the two samplings is 50 frames. The two images annotated by the binary mask form an image pair (target template-search area image pair) for training the segmentation and tracking model.

[0107] 2. Training and learning segmentation tracking model

[0108] 2.1) For each training image pair, a backbone network (such as ResNet-50, ResNet-101, MobileNet, etc.) is used to extract deep features with a feature dimension of 1024x24x24, and then two layers of convolution are used to reduce the feature dimension and transform it. The transformed feature dimension is 64x24x24;

[0109] 2.2) The cross-correlation network is used to extract the position features of the target, with a feature dimension of 64x24x24. Then, the gating weights are generated by the attention gating network and multiplied with the position features to suppress the position feature output of similar objects.

[0110] 2.3) The position features processed by the gating network are input into the segmentation network, and the loss function defined in formula (7) is used to train the segmentation model. The model is trained for 50 epochs, each training epoch contains 1500 iterations, where the learning rate is set to 0.001, and the learning rate of the model decays to 0.2 times the original after every 20 training epochs. Each training iteration samples 32 pairs of images, that is, the batch size is set to 32, and the image size is set to 384x384;

[0111] 2.4) The rectangular box output by the segmentation network is input into the rectangular box optimization network based on IoUNet to further optimize the accuracy of the output tracking rectangular box.

[0112] 3. Online Target Tracking

[0113] 3.1) The test data sets VOT2016 and VOT2018 are respectively input into the online tracking system. The tracking algorithm can continuously and in real time track the target in the subsequent video frames according to the initial rectangular frame given in the first frame. The test results are shown in Table 1, where AGST-BR is the target tracking method based on the attention gated network proposed in the present invention. AGST is its lightweight version and does not include the rectangular frame optimization network;

[0114] 3.2) In addition, the system has an interactive interface that can take the picture frames taken by the camera in real time as input, and select any target of interest by clicking and dragging the mouse, and then continuously track the position and scale information of the target in subsequent frames.

[0115] Table 1 Comparison results of different tracking algorithms on VOT2016 and VOT2018 datasets

[0116]

[0117] In summary, this paper proposes an effective target tracking method and system based on attention gated network, and obtains excellent tracking results on two common tracking data sets, VOT2016 and VOT2018. Figure 2 As shown, the present invention realizes accurate and robust segmentation tracking in complex scenes. The present invention is easy to implement, has stable tracking performance, and has high tracking accuracy and robustness. The attention gating network proposed in the present invention effectively suppresses the position feature output of similar objects and improves the robustness of the tracking algorithm. Compared with the traditional segmentation tracking algorithm, the present invention has a wider application prospect and greater market value.

[0118] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A target tracking method based on an attention gated network, characterized in that: The following steps are involved: Obtain target tracking video sequences, perform image sampling, and construct image pairs including target template images and search area images; The backbone network is used to extract template features and search area features from the target template image and search area image in the image pair respectively; The cross-correlation network is used to generate the location features of the target according to the template features and the search area features; The attention gating network is used to generate sparse gating weights according to the template features and the search area features. The gating weights are dot-producted with the position features to obtain the processed position features. The processed position features are input into the segmentation network to generate a segmentation mask of the target, and a target rectangular frame is output according to the largest segmentation area, and the segmentation mask and the target rectangular frame are displayed at corresponding positions of the target tracking video sequence.

2. According to claim 1, a target tracking method based on an attention gated network is characterized in that: The attention gating network includes multiple convolution modules and multiple cascaded multi-head attention modules. The multiple convolution modules are used to reduce the dimension and transform the template features and the search area features, and map them to the same feature space; the multiple cascaded multi-head attention modules are used to fuse the semantic information of the results processed by the multiple convolution modules step by step, and finally generate sparse gating weights through Sigmoid operation.

3. The target tracking method based on the attention gated network according to claim 2, characterized in that: The multi-head attention module divides the input features into multiple groups of subspaces based on the feature dimension, and independently calculates the attention of each group of subspaces, and finally fuses them; The calculation expression corresponding to the multi-head attention module is: MHAtt(Q,K,V)=Concat(h1,...,h n )W O h i =That(q i ,k i ,v i ) Where MHAtt represents the multi-head attention mechanism, Q, K, and V represent the query, key, and value in the attention, respectively, n represents the number of input features grouped, and W O represents the parameter matrix of the output linear layer, i represents the i-th feature group, and Att represents the single-head scale dot product attention mechanism.

4. The target tracking method based on the attention gated network according to claim 1, characterized in that: The cross-correlation network first performs dimension reduction and transformation processing on the template features and the search area features through multiple convolution modules, and then calculates the similarity score of each position between the template features and the search area features through pixel-by-pixel cross-correlation operation, and divides them into a foreground similarity map and a background similarity map according to the mask of the template; then, a TopK operation is performed to generate the final single-channel foreground similarity score and background similarity score, and a Softmax calculation is performed on the foreground similarity score and the background similarity score to obtain a target position similarity map; The DCF module is used to locate the center position of the target, thereby generating a position map; The foreground similarity map, target position similarity map and position mapping map are concatenated and input into the convolution module to obtain the position features of the target.

5. The target tracking method based on the attention gated network according to claim 4, characterized in that: The expression of the pixel-by-pixel cross-correlation operation is: In the formula, S i,j is the similarity score of each position between the template feature and the search area feature, i, j represents the feature position, c represents the feature dimension, and F t and F r Represent the template features and search area features respectively; The expression of the TopK operation is: In the formula, S j is the similarity score of position j, m and k represent the first m maximum values ​​and the kth maximum value respectively; The calculation expression of the position map is: Where LM is the position map, point (p x ,p y ) represents the predicted target center position, x and y are the horizontal and vertical coordinates of the input image respectively.

6. The target tracking method based on the attention gated network according to claim 1, characterized in that: The segmentation network includes a plurality of upsampling modules, and the mask of the target is refined step by step through the plurality of upsampling modules; The processing process of each upsampling module includes: using a convolutional layer to reduce the dimension of the processed position features and the template features extracted by the backbone network, and transforming them to the same feature space, and then performing an element-by-element summation operation; using a double-resolution upsampling layer to improve the resolution of the summation operation result.

7. The target tracking method based on the attention gated network according to claim 6, characterized in that: The loss function in the segmentation network training process introduces an edge loss term, and the calculation expression of the edge loss term is: In the formula, is the edge loss term, N represents the total number of pixels in the input image, FG and BG represent the foreground pixel and background pixel sets respectively, EG represents the edge pixel set, that is, the set of points with a target edge thickness of 1 pixel in the real mask, and y i and p i represent the true label and predicted mask respectively.

8. The target tracking method based on the attention gated network according to claim 1, characterized in that: The method uses a rectangular frame optimization network based on IoUNet to optimize the output target rectangular frame based on the output result of the segmentation network. The processing process of the rectangular frame optimization network based on IoUNet includes: In the first frame of the target tracking video sequence, the backbone network is used to extract the target template feature x0, thereby initializing the IoU modulation vector; according to the rectangular box B0 output by the segmentation network, a set of proposed rectangular boxes B are generated by random jittering. m , and extract the corresponding search feature x for the proposed rectangle t ; According to the IoU modulation vector and the proposed rectangular box B m and search feature x t The constructed search vector maximizes the IoU value of the rectangular box to obtain the optimized target rectangular box.

9. The target tracking method based on the attention gated network according to claim 8, characterized in that: The calculation expression of the IoU value of the maximized rectangular box is: In the formula, IoU(B m ) is the optimized target rectangular box, v(x0,B0) is the IoU modulation vector generated based on the target template feature x0 and the rectangular box B0 output by the segmentation network, v(x t ,B m ) is based on the proposed rectangle B m and search feature x t The constructed search vector, For IoU prediction network.

10. A target tracking system based on an attention gated network, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of any one of the methods according to claims 1 to 9.