Rotational attention single target tracking method and device based on deep neural network

Abstract

This application discloses a method and apparatus for single-target tracking based on deep neural networks using rotational attention. The method involves extracting features from the current frame image and features from a template image including the target; fusing the extracted features using a rotational attention module, and then extracting the features from the current frame image again using a preset decoder; fusing the features fused using the rotational attention module and the features from the newly extracted current frame image again using a sparse attention module to obtain target features; performing classification and regression on the target features to obtain classification and regression results; and determining the position of the target in the current frame image based on the classification and regression results. This method achieves high-precision tracking of a single target even in extreme environments such as partial occlusion, complete occlusion, and changes in lighting.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a rotational attention single-target tracking method and apparatus based on deep neural networks. Background Technology

[0002] Single-object tracking is one of the fundamental and widely applicable tasks in the field of computer vision. It aims to obtain feature information of the region where the target is located in the first frame of a video, and then use this information to estimate the state of the target and accurately locate it in subsequent frames. Single-object tracking technology has applications in many fields, such as intelligent video surveillance, autonomous driving, intelligent human-computer interaction, and human motion capture or eye tracking.

[0003] Single-object tracking is highly challenging due to changes in the target and its shape, such as pose variations, lighting conditions, and partial and complete occlusion. The real-time requirements of single-object tracking further complicate these challenges. The dynamic changes in the target introduce significant noise into the tracking information, and the spatial relationships between local features become blurred, leading to large discrepancies between the template image features and the tracking features, resulting in tracking failure. Since the target can be occluded by any other object, single-object tracking algorithms often experience severe degradation under partial and complete occlusion. While attention-based algorithms can capture global relationships between features, they involve substantial computation, making it difficult for the network to meet real-time requirements. This causes face alignment algorithms that perform well under constrained conditions to fail smoothly on devices with lower performance.

[0004] Therefore, how to achieve high-precision single-target tracking in extreme environments such as partial occlusion, complete occlusion, and changes in lighting is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a rotational attention single-target tracking method and apparatus based on deep neural networks, aiming to solve the aforementioned technical problems.

[0006] In a first aspect, this application provides a rotational attention single-target tracking method based on a deep neural network, the method comprising the following steps:

[0007] Extract features from the current frame image and features from the template image, including the tracked target;

[0008] The extracted features are fused based on the rotation attention module, and the features of the current frame image are extracted again through a preset decoder;

[0009] Based on the sparse attention module, the features fused by the rotation attention module and the features extracted from the current frame image are fused again to obtain the target features;

[0010] The target features are classified and regressed to obtain classification and regression results;

[0011] The location of the tracked target in the current frame image is determined based on the classification results and the regression results.

[0012] In some embodiments, extracting features of the current frame image and features of the template image including the tracking target includes:

[0013] The tracking target features and background features in the template image are extracted using a preset Swin-Transformer deep neural network, and the corresponding template image feature map is obtained.

[0014] The tracking target features and background features in the current frame image are extracted using the Swin-Transformer deep neural network, and the corresponding current frame image feature map is obtained.

[0015] The Swin-Transformer deep neural network includes:

[0016] The first-stage feature extraction network consists of: one linear transformation unit, two first basic units, and one merging unit connected in sequence.

[0017] The second-stage feature extraction network includes: two first basic units and one merging unit connected in sequence;

[0018] The third-stage feature extraction network includes: four first basic units connected in sequence;

[0019] The linear transformation unit is used to map each 4×4 scale region in the template image and the current frame image to a vector with a dimension of 96. The first basic unit includes a skip-linked attention mechanism and a multilayer perceptron. The merging unit includes a 2×2 convolution with a stride of 2 and a normalization unit. The merging unit is used to increase the dimension of the vector.

[0020] In some embodiments, the rotational attention module includes:

[0021] The first-stage rotating attention module includes four second basic units and one merging unit.

[0022] The second stage rotating attention module consists of four second basic units;

[0023] The second basic unit includes: four rotational attention mechanisms with skip connections and one multilayer perceptron, as well as a three-layer classification head, which includes three multilayer perceptrons and is activated by the ReLU function;

[0024] The decoder consists of window self-attention, and the decoder includes:

[0025] The first-stage decoding network consists of four first basic units, one batch normalization unit, and one 3×3 convolution with a stride of 1.

[0026] The second-stage decoding network comprises four of the first basic units;

[0027] Both the first-stage decoding network and the second-stage decoding network are connected to the rotating attention module through sparse attention.

[0028] In some embodiments, the extracted features are fused based on the rotation attention module, and the features of the current frame image are extracted again through a preset decoder, including:

[0029] For the current frame image feature map f x and the template image feature map f z The input vector for the rotation attention module is obtained by performing a linear transformation along the dimensions of image height H and image width W. The linear transformation process is as follows:

[0030]

[0031] Q1 = concat(P(M×f) z )),P(M×f x )))

[0032] K1 = V1 = concat(P(f z ,P(f x ))

[0033] Where M is the transformation matrix, Q1 is the first input vector of the rotation attention module, K1 is the second input vector of the rotation attention module, V1 is the third input vector of the rotation attention module, and P is a linear transformation function, which is used to transform the current frame image feature map f x and template image feature map f z Divide the window into four non-overlapping sections along the W and H dimensions;

[0034] The first input vector Q1 and the third input vector V1 of the rotation attention module are matrix multiplied along the dimensions N and C / h of the feature map, and the feature relationship matrix ξ between the features of the current frame image and the features of the template image is calculated using a normalization function.ij Where N = W × H;

[0035] The feature relation matrix ξ ij Multiply the current frame image features with the second input vector V1 to obtain the long-term dependency relationship between the features of the current frame image and the features of the template image;

[0036] Based on the long-term dependency relationship, features related to the tracking target are enhanced, while features not related to the tracking target are suppressed, resulting in the feature map f fused by the rotational attention module. h .

[0037] In some embodiments, the sparse attention module includes:

[0038] A sparse attention mechanism and a multilayer perceptron are used for jump links. The sparse attention module is connected to the decoding networks of the rotation attention module at each stage through sparse attention.

[0039] In some embodiments, the step of classifying and regressing the target features to obtain classification and regression results includes:

[0040] The target features are classified and regressed using a dual-head classification and regression network;

[0041] The dual-head classification and regression network includes one convolutional regression head and one classification regression head;

[0042] The convolutional regression head includes three convolutional modules consisting of a 3×3 convolutional layer with a stride of 1, a batch-normalized single clique, and a ReLU activation function.

[0043] The classification regression head consists of a 3-layer multilayer perceptron and a ReLU activation function.

[0044] In some embodiments, the computation of the sparse attention module includes:

[0045] The feature map f after fusing the rotational attention module h The input is given to the classification header to obtain the classification result S, where S∈R W×H ;

[0046] According to the largest top 12 2 Each classification result generates an index i, where i = topk(S, 12) 2 );

[0047] Based on index i, the feature map f of the current frame image x The selection process yields the input vector for the sparse attention module:

[0048] Q2 = concat(fz ,f x [i])

[0049]

[0050]

[0051] Where Q2 is the first input vector of the sparse attention module, K2 is the second input vector of the sparse attention module, V2 is the first input vector of the sparse attention module, and f z ,f x Features extracted by the rotation attention module are fused together. The features extracted again by the decoder.

[0052] In some embodiments, determining the location of the tracked target in the current frame image based on the classification result and the regression result includes:

[0053] The classification results were enhanced using a Hanning window.

[0054] The location of the tracking target in the current frame image is determined based on the regression result corresponding to the classification result with the highest enhanced score.

[0055] In some embodiments, before extracting features from the current frame image and features from the template image including the tracking target, the method further includes:

[0056] The template image and the current frame image are segmented and scaled to the corresponding preset scale;

[0057] The template image, after being cut and scaled, includes the tracking target and the background.

[0058] Secondly, this application also provides a rotational attention single-target tracking device based on a deep neural network, the device comprising:

[0059] The feature extraction module is used to extract features from the current frame image and features from the template image including the tracked target;

[0060] A rotational attention module is used to fuse the extracted features and extract the features of the current frame image again through a preset decoder;

[0061] The sparse attention module is used to fuse the features fused by the rotation attention module and the features extracted from the current frame image again to obtain the target features;

[0062] A dual-head classification and regression network is used to perform classification and regression on the target features to obtain classification and regression results.

[0063] A determination module is used to determine the position of the tracked target in the current frame image based on the classification result and the regression result.

[0064] This application provides a method and apparatus for single-target tracking based on deep neural networks with rotational attention. The method involves extracting features from the current frame image and features from a template image including the target; fusing the extracted features using a rotational attention module, and then extracting the features from the current frame image again using a preset decoder; fusing the features fused by the rotational attention module and the features extracted from the current frame image again using a sparse attention module to obtain target features; performing classification and regression on the target features to obtain classification and regression results; and determining the position of the target in the current frame image based on the classification and regression results. This method achieves high-precision tracking of a single target even in extreme environments such as partial occlusion, complete occlusion, and changes in lighting. Attached Figure Description

[0065] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A flowchart illustrating a rotational attention single-target tracking method based on a deep neural network, provided in an embodiment of this application;

[0067] Figure 2 This is a diagram of the overall framework based on deep neural networks;

[0068] Figure 3 This is a schematic diagram of a rotating attention structure;

[0069] Figure 4 This is a schematic diagram of a sparse attention structure;

[0070] Figure 5 This is a schematic block diagram of a rotational attention single-target tracking device based on a deep neural network, provided in an embodiment of this application.

[0071] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0074] This application provides a method and apparatus for rotational attention single-target tracking based on deep neural networks. This method can be applied to computer devices, such as laptops and desktop computers.

[0075] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0076] like Figures 1 to 4 As shown, the embodiment of this application provides a rotational attention single-target tracking method based on a deep neural network, which specifically includes the following steps:

[0077] like Figure 1 As shown, the method includes steps S1 to S5.

[0078] Step S1: Extract the features of the current frame image and the features of the template image including the tracked target.

[0079] It is worth noting that the template image is a reference image used when performing single-target tracking, which includes the target to be tracked and the background, while the current frame image is the image that needs to be analyzed.

[0080] In a preferred embodiment, before extracting the template image of the current frame and the template image of the tracking target, the method further includes: cutting and scaling the template image and the current frame image to a corresponding preset scale; wherein the cut and scaled template image includes the tracking target and the background.

[0081] Specifically, for a given template image and the current frame image, they are scaled and cropped to limit the search area to a smaller range. During cropping, the background around the tracking target is preserved, ensuring that the template image contains background information about the tracking target. The specific operations for scaling and cropping the image include scaling the image to a preset scale according to an image scaling factor: I = (w+h) / 4, s(w+2p)×s(h+2p). Here, for any given image z, it includes the search range s, the position of the tracking target b = (x0, y0, w, h), the preset image scale I, and the image scaling factor p, where x and y are the coordinates of the image center, and w and h are the width and height of the tracking target. In this embodiment, for the template image I = 192... 2 p=2, for the current frame image I=384 2 p = 4.

[0082] Specifically, the extraction of features from the current frame image and features from the template image including the tracking target includes: extracting tracking target features and background features from the template image using a preset Swin-Transformer deep neural network, and obtaining the corresponding template image feature map; and extracting tracking target features and background features from the current frame image using the Swin-Transformer deep neural network, and obtaining the corresponding current frame image feature map.

[0083] The Swin-Transformer deep neural network comprises three stages. The first stage feature extraction network includes: one linear transformation unit, two first basic units, and one merging unit connected in sequence; the second stage feature extraction network includes: two first basic units and one merging unit connected in sequence; and the third stage feature extraction network includes: four first basic units connected in sequence.

[0084] Exemplary, the first-stage feature extraction network includes a linear transformation unit that maps each 4×4 region of the image to a 96-dimensional vector, two first basic units consisting of one attention layer and one multilayer perceptron layer, with skip connections between the attention and perceptron layers in each unit, and a merging unit comprising a 2×2 convolution with a stride of 2 and a normalization unit, which increases the vector dimension to four times the original. The second-stage feature extraction network includes the two first basic units and the merging unit as described above, connected sequentially. The third-stage feature extraction network includes the four first basic units as described above, connected sequentially.

[0085] Preferably, the features extracted from the current frame image and the features extracted from the template image are placed in the same space to facilitate better fusion.

[0086] Step S2: The extracted features are fused based on the rotation attention module, and the features of the current frame image are extracted again through a preset decoder.

[0087] Specifically, the extracted features are fused using a rotational attention module. Simultaneously, a decoder composed of window self-attention is used to further extract features from the current frame. The fused features are then further fused with the further extracted features using a sparse attention module to obtain the extracted target features.

[0088] It is worth noting that the rotational attention module comprises a three-stage rotational attention module. The first-stage rotational attention module comprises four second basic units and one merging unit. The second-stage rotational attention module comprises four second basic units; wherein, the second basic unit comprises: four rotational attention mechanisms with skip connections and one multilayer perceptron, and a three-layer classification head, wherein the classification head comprises three multilayer perceptrons activated by the ReLU function.

[0089] The decoder consists of window self-attention and includes a two-stage decoding network: a first-stage decoding network, which includes four of the first basic units, one batch normalization unit, and one 3×3 convolution with a stride of 1; and a second-stage decoding network, which includes four of the first basic units. Both the first-stage decoding network and the second-stage decoding network are connected to the rotation attention module through sparse attention.

[0090] As an example, the specific network hierarchy of the rotating attention module is as follows: the rotating attention module consists of two stages, each connected to the sparse attention module via sparse attention. Each stage contains four second basic units, each consisting of a rotating attention mechanism, a one-layer multilayer perceptron, and a three-layer classification head. There are skip links between the attention and perceptron of each unit. The first stage contains a single merging unit with the same structure as described above, used for dimensionality enhancement, but with the dimensionality increased by 1.5 times. The classification head consists of three layers of multilayer perceptrons, activated by the ReLU function.

[0091] The specific network hierarchy of the decoder is as follows. The decoder also consists of two stages, each of which is connected to the rotation attention module through sparse attention. Each stage contains four basic units. In the first stage, in order to align the dimensions with the rotation attention module, a 3×3 convolution consisting of one batch normalization and one stride of 1 is used to increase the dimension by 1.5 times.

[0092] Step S3: Based on the sparse attention module, the features fused by the rotation attention module and the features extracted from the current frame image are fused again to obtain the tracking target features.

[0093] Specifically, the sparse attention module includes a sparse attention mechanism with skip links and a multilayer perceptron. The sparse attention module is connected to the decoding networks of each stage of the rotating attention module through sparse attention.

[0094] Step S4: Perform classification and regression on the features of the tracked target to obtain classification and regression results.

[0095] Specifically, the classification and regression of the target features to obtain classification and regression results includes:

[0096] The target features are classified and regressed using a dual-head classification and regression network; wherein the dual-head classification and regression network includes one convolutional regression head and one classification regression head; the convolutional regression head includes three convolutional modules consisting of one 3×3 convolutional layer with a stride of 1, one batch normalized single clique, and one ReLU activation function; the classification regression head includes three multilayer perceptron layers and one ReLU activation function.

[0097] As an example, the dual-head classification and regression head consists of one convolutional regression head and one classification head. The convolutional regression head comprises three convolutional modules, each consisting of a 3×3 convolutional layer with a stride of 1, a batch normalization function, and a ReLU activation function. The classification head consists of three multilayer perceptron layers activated by the ReLU function.

[0098] Step S5: Determine the position of the tracked target in the current frame image based on the classification result and the regression result.

[0099] Specifically, the classification results are enhanced using a Hanning window; the enhanced results are sorted, and the location of the tracking target in the current frame image is determined based on the regression result corresponding to the classification result with the highest enhanced score.

[0100] It's worth noting that real-time performance is crucial in target tracking, while maintaining a certain level of accuracy is also essential. Window attention is well-suited for this task; however, typical trackers require processing two inputs simultaneously, making window attention unsuitable for direct application. Rotation attention was designed to achieve a balance between performance and speed. After extracting features from the template image and the current frame image, to further extract and model the relationships between features, and to accurately locate the target position, this invention simultaneously inputs the feature maps into the rotation attention module and the decoder composed of basic units. The two modules interact using a sparse attention module.

[0101] As a preferred implementation, as shown in Figure 3, for the current frame image feature map f extracted by the deep neural network in step S1... x ,in and template image feature map f z ,in Where W is the width of the feature map, H is the height of the feature map, and C is the dimension of the feature map. After obtaining the feature map f of the current frame image... x and template image feature map f z Then, in order to obtain multi-scale features, they are first subjected to a linear transformation and then rearranged into... That is, its dimensions are divided into h groups, where h is the number of groups. Since this method uses the window attention mechanism, in order to exchange information between windows, the following transformation is performed on the dimensions of W and H:

[0102]

[0103] Q1 = concat(P(M×f) z )),P(M×f x )))

[0104] K1 = V1 = concat(P(f z ),P(f x ))

[0105] Where M is the transformation matrix, Q1 is the first input vector of the rotation attention module, K1 is the second input vector of the rotation attention module, V1 is the third input vector of the rotation attention module, and P is a linear transformation function, which is used to transform the current frame image feature map f x and template image feature map f z Divide the window into four non-overlapping windows along the W and H dimensions.

[0106] After creating four non-overlapping windows, the feature maps are rearranged back to their previous form, i.e., f∈R. N×h×C / h Where N = W × H, Q1, K1, and V1 are then used as inputs for attention. To model long-term dependencies and compute attention, Q1 and V1 are multiplied along the dimensions of N and C / h, and then normalized to obtain the feature relation matrix ξ. ij Next, we will examine the relation matrix ξ. ijMatrix multiplication is performed between the second input vector V1 to obtain the long-term dependency relationship between the features of the current frame image and the features of the template image. Based on the long-term dependency relationship, features related to the tracking target are enhanced, and features not related to the tracking target are suppressed, resulting in a new feature map f. h ,f h ∈R N×h×c / h Finally, a linear transformation is performed to aggregate the multi-scale features to obtain the final feature map f. h =f h W 0 ,f h ∈R N×C W 0 For a linear transformation, the angel increases by π / 4 for each rotational attention step in the rotational attention module. Through this attention mechanism, the network can selectively aggregate features, thereby focusing on more effective image regions.

[0107] In some preferred embodiments, such as Figure 4 The sparse attention algorithm shown includes:

[0108] The resulting feature map f of rotational attention h After passing through a classification head consisting of a three-layer multilayer perceptron (MLP), the classification result S, S∈R is obtained. W×H Select the top 12 S-sizes 2 The classification results yield an index i, where i = topk(S, 12). 2 For feature map f x To make a selection, during the attention process, the network focuses more on the features of the target itself. The cross-attention calculation is as follows:

[0109] Q2 = concat(f z ,f x [i])

[0110]

[0111]

[0112] Where Q2 is the first input vector of the sparse attention module, K2 is the second input vector of the sparse attention module, V2 is the first input vector of the sparse attention module, and f z ,f x Features extracted by the rotation attention module are fused together. The features extracted again by the decoder.

[0113] This embodiment provides a rotational attention single-target tracking method based on deep neural networks. The rotational attention network proposed in this application ensures that the network can better integrate the features of the tracking target and the template while extracting the features of the tracking target, and model the features of the tracking target to enhance the robustness of the algorithm under partial occlusion. The present invention also designs two novel attention mechanisms, namely rotational attention mechanism and sparse attention mechanism, which make the network pay more attention to the features that are helpful to the tracking target, while suppressing other irrelevant features, thereby enhancing the expressive power of the network while maintaining a satisfactory speed.

[0114] The beneficial effects of this application are as follows: the single-target tracking algorithm of this application can perform high-precision tracking of a single target under extreme environments such as pose changes, partial occlusion, and background blur. At the same time, it can maintain a speed of 40FPS on the RTX3060, demonstrating the robustness and high speed lower limit of the single-target tracking of this application.

[0115] like Figure 5 As shown, Figure 5 A schematic block diagram of a rotational attention single-tracking target tracking device based on a deep neural network, provided for embodiments of this application. The device includes:

[0116] The feature extraction module is used to extract features from the current frame image and features from the template image including the tracked target;

[0117] A rotational attention module is used to fuse the extracted features and extract the features of the current frame image again through a preset decoder;

[0118] The sparse attention module is used to fuse the features fused by the rotation attention module and the features extracted from the current frame image again to obtain the target features;

[0119] A dual-head classification and regression network is used to perform classification and regression on the target features to obtain classification and regression results.

[0120] A determination module is used to determine the position of the tracked target in the current frame image based on the classification result and the regression result.

[0121] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described device and its modules and units can be referred to the corresponding processes in the foregoing embodiments, and will not be repeated here.

[0122] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0123] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A rotational attention single-target tracking method based on deep neural networks, characterized in that, include: Extract features from the current frame image and features from the template image, including the tracked target; The extracted features are fused based on the rotation attention module, and the features of the current frame image are extracted again through a preset decoder; Based on the sparse attention module, the features fused by the rotation attention module and the features extracted from the current frame image are fused again to obtain the target features; The target features are classified and regressed to obtain classification and regression results; The location of the tracked target in the current frame image is determined based on the classification results and the regression results; The process involves fusing the extracted features based on the rotation attention module and then extracting the features of the current frame image again using a preset decoder, including: For the current frame image feature map and the template image feature map The input vector for the rotation attention module is obtained by performing a linear transformation along the dimensions of image height H and image width W. The linear transformation process is as follows: Where M is the transformation matrix, Let be the first input vector of the rotation attention module. This is the second input vector of the rotation attention module. Let P be the third input vector of the rotation attention module, and let P be a linear transformation function used to transform the current frame image feature map. and template image feature map Divide the window into four non-overlapping sections along the W and H dimensions; The first input vector of the rotation attention module and the third input vector of the rotation attention module Matrix multiplication is performed along the dimensions N and C / h of the feature map, and the feature relationship matrix between the features of the current frame image and the features of the template image is calculated using a normalization function. Where N = W × H; The feature relationship matrix and the second input vector Perform matrix multiplication to obtain the long-term dependency between the features of the current frame image and the features of the template image; Based on the long-term dependency relationship, features related to the tracking target are enhanced, while features unrelated to the tracking target are suppressed, resulting in a feature map fused by the rotational attention module. ; The computation of the sparse attention module includes: Feature map after fusion of rotational attention modules Input into the classification header to obtain the classification result. in ; According to the largest front Each classification result generates an index i, where ; Based on index i, the feature map of the current frame image The selection process yields the input to the sparse attention module. : =Attention( , , ) in, This is the first input vector of the sparse attention module. This serves as the second input direction for the sparse attention module. This is the first input vector of the sparse attention module. Features extracted by the rotational attention module are fused together. The features extracted again by the decoder.

2. The rotational attention single-target tracking method based on deep neural networks as described in claim 1, characterized in that, The extraction of features from the current frame image and features from the template image including the tracked target includes: The tracking target features and background features in the template image are extracted using a preset Swin-Transformer deep neural network, and the corresponding template image feature map is obtained. The tracking target features and background features in the current frame image are extracted using the Swin-Transformer deep neural network, and the corresponding current frame image feature map is obtained. The Swin-Transformer deep neural network includes: The first-stage feature extraction network consists of: one linear transformation unit, two first basic units, and one merging unit connected in sequence. The second-stage feature extraction network includes: two first basic units and one merging unit connected in sequence; The third-stage feature extraction network includes: four first basic units connected in sequence; The linear transformation unit is used to transform each element in the template image and the current frame image. The scale region is mapped to a 96-dimensional vector. The first basic unit includes a 1-layer attention mechanism with skip links and a 1-layer multilayer perceptron. The merging unit includes a step size of 2. The vector consists of a convolution and a normalization unit, the merging unit being used to increase the dimension of the vector.

3. The rotational attention single-target tracking method based on deep neural networks as described in claim 2, characterized in that, The rotational attention module includes: The first-stage rotating attention module includes four second basic units and one merging unit. The second stage rotating attention module consists of four second basic units; The second basic unit includes: four rotational attention mechanisms with skip connections and one multilayer perceptron, as well as a three-layer classification head, which includes three multilayer perceptrons and is activated by the ReLU function; The decoder consists of window self-attention, and the decoder includes: The first-stage decoding network comprises four of the first basic units, one batch normalization unit, and one step-size unit. convolution; The second-stage decoding network comprises four of the first basic units; Both the first-stage decoding network and the second-stage decoding network are connected to the rotating attention module through sparse attention.

4. The rotational attention single-target tracking method based on deep neural networks as described in claim 1, characterized in that, The sparse attention module includes: A sparse attention mechanism and a multilayer perceptron are used for jump links. The sparse attention module is connected to the decoding networks of the rotation attention module at each stage through sparse attention.

5. The rotational attention single-target tracking method based on deep neural networks as described in claim 4, characterized in that, The classification and regression of the target features to obtain classification and regression results includes: The target features are classified and regressed using a dual-head classification and regression network; The dual-head classification and regression network includes one convolutional regression head and one classification regression head; The convolutional regression head consists of 3 layers with a stride of 1. A convolutional layer consists of a convolutional module composed of a batch-normalized single clique and a ReLU activation function; The classification regression head consists of a 3-layer multilayer perceptron and a ReLU activation function.

6. The rotational attention single-target tracking method based on deep neural networks as described in claim 1, characterized in that, Determining the location of the tracked target in the current frame image based on the classification result and the regression result includes: The classification results were enhanced using a Hanning window. The location of the tracking target in the current frame image is determined based on the regression result corresponding to the classification result with the highest enhanced score.

7. The rotational attention single-target tracking method based on deep neural networks as described in claim 1, characterized in that, Before extracting features from the current frame image and features from the template image including the tracked target, the process also includes: The template image and the current frame image are segmented and scaled to the corresponding preset scale; The template image, after being cut and scaled, includes the tracking target and the background.

8. A rotational attention single-target tracking device based on a deep neural network, characterized in that, include: The feature extraction module is used to extract features from the current frame image and features from the template image including the tracked target; A rotational attention module is used to fuse the extracted features and extract the features of the current frame image again through a preset decoder; The sparse attention module is used to fuse the features fused by the rotation attention module and the features extracted from the current frame image again to obtain the target features; A dual-head classification and regression network is used to perform classification and regression on the target features to obtain classification and regression results. A determination module is used to determine the position of the tracked target in the current frame image based on the classification result and the regression result; The rotational attention module is also used for: For the current frame image feature map and the template image feature map The input vector for the rotation attention module is obtained by performing a linear transformation along the dimensions of image height H and image width W. The linear transformation process is as follows: Where M is the transformation matrix, Let be the first input vector of the rotation attention module. This is the second input vector of the rotation attention module. Let P be the third input vector of the rotation attention module, and let P be a linear transformation function used to transform the current frame image feature map. and template image feature map Divide the window into four non-overlapping sections along the W and H dimensions; The first input vector of the rotation attention module and the third input vector of the rotation attention module Matrix multiplication is performed along the dimensions N and C / h of the feature map, and the feature relationship matrix between the features of the current frame image and the features of the template image is calculated using a normalization function. Where N = W × H; The feature relationship matrix and the second input vector Perform matrix multiplication to obtain the long-term dependency between the features of the current frame image and the features of the template image; Based on the long-term dependency relationship, features related to the tracking target are enhanced, while features unrelated to the tracking target are suppressed, resulting in a feature map fused by the rotational attention module. ; The sparse attention module is further used for: Feature map after fusion of rotational attention modules Input into the classification header to obtain the classification result. in ; According to the largest front Each classification result generates an index i, where ; Based on index i, the feature map of the current frame image The selection process yields the input to the sparse attention module. : =Attention( , , ) in, This is the first input vector of the sparse attention module. This serves as the second input direction for the sparse attention module. This is the first input vector of the sparse attention module. Features extracted by the rotational attention module are fused together. The features extracted again by the decoder.

Images