Single target tracking method and device based on global hole attention and target focusing
Through the global hollow attention and target focus network, the robustness problem of the single-object tracking method in complex scenarios is solved, the goal embedding is enhanced and background interference is eliminated, and the tracking accuracy is improved.
Patent Information
- Application Number
- CN202310371679.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-10
- Publication Date
- 2025-08-19
- Estimated Expiration
- 2043-04-10
AI Technical Summary
The existing single-objective tracking methods are not robust enough to deal with rapid deformation, occlusion and background clutter, and self-attention methods are difficult to effectively enhance target embedding and are prone to noise introduction.
The global hollow attention and target focus network are adopted, and the feature semantics are enhanced through the local focus module, hollow attention module and channel adaptive module. The target focus network is combined with the target focus network to establish a point-to-point correlation between the template and the search area, and the target area prediction head is used to predict the target area.
It improves the robustness of target tracking, effectively enhances target embedding, weakens background interference, and realizes accurate tracking in complex scenarios.
Smart Images
Figure CN116596961B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a single target tracking method and device based on global void attention and target focusing. Background Art
[0002] A fundamental task in single-target computer vision, which aims to predict the state of a target given an initial state in a video sequence, has been widely used in various applications such as visual surveillance and autonomous driving. However, despite many efforts in recent years, developing a robust and accurate tracker remains challenging due to various obstacles that often occur in tracking, such as rapid deformation, occlusion, and background clutter. The existing methods introduce self-attention, which has been widely used in visual tracking due to its excellent ability to establish long-range dependencies. Self-attention is introduced into the field of single-target tracking to perform global semantic modeling and achieve feature fusion between the template and the search region to replace the cross-correlation operation. However, these self-attention-based methods still have the following two typical problems: (1) The global receptive field of self-attention pays less attention to local structural information and ignores the correlation between channels, which limits its ability to effectively distinguish the semantics of the object and the background. (2) The existing methods use the features of the search region as the query, perform linear processing on the template features, and use feature fusion to enhance the target embedding. This process is difficult to effectively enhance the target embedding and is prone to introduce noise from the search region. Summary of the Invention
[0003] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology, provide a single target tracking method and device based on global void attention and target focus, construct a new global semantic modeling method to enhance semantic objects while eliminating background information interference, and realize partial-level enhancement of target features in the search area, effectively enhancing target embedding while weakening non-target objects.
[0004] In order to achieve the above object, the present invention adopts the following technical solutions:
[0005] In a first aspect, the present invention provides a single target tracking method based on global hole attention and target focusing, comprising the following steps:
[0006] Crop the labeled single target tracking dataset and combine the cropped single target tracking dataset into a "template image-search image" image pair for training;
[0007] Perform feature extraction on the template image and the search image, and output their respective features after feature extraction through a deep convolutional neural network to obtain the template image features and the search image features;
[0008] A global target focusing network based on global hole attention and target focusing network is constructed; the global hole attention is used to effectively enhance the features of the template image and the search image to enhance the feature semantics in the spatial and channel dimensions, including a local focusing module, a hole attention module and a channel adaptation module. The local focusing module is used to construct local structural information and local dependencies, and the hole attention module is used to construct long-range dependencies of features; the target focusing network is used to establish point-to-point associations between the template image features and the search image features, transfer information from the template to the search area, and further enhance the features of the target;
[0009] The enhanced target features are input into the cross-correlation based prediction head to obtain the tracking target region.
[0010] As a preferred technical solution, the template image and the search image are both from the same video sequence. The template image comes from the first frame of the video, and the target area to be tracked is marked in the first frame image. The search image comes from the remaining frames except the first frame, and includes a large search area including the target to be tracked.
[0011] As a preferred technical solution, the global hole attention GDA is defined as follows:
[0012] F output =F input ·C(D(L(F input )))
[0013] Among them F input and F output are the input and output of the global hole attention GDA, L is the local focus module, D is the hole attention module, and C is the channel adaptation module.
[0014] As a preferred technical solution, the local focusing module uses deep convolution to construct local structural information and local dependencies, specifically:
[0015] Use deep convolution to capture local structural information and local dependencies in feature maps. Channels and convolution kernels have a one-to-one correspondence, thus enhancing the local focusing ability of the network.
[0016] The channel adaptation module uses 1×1 convolution to achieve information exchange between channels and uses semantic adaptive enhancement to enhance the feature expression of the channels.
[0017] As a preferred technical solution, the long-range dependency relationship of the features constructed by the hole attention module is specifically as follows:
[0018] First, the feature map is sampled at intervals through hole segmentation to obtain i groups, and then for each group Xi Perform multi-head self-attention operations to build long-range dependencies of features. In multi-head attention, one self-attention constitutes the calculation of one head and is the key operator for multi-head attention to build long-range dependencies of features. Finally, the i groups are spliced together; the output sequence of the multi-head attention is processed using a feedforward network to strengthen the information exchange within the elements themselves; finally, addition and normalization operations are used to obtain a robust feature image for subsequent operations. The self-attention mechanism is defined as follows:
[0019]
[0020] The multi-head self-attention mechanism is defined as follows:
[0021] MSAM(X i )=X i +FFN(MSA(X i ))
[0022] Among them, X i represents the input vector of self-attention, They represent the linear operations for generating query value, key value, and value, respectively. x Represents the dimension of the key value, self-attention is achieved through The query value and the key value are matrix multiplied to build long-range dependency relationships of features, so that distant elements are related to each other. Represents the mapping of each element itself so that it is changed when it cannot be associated with other elements. FFN represents feedforward neural network.
[0023] As a preferred technical solution, the target focusing network uses the template as the query value Q, the search area as the key value K and the value V, and transfers the target information from the template to the search area to establish a point-to-point similarity matrix between the target and the search area. The point-to-point similarity matrix S is as follows:
[0024]
[0025] Among them, d k Represents the dimension of the key value, Q T represents the transposed matrix of the query value Q;
[0026] Then, the probability map P is obtained by summing up the columns in the similarity matrix, obtaining the similarity between each point in the search area feature and the entire target feature, and further realizing partial-level target focusing. The probability map is defined as follows:
[0027]
[0028] Among them, S i,jRepresents the point-to-point similarity matrix, i represents the point in the search image feature, j represents the point in the template image feature, n t Represents the product of the length and width of the template image feature.
[0029] As a preferred technical solution, the cross-correlation-based prediction head is implemented by a cross-correlation operation, which is to use the template features as a convolution kernel to perform a convolution operation on the output of the above-mentioned target focusing network, so as to obtain the accurate target area.
[0030] In a second aspect, the present invention discloses a single target tracking system based on global void attention and target focusing network, which is applied to the single target tracking method based on global void attention and target focusing, including a data set preprocessing module, a feature extraction network module, a global target focusing network module and a prediction head module;
[0031] The dataset preprocessing module is used to perform a cropping operation on the annotated single target tracking dataset and combine the cropped single target tracking dataset into an image pair format of "template image-search image" for training;
[0032] The feature extraction network module is used to extract features from the template image and the search image, and output their respective features after feature extraction through a deep convolutional neural network to obtain template image features and search image features;
[0033] The global target focusing network module is used to construct a global target focusing network based on the global void attention and target focusing network; the global void attention is used to effectively enhance the features of the template image and the search image to enhance the feature semantics in the spatial and channel dimensions, including a local focusing module, a void attention module and a channel adaptation module. The local focusing module is used to construct local structural information and local dependencies, and the void attention module is used to construct long-range dependencies of features; the target focusing network is used to establish a point-to-point association between the template and the search area, transfer information from the template to the search area, and further enhance the characteristics of the target;
[0034] The prediction head module is used to input the enhanced target features into the cross-correlation-based prediction head to obtain the tracking target area.
[0035] In a third aspect, the present invention provides an electronic device, comprising:
[0036] at least one processor; and,
[0037] a memory communicatively connected to the at least one processor; wherein,
[0038] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to perform the single target tracking method based on global hole attention and target focusing.
[0039] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, characterized in that when the program is executed by a processor, the single target tracking method based on global void attention and target focus is implemented.
[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0041] 1. This paper defines global void attention to provide global semantic modeling to enhance the semantics of features. It greatly enhances the representation of semantic objects, eliminates the interference of non-semantic background, and improves the robustness of the tracker.
[0042] 2. This paper defines a global object focusing network to build point-to-point associations between templates and search regions. It effectively distinguishes target and non-target objects with rich semantics and further successfully achieves part-level object embedding enhancement based on the semantic enhancement feature of global hole attention.
[0043] 3. We propose a new tracking method, named Single Object Tracking with Global Hollow Attention and Target Focusing, by using a global hole attention and target focus network. Experiments on challenging benchmarks (LaSOT, TrackingNet, GOT-10k, OTB-100) demonstrate that our single object tracking method outperforms most state-of-the-art tracking methods and achieves leading performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0045] Figure 1 is a flow chart of a single target tracking method based on global hole attention and target focusing according to an embodiment of the present invention;
[0046] Figure 2 Schematic diagram of a network based on global void attention and target focus according to an embodiment of the present invention;
[0047] Figure 3Schematic diagram of global hole attention in an embodiment of the present invention;
[0048] Figure 4 is a schematic diagram of a target-focused network according to an embodiment of the present invention;
[0049] Figure 5 is a comparison diagram of a target-focused network and a conventional feature fusion network in an embodiment of the present invention;
[0050] Figure 6 This is a comparison chart of the effects of the present invention and several advanced single target tracking algorithms;
[0051] Figure 7 Schematic diagram of the structure of a single target tracking system based on global void attention and target focus according to an embodiment of the present invention;
[0052] Figure 8 2 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0053] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0054] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.
[0055] See also Figure 1 In this embodiment, a single target tracking method based on global hole attention and target focus includes the following steps:
[0056] S1. Separate a template image and a search image from the same video sequence. The template image is from the first frame of the video, which contains the target area to be tracked. The search image is from the remaining frames except the first frame, which contains a large search area including the target to be tracked.
[0057] S2. Combine the cropped single target tracking dataset into a “template image-search image” image pair for training.
[0058] In one embodiment of the present application, the size of the cropped image is set, the template image is set to 127×127, and the search image is set to 289×289.
[0059] S3. Use a deep convolutional neural network, i.e., use GoogLeNet (Inveption v3) to extract features of the template image and the search image respectively, and obtain the template image features and the search image features respectively.
[0060] See also Figure 2 The two trapezoids in the feature extraction network represent the same pre-trained deep neural network, GoogLeNet (Inveption v3), which is a 22-layer convolutional neural network stacked using the improved Inception structure. Figure 2 The shared parameters in the above sentence refer to the following: the two pre-trained deep neural networks need to fine-tune the parameters during the feature extraction process of the template image and the search image. The shared parameters can better learn the common features of the template image and the search image. Figure 2 The details of the global hole attention and target focus network in Figure 3 and Figure 4 Presented in.
[0061] It can be understood that the above-mentioned GoogLeNet (Inveption v3) is a feature extraction network in this embodiment, and other feature extraction networks suitable for this application are also applicable.
[0062] S4. Effectively enhance the template image features and search image features respectively to enhance the feature semantic information in the spatial and channel dimensions.
[0063] Furthermore, step S4 is specifically as follows:
[0064] S41. Global void attention is adopted, which consists of a local focus module, a void attention module and a channel adaptation module.
[0065] The global hole attention is defined as follows:
[0066] F output =F input ·C(D(L(F input ))) (1)
[0067] Among them F input and F output are the input and output of GDA, L is the local focus module, D is the hole attention module, and C is the channel adaptation module;
[0068] S42. Subsequently, a local focusing module is used to construct local structural information and local dependencies.
[0069] Furthermore, in this embodiment, the local focusing module specifically uses deep convolution to capture local structural information and local dependencies in the feature map, and the channels and convolution kernels have a one-to-one correspondence, thereby enhancing the local focusing ability of the network.
[0070] S43. Using hollow attention to construct long-range dependencies of features and establish semantic connections between different locations in the image can better capture global dependencies.
[0071] Further, Figure 3 represents the global hole attention module in the present invention, Indicates that the elements are accumulated pixel by pixel; Indicates matrix multiplication of two matrices; Denotes positional encoding; depthwise convolution uses 3×3 grouped convolution with 256 groups. Compared to ordinary convolution, depthwise convolution requires fewer parameters. It changes the previous ordinary convolution operation to consider both channels and regions at the same time, first considering only regions and then channels, thus achieving channel-region separation. The activation function uses the rectified linear unit (ReLU). Figure 3 In the hole segmentation, different hole coefficients are set for the template image features and the search image features. The hole coefficient can control the sampling step size, thereby achieving the effect of focusing on local information. The hole coefficient is set to 2 for the template image features and 3 for the search image features. The feedforward network is a fully connected neural network.
[0072] Furthermore, the hole attention module consists of the following three steps: first, the feature map is sampled at intervals through hole segmentation to obtain i groups, and then for each group X i Perform multi-head self-attention operations to build long-range dependencies of features. In multi-head attention, one self-attention constitutes the calculation of one head and is the key operator for multi-head attention to build long-range dependencies of features. Finally, the i groups are spliced together; the output sequence of the multi-head attention is processed using a feedforward network to strengthen the information exchange within the elements themselves; finally, addition and normalization operations are used to obtain a robust feature image for subsequent operations. The self-attention mechanism is defined as follows:
[0073]
[0074] The multi-head self-attention mechanism is defined as follows:
[0075] MSAM(X i )=X i+FFN(MSA(X i )) (3)
[0076] Among them, X i represents the input vector of self-attention, They represent the linear operations for generating query value, key value, and value, respectively. x Represents the dimension of the key value, self-attention is achieved through The query value and the key value are matrix multiplied to build long-range dependency relationships of features, so that distant elements are related to each other. A map that represents each element itself, allowing it to be changed when no association with other elements can be established.
[0077] In one embodiment of the present application, the present invention trains for 20 epochs on an RTX3090, setting the initial step size to 0.0001 and the number of heads in the multi-head self-attention to 8. The present invention uses 600,000 image pairs for training, and sets the size of the cropped images to 127×127 for the template image and 289×289 for the search image.
[0078] S44. Furthermore, the channel adaptation module uses 1×1 convolution to implement information exchange between channels and uses semantics to adaptively enhance the feature expression of the channel to output the most critical feature expression in the template image and the search image, providing the most robust feature information for subsequent steps;
[0079] S5. Using the target focus network, the template is used as the query value Q, the search area is used as the key value K and the value V, and the target information is transferred from the template to the search area to establish a point-to-point similarity matrix between them, so as to establish a point-to-point association between the template and the search area. The point-to-point similarity matrix is as follows:
[0080]
[0081] Among them, d k Represents the dimension of the key value.
[0082] It is understandable that Figure 4 middle It means to perform matrix multiplication on two matrices; skip operation means to copy the matrix and skip the rest of the intermediate processing, thus retaining the original information and avoiding information loss before performing subsequent operations.
[0083] Then, the probability map is obtained by summing up the columns (along the dimension representing the template) in the similarity matrix to obtain the similarity between each point in the search area feature and the entire target feature, and further achieve partial-level target focusing. The probability map is defined as follows:
[0084]
[0085] Figure 5 A comparison diagram of the target-focused network in the present invention and the conventional feature fusion network. Figure 5 The left side of the figure shows the structure of the conventional feature fusion network, and the right side shows the structure of the target focus network. Figure 5 Q, k, and V in the figure are feature vectors. On the left, the search area is used as the query value Q, the template is used as the key value k, and the value V; on the right, the template is used as the query value Q, the search area is used as the key value K, and the value V.
[0086] S6. Use a cross-correlation-based prediction head, which is composed of a cross-correlation operation. The cross-correlation operation is to use the template feature as a convolution kernel to perform a convolution operation on the output of the above-mentioned target focusing network, so as to obtain the accurate target area.
[0087] See also Figure 6 , showing the objective analysis results of the feature map of the method of the present invention. The first column is the real labeling of the target search area; the second column is the features after feature network extraction, which is subjectively very messy and disordered; the third column is the features after global attention processing, which has excluded some background information subjectively; the fourth column is the features after target focus network processing, which is subjectively very focused on the target itself, almost the same as the real labeling.
[0088] The method of the present invention can better cope with tracking scenarios under complex conditions, fully eliminate background information interference, and exclude the influence of non-targets, so that the tracker can focus more on the target itself rather than background information or similar targets.
[0089] The objective evaluation results of the method are shown in Table 1. The present invention uses the IOU value (AUC) between the real bounding box and the bounding box generated by the tracker, the normalized precision (P Norm ) and precision (P) are used as evaluation metrics for the LaSOT and TrackingNet datasets, using the average overlap (AO). The proportion of successfully tracked frames with an overlap of more than 0.5 (SR 0.5 ) and the ratio of successfully tracked frames with overlap greater than 0.75 (SR 0.75 ), the higher the value, the more accurate the tracker is in tracking the target.
[0090] Table 1
[0091]
[0092] The present invention proposes a robust single target tracking method based on global void attention and target focus by defining global void attention and target focus networks. Through the proposed global void attention, comprehensive feature semantic extraction from the spatial dimension and channel dimension is achieved. Using the proposed void attention module, it performs void sampling on the feature map, divides it into different groups, and then establishes global and sparse long-range dependencies in these independent groups to enhance the semantics of all objects. In addition, local structural information and local dependencies are constructed through the proposed local focus module, and inter-channel associations are established through the proposed channel adaptation module to adaptively select and enhance channels with rich semantics and eliminate background information interference. And through the proposed target focus network, a point-to-point association between the template and the search area is established to achieve partial-level enhancement of the target features in the search area, effectively enhancing the target embedding while weakening non-target objects, thereby achieving robust target tracking.
[0093] See also Figure 7 In another embodiment of the present application, a single target tracking system 100 based on global hole attention and target focus is provided, including a data set preprocessing module 101, a feature extraction network module 102, a global target focus network module 103 and a prediction head module 104; the video sequence is processed by the data set preprocessing module to obtain a template image and a search image; then, the template image and the search image are respectively input into the feature extraction network module, and the template image features and the search image features are extracted using a pre-trained deep convolutional neural network; further, the template image features and the search image features are input into the global target focus network module, and the template image features and the search image features are enhanced using global hole attention to obtain robust template image features and search image features, and then, the target focus network is used to perform feature fusion on the two to establish a point-to-point association between the template and the search area; finally, the template image features and the search image features are input into the global target focus network module, and the template image features and the search image features are enhanced using global hole attention to obtain robust template image features and search image features, and then, the target focus network is used to perform feature fusion on the two to establish a point-to-point association between the template and the search area; finally, the template image features and the search image features are input into the global target focus network module, and the template image features and the search image features are enhanced using global hole attention to obtain robust template image features and search image features.
[0094] The dataset preprocessing module 101 is used to perform a cropping operation on the labeled single target tracking dataset and combine the cropped single target tracking dataset into an image pair format of "template image-search image" for training;
[0095] The feature extraction network module 102 is used to extract features from the template image and the search image, and output their respective features after feature extraction using a deep convolutional neural network to obtain template features and search features;
[0096] The global target focusing network module 103 uses global void attention, which is composed of a local focusing module, a void attention module, and a channel adaptation module to effectively enhance the features of the template and search area to enhance the feature semantics in the spatial and channel dimensions. The local focusing module is used to construct local structural information and local dependencies, the void attention is used to construct long-range dependencies of features, and the channel adaptation module is used to construct inter-channel associations. The target focusing network is used to establish point-to-point associations between the template and the search area, transferring information from the template to the search area to further enhance the characteristics of the target.
[0097] The prediction head module 104 is configured to input the enhanced target features into a cross-correlation-based prediction head to generate a target region.
[0098] It should be noted that the single target tracking system based on global void attention and target focusing network of the present invention corresponds one to one with the single target tracking method based on global void attention and target focusing of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the single target tracking method based on global void attention and target focusing are applicable to the embodiment of the single target tracking based on global void attention and target focusing network. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.
[0099] In addition, in the implementation of the single target tracking system based on global void attention and target focusing network in the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the single target tracking system based on global void attention and target focusing network is divided into different program modules to complete all or part of the functions described above.
[0100] See also Figure 8 In one embodiment, an electronic device for implementing a single target method based on global hole attention and target focusing network is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a single target tracking program 203 based on global hole attention and target focusing network.
[0101] The first memory 202 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the first memory 202 may be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 may also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 200. Furthermore, the first memory 202 may include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can be used not only to store application software and various data installed on the electronic device 200, such as the code of the single target tracking program 203 based on global void attention and target focus network, but also to temporarily store data that has been output or is about to be output.
[0102] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 201 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the first memory 202, as well as calling data stored in the first memory 202, to perform various functions of the electronic device 200 and process data.
[0103] Figure 8 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 8 The structure shown does not constitute a limitation on the electronic device 200 , and the electronic device 200 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0104] The single target tracking program 203 based on global hole attention and target focus network stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve:
[0105] Crop the labeled single target tracking dataset and combine the cropped single target tracking dataset into a "template image-search image" image pair for training;
[0106] Perform feature extraction on the template image and the search image, and output their respective features after feature extraction through a deep convolutional neural network to obtain template features and search features;
[0107] A global target focusing network is constructed, using global void attention. It consists of a local focusing module, a void attention module, and a channel adaptation module to effectively enhance the features of the template and search area to enhance the feature semantics in the spatial and channel dimensions. The local focusing module is used to construct local structural information and local dependencies, the void attention module is used to construct long-range dependencies of features, and the channel adaptation module is used to build inter-channel associations. The target focusing network is used to establish point-to-point associations between the template and the search area, transferring information from the template to the search area to further enhance the characteristics of the target.
[0108] The enhanced target features are fed into the cross-correlation based prediction head to generate the target region.
[0109] Furthermore, if the modules / units integrated in the electronic device 200 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0110] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0111] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0112] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.
Claims
1. Single target tracking method based on global hole attention and target focus, characterized by: The steps include: Crop the labeled single target tracking dataset and combine the cropped single target tracking dataset into a "template image-search image" image pair for training; Perform feature extraction on the template image and the search image, and output their respective features after feature extraction through a deep convolutional neural network to obtain the template image features and the search image features; Construct a global target focusing network based on global void attention and target focusing network; The global hole attention is used to effectively enhance the features of the template image and the search image to enhance the feature semantics in the spatial and channel dimensions. It includes a local focus module, a hole attention module, and a channel adaptation module. The local focus module is used to construct local structural information and local dependencies, and the hole attention module is used to construct long-range dependencies of features. The target focus network is used to establish point-to-point associations between the template image features and the search image features, transferring information from the template to the search area to further enhance the features of the target. The enhanced target features are input into the cross-correlation based prediction head to obtain the tracking target region.
2. The single target tracking method based on global hole attention and target focus according to claim 1, characterized in that: The template image and the search image are both from the same video sequence. The template image comes from the first frame of the video, and the target area to be tracked is marked in the first frame image. The search image comes from the remaining frames except the first frame, and includes a large search area including the target to be tracked.
3. The single target tracking method based on global hole attention and target focus according to claim 1, characterized in that: The global hole attention GDA is defined as follows: F output =F input ·C(D(L(F input ))) Among them F input and F output are the input and output of the global hole attention GDA, L is the local focus module, D is the hole attention module, and C is the channel adaptation module.
4. The single target tracking method based on global hole attention and target focusing according to claim 1, characterized in that: The local focusing module uses deep convolution to construct local structural information and local dependencies, specifically: Use deep convolution to capture local structural information and local dependencies in feature maps. Channels and convolution kernels have a one-to-one correspondence, thus enhancing the local focusing ability of the network. The channel adaptation module uses 1×1 convolution to achieve information exchange between channels and uses semantic adaptive enhancement to enhance the feature expression of the channels.
5. The single target tracking method based on global hole attention and target focus according to claim 1, characterized in that: The long-range dependency relationship of the features constructed by the hole attention module is specifically: First, the feature map is sampled at intervals through hole segmentation to obtain i groups, and then for each group X i Perform multi-head self-attention operations to build long-range dependencies of features. In multi-head attention, one self-attention constitutes the calculation of one head and is the key operator for multi-head attention to build long-range dependencies of features. Finally, the i groups are spliced together; the output sequence of the multi-head attention is processed using a feedforward network to strengthen the information exchange within the elements themselves; finally, addition and normalization operations are used to obtain a robust feature image for subsequent operations. The self-attention mechanism is defined as follows: The multi-head self-attention mechanism is defined as follows: MSAM(X i )=X i +FFN(MSA(X i )) Among them, X i represents the input vector of self-attention, They represent the linear operations for generating query value, key value, and value, respectively. x Represents the dimension of the key value, self-attention is achieved through The query value and the key value are matrix multiplied to build long-range dependency relationships of features, so that distant elements are related to each other. Represents the mapping of each element itself so that it is changed when it cannot be associated with other elements. FFN represents feedforward neural network.
6. The single target tracking method based on global hole attention and target focus according to claim 1, characterized in that: The target focusing network uses the template as the query value Q, the search area as the key value K and the value V, and transfers the target information from the template to the search area to establish a point-to-point similarity matrix between the target and the search area. The point-to-point similarity matrix S is as follows: Among them, d k Represents the dimension of the key value, Q T represents the transposed matrix of the query value Q; Then, the probability map P is obtained by summing up the columns in the similarity matrix, obtaining the similarity between each point in the search area feature and the entire target feature, and further realizing partial-level target focusing. The probability map is defined as follows: Among them, S i,j Represents the point-to-point similarity matrix, i represents the point in the search image feature, j represents the point in the template image feature, n t Represents the product of the length and width of the template image feature.
7. The single target tracking method based on global hole attention and target focusing according to claim 1, characterized in that: The cross-correlation-based prediction head is implemented by a cross-correlation operation, which is to use the template features as a convolution kernel to perform a convolution operation on the output of the above-mentioned target focusing network, so as to obtain the accurate target area.
8. Single target tracking system based on global void attention and target focus network, characterized by: A single target tracking method based on global hole attention and target focusing applied to any one of claims 1-7, comprising a data set preprocessing module, a feature extraction network module, a global target focusing network module, and a prediction head module; The dataset preprocessing module is used to perform a cropping operation on the labeled single target tracking dataset and combine the cropped single target tracking dataset into an image pair format of "template image-search image" for training; The feature extraction network module is used to extract features from the template image and the search image, and output their respective features after feature extraction through a deep convolutional neural network to obtain template image features and search image features; The global target focusing network module is used to construct a global target focusing network based on global void attention and target focusing network; The global hole attention is used to effectively enhance the features of the template image and the search image to enhance the feature semantics in the spatial and channel dimensions. It includes a local focus module, a hole attention module, and a channel adaptation module. The local focus module is used to construct local structural information and local dependencies, and the hole attention module is used to construct long-range dependencies of features. The target focus network is used to establish a point-to-point association between the template and the search area, transfer information from the template to the search area, and further enhance the features of the target. The prediction head module is used to input the enhanced target features into the cross-correlation-based prediction head to obtain the tracking target area.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the single target tracking method based on global hole attention and target focusing as described in any one of claims 1-7.
10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the single target tracking method based on global hole attention and target focus according to any one of claims 1 to 7 is implemented.