Visual target tracking methods, systems and devices, and computer-readable storage media

Abstract

This disclosure belongs to the field of computer vision, specifically relating to visual target tracking methods, systems and devices, and computer-readable storage media; it uses the similarity between the foreground-background distribution of features in the template and search region for feature aggregation and learning; it effectively suppresses the influence of similar-looking backgrounds on foreground targets, significantly improving the tracking performance of the tracking method under complex backgrounds and similar-looking interference objects; it can be applied to visual target tracking application systems in practical scenarios such as video surveillance, anti-drone tracking, and autonomous driving, to locate any specified target in the video.

Description

Technical Field

[0001] This disclosure pertains to the field of computer vision, specifically relating to visual target tracking methods, systems and apparatuses, and computer-readable storage media. Background Technology

[0002] Visual object tracking aims to locate the position of an unidentified target within a video sequence, given a target in the first frame. This is a fundamental and essential research topic in computer vision. Due to its enormous application potential, such as deployments in video surveillance, anti-drone tracking, and autonomous driving, visual object tracking has attracted widespread attention and experienced significant development.

[0003] However, visual object tracking, as a video processing task, still faces significant challenges due to the characteristics of the tracked object, such as deformation, motion blur, and susceptibility to background interference. With the successful development of deformers in natural language processing, the potential of deformers' ability to capture long-range global feature information in object tracking is being rapidly explored. An increasing number of tracking methods utilize deformers' ability to capture global image feature information and perform global inference to model the dependencies of long-range relationships in image content or encode global target features across multiple frames. Therefore, these deformer-based tracking methods have achieved excellent performance.

[0004] However, tracking is essentially a binary classification task involving foreground and background, and the aforementioned deformer-based tracking paradigm uses a simple attention mechanism to establish information interaction between the template and the search region. Since the similarity score is obtained through appearance similarity, the features of the template and the search region are affected by complex backgrounds and similar-looking distractors, leading to the incorrect aggregation of background information. This results in limited ability of the search region's features to distinguish the target. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this disclosure is to provide a visual target tracking method, system, apparatus, and computer-readable storage medium, which solves the problem in existing technologies that the template and search area features are affected by complex backgrounds and similar-looking interference objects, leading to the erroneous aggregation of background information and thus limiting the ability of the search area features to distinguish targets.

[0006] Firstly, a visual target tracking method for foreground and background distribution recognition is provided. This visual target tracking method includes:

[0007] The system receives the image cropped from the first frame and the search region image cropped from the target position in the previous frame, wherein the image cropped from the first frame is used as a template and the search region image cropped from the target position in the previous frame is used as the search region feature map.

[0008] The learned labels are embedded in the small block sequence features corresponding to the template and search region feature maps, respectively, to generate template labels and search region labels.

[0009] The template marker and search region marker are input into multiple foreground distribution modeling converter blocks in the iterative process, and the last foreground distribution modeling converter block outputs the final search region marker;

[0010] The final search region markers are then converted into two dimensions. These two-dimensional markers are input into the target bounding box regression head for output. The output includes a classification score map, a local target offset, and the target bounding box size. The location with the highest score in the output classification score map is taken as the target, and the corresponding local target offset and target bounding box size determine the target location.

[0011] The foreground distribution modeling converter block includes:

[0012] Receive template markers and search region markers, aggregate the information inside or outside the target box corresponding to the template marker and the pseudo target box corresponding to the search region marker, and generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively;

[0013] Using the updated template foreground proxy, template background proxy, search region foreground proxy, and search region background proxy as the query and key, and aiming to maximize the similarity between the search region tag and template tag and the query and key, respectively, output the obtained template tag and search region tag.

[0014] In some publicly available methods, the template and search region feature maps are segmented into blocks to obtain template block sequence features and search region block sequence features, respectively. The template block sequence features and search region block sequence features are mapped to one dimension, and learned labels are embedded into the one-dimensional template block sequence features and search region block sequence features, respectively, to generate template labels and search region labels.

[0015] In some disclosures, the input quantities of the foreground distribution modeling converter block are all normalized. The input quantities include the template markers and search region markers input for the first time, as well as the template markers and search region markers processed by the foreground distribution modeling converter block.

[0016] In some publicly available methods, the acquisition of pseudo-target boxes corresponding to search region markers includes:

[0017] Receive template markers and search region markers, and perform two-dimensional transformations on the template markers and search region markers respectively to obtain a first template feature map and a first search region feature map. Use the target box corresponding to the template feature as a reference to obtain the target feature of the first template feature map.

[0018] For each point of the target feature in the first template feature map, obtain the corresponding matching point in the first search region feature map using a similarity matching method. Obtain a set of matching points from each point in the first template feature map to the first search region feature map. Based on the obtained matching points, obtain the pseudo target box corresponding to the search region marker.

[0019] In some publicly available methods, similarity matching involves obtaining a set of matching points from each point in the corresponding first template feature map to the first search region feature map based on probability D. l (k,i) represents the probability D. l (k,i),

[0020]

[0021] Where k and i are respectively and All two-dimensional positions; First template feature map; First search region feature map; s is the stride of the backbone network; Each point in A set of matching points Where K = h t w t h t w represents the height of the first template feature map. t x is the width of the first template feature map. i Let y be the x-coordinate of the matching point from the i-th region of the first template feature map to the first search region feature map. i The ordinate is the vertical coordinate of the matching point from the i-th region of the first template feature map to the feature map of the first search region.

[0022] In some publicly available methods, pseudo-target boxes corresponding to the search region markers are obtained based on the acquired matching points, including obtaining pseudo-target boxes of the search region based on the mean and standard deviation of the matching points.

[0023] In some public disclosures, information about the inside or outside of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker is aggregated separately to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively. This includes: using a decoder with a mask cross-attention mechanism to aggregate the foreground or background information of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker respectively to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively.

[0024] In some disclosures, the target bounding box regression head includes: a fully convolutional network, which includes N stacked convolutional layers per output - batch regularization - ReLU function layers.

[0025] Secondly, a visual target tracking system for foreground and background distribution recognition is provided, including:

[0026] The input module is used to receive the image cropped from the first frame and the search region image cropped from the target position in the previous frame, wherein the image cropped from the first frame is used as the template and the search region image cropped from the target position in the previous frame is used as the search region feature map.

[0027] The labeling module is used to embed the learned labels into the small block sequence features corresponding to the template and search region feature maps, respectively, to generate template labels and search region labels.

[0028] The processing module is used to input template tags and search region tags into multiple foreground distribution modeling converter blocks in the iterative processing, and the last foreground distribution modeling converter block outputs the final search region tags;

[0029] This is used to convert the final search region label into two dimensions. The two-dimensional final search region label is input into the target bounding box regression head for output. The output includes a classification score map, target local offset, and target bounding box size. The position with the highest score in the output classification score map is taken as the target, and the corresponding target local offset and target bounding box size determine the target location; wherein,

[0030] The foreground distribution modeling converter block includes:

[0031] The foreground-background proxy learning module receives template markers and search region markers, and aggregates information from the inside or outside of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker, respectively, to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy.

[0032] The distribution-aware attention module is used to take the updated template foreground proxy, template background proxy, search region foreground proxy, and search region background proxy as queries and keys, and aims to maximize the similarity between the search region tag and the template tag and the query and key, respectively, and output the obtained template tag and search region tag.

[0033] In some publicly available methods, the template and search region feature maps are segmented into blocks to obtain template block sequence features and search region block sequence features, respectively. The template block sequence features and search region block sequence features are mapped to one dimension, and learned labels are embedded into the one-dimensional template block sequence features and search region block sequence features, respectively, to generate template labels and search region labels.

[0034] In some disclosures, the input quantities of the foreground distribution modeling converter block are all normalized. The input quantities include the template markers and search region markers input for the first time, as well as the template markers and search region markers processed by the foreground distribution modeling converter block.

[0035] In some publicly available methods, the acquisition of pseudo-target boxes corresponding to search region markers includes:

[0036] Receive template markers and search region markers, and perform two-dimensional transformations on the template markers and search region markers respectively to obtain a first template feature map and a first search region feature map. Use the target box corresponding to the template feature as a reference to obtain the target feature of the first template feature map.

[0037] For each point of the target feature in the first template feature map, obtain the corresponding matching point in the first search region feature map using a similarity matching method. Obtain a set of matching points from each point in the first template feature map to the first search region feature map. Based on the obtained matching points, obtain the pseudo target box corresponding to the search region marker.

[0038] In some publicly available methods, similarity matching involves obtaining a set of matching points from each point in the corresponding first template feature map to the first search region feature map based on probability D. l (k,i) represents the probability D. l (k,i),

[0039]

[0040] Where k and i are respectively and All two-dimensional positions; First template feature map; First search region feature map; s is the stride of the backbone network; Each point in A set of matching points Where K = h t w t h t w represents the height of the first template feature map. t x is the width of the first template feature map. i Let y be the x-coordinate of the matching point from the i-th region of the first template feature map to the first search region feature map. i The ordinate is the vertical coordinate of the matching point from the i-th region of the first template feature map to the feature map of the first search region.

[0041] In some publicly available methods, pseudo-target boxes corresponding to the search region markers are obtained based on the acquired matching points, including obtaining pseudo-target boxes of the search region based on the mean and standard deviation of the matching points.

[0042] In some public disclosures, information about the inside or outside of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker is aggregated separately to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively. This includes: using a decoder with a mask cross-attention mechanism to aggregate the foreground or background information of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker respectively to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively.

[0043] In some disclosures, the target bounding box regression head includes: a fully convolutional network, which includes N stacked convolutional layers per output - batch regularization - ReLU function layers.

[0044] Thirdly, an image recognition device is provided, comprising: a processor and a memory;

[0045] The memory is used to store computer programs, the computer programs including program instructions;

[0046] The processor is used to invoke the computer program to implement the visual target tracking method as described in the first aspect.

[0047] Fourthly, a computer-readable storage medium is provided, on which instructions are stored, which, when executed by a processor, implement the visual target tracking method of the first aspect.

[0048] Beneficial effects of the present disclosure:

[0049] By calculating the similarity between the foreground and background distributions of the template and the search region features for feature aggregation and learning, the influence of similar-looking backgrounds on the foreground target is effectively suppressed, and the tracking performance of the tracking method under complex backgrounds and similar-looking interference is significantly improved. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0051] Figure 1 This is an overall method flowchart of an embodiment of this disclosure;

[0052] Figure 2 This is the overall tracking method framework of the embodiments of this disclosure;

[0053] Figure 3This is a schematic diagram of a foreground-background proxy decoder according to an embodiment of this disclosure;

[0054] Figure 4 This is a schematic diagram of the target bounding box regression head according to an embodiment of this disclosure. Detailed Implementation

[0055] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0056] like Figure 1 As shown, this invention proposes a visual target tracking method for foreground and background distribution recognition. The visual target tracking method includes:

[0057] The system receives the image cropped from the first frame and the search region image cropped from the target position in the previous frame, wherein the image cropped from the first frame is used as a template and the search region image cropped from the target position in the previous frame is used as the search region feature map.

[0058] The learned labels are embedded in the small block sequence features corresponding to the template and search region feature maps, respectively, to generate template labels and search region labels.

[0059] The template marker and search region marker are input into multiple foreground distribution modeling converter blocks in the iterative process, and the last foreground distribution modeling converter block outputs the final search region marker;

[0060] The final search region marker is converted into two dimensions. The two-dimensional final search region marker is input into the target bounding box regression head for output. The output includes a classification score map, target local offset, and target bounding box size. The position with the highest score in the output classification score map is taken as the target, and the corresponding target local offset and target bounding box size determine the target position.

[0061] By calculating the similarity between the foreground and background distributions of the template and the search region features for feature aggregation and learning, the influence of similar-looking backgrounds on the foreground target is effectively suppressed, and the tracking performance of the tracking method under complex backgrounds and similar-looking interference is significantly improved.

[0062] For example, the template and search region feature maps can be segmented into blocks to obtain template block sequence features and search region block sequence features, respectively. These features are then mapped to one dimension. Learned labels are embedded into these one-dimensional template block sequence features and search region sequence features, respectively, generating template labels and search region labels. Examples of possible solutions include:

[0063] Image cropped from the first frame And the search region image cropped based on the target position of the previous frame during tracking. As input.

[0064] Here, (H) z ×W z ) and (H x ×W x The numbers ) represent the size of the template and the feature map of the search region, respectively. First, they are divided into small blocks and flattened to obtain the template. and search area The small-block sequence features, where (p,p) is the resolution of the small block, and N z =(H z W z ) / p 2 and N x =(H x W x ) / p 2 These represent the number of small blocks into which the template and the search region are divided, respectively.

[0065] Similar to ordinary visual transformers, the template z p and search area x p Small-block sequence features are mapped to a C-dimensional dimension via linear projection and then combined with learnable positional embeddings to obtain template label features. and search area labeling features Then, and Connected together as input to the foreground distribution modeling converter block

[0066] Here, Represented as the tag feature output from the l-th converter block, where The template marking feature of the output of the l-th converter block, The search region is marked with features for the output of the l-th converter block.

[0067] Finally, for the labeled features output from the last block, the labeled features of the search region are rearranged. Shape The feature maps of the channel dimension C are input into the target bounding box regression head to estimate the target state.

[0068] The foreground distribution modeling converter block includes:

[0069] The foreground-background proxy learning module receives template markers and search region markers, and aggregates information from the inside or outside of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker, respectively, to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy.

[0070] The distribution-aware attention module is used to take the updated template foreground proxy, template background proxy, search region foreground proxy, and search region background proxy as queries and keys, and aims to maximize the similarity between the search region tag and the template tag and the query and key, respectively, and output the obtained template tag and search region tag.

[0071] For example: Foreground-background proxy learning module: given labeled features from the (l-1)th transformer block. They are normalized using layer normalization to obtain and These normalization features and The data is fed into the foreground-background proxy learning module to generate foreground-background proxies for the template and search region. The goal of this module is to generate foreground-background masks for the template and search region to provide prior foreground-background knowledge.

[0072] Specifically, such as Figure 2 As shown, the foreground-background proxy learning module includes a pseudo-target box generation strategy and a foreground-background proxy decoder.

[0073] First and Projected onto C e Dimensions obtained and Template features; The search region is characterized to reduce computational overhead. Then, a pseudo-boundary box generation strategy is employed to obtain pseudo-boundaries for the template and search region. Finally, foreground-background proxies for the template and search region can be obtained by aggregating information inside and outside the bounding boxes using a foreground-background proxy decoder. The pseudo-boundary box generation strategy and foreground-background proxy decoder will be described in detail below.

[0074] Pseudo-target bounding box generation strategy. Because the target bounding box b of the template... z The template is pre-defined; the challenge lies in generating pseudo-target boxes for the search region. To address this, the strategy establishes a coarse match between the template and the search region, which helps generate pseudo-target boxes for the search region.

[0075] Specifically, given and First, rearrange their shapes to obtain the template feature map. and search region feature map Then, based on the target box b given in the template... z Crop the target features from the template Here, (h) t ×w t () represents the size of the target feature map cropped from the template. Next, similarity matching is performed based on probability D. l for Calculate each point in The matching point in the matrix. The formula for obtaining the matching point is as follows:

[0076]

[0077]

[0078] Where k and i enumerate the following: and All two-dimensional positions are given. s is the stride of the backbone network. This allows us to obtain... Each point in A set of matching points Where K = h t w t Finally, based on the mean and standard deviation of all key points, the pseudo-target bounding box of the search region is derived. The formula is as follows:

[0079]

[0080]

[0081] Where, λ w and λ h It is a scaling factor parameter that adapts to any input target size, based on the size of the target template.

[0082] Foreground-background proxy decoder. In obtaining the template target box b... z and the pseudo-target bounding box of the search area Next, the goal of the foreground-background proxy decoder is to aggregate information from inside or outside the target bounding box. To do this, a set of foreground prototype proxies is first initialized. and background prototype proxy Foreground-background proxy decoder process details are as follows Figure 3 The decoder uses a masked cross-attention (M-CA) mechanism to aggregate. Corresponding target box b z The information of the target foreground is used to generate a template foreground proxy, as shown in the following formula:

[0083]

[0084]

[0085] Where Softmax is the Softmax function, W Q W K and W V These are linear projection parameters. And if the template foreground mask... The element in the target box b corresponds to the position in the target box b. z If the value is zero, the element is set to 0. Otherwise, it will be set to -∞. Similarly, for template background codes... If the position corresponding to one of the elements is within the target box b z Otherwise, the element will be set to 0. Otherwise, it will be set to -∞. This allows you to obtain the template background proxy.

[0086] For search region features Based on pseudo-target bounding box generate and Due to the generated pseudo-target bounding box Not precise enough, so we're using an updated template foreground-background proxy here. As a query, to provide a prototype proxy Clearer foreground and background priors. The formula for generating the foreground-background proxy for the search region is as follows:

[0087]

[0088]

[0089] Finally, the template foreground-background proxy was obtained in the l-th block search region. and search area foreground-background proxy

[0090] Distributed attention module. This module obtains the foreground-background proxy template. and search area foreground-background proxy Next, the distribution-aware attention module models the foreground-background distribution for each image label. Specifically, given the image label features... in The template marking feature of the output of the l-th converter block, Mark the features of the search region output by the l-th converter block;

[0091] Initial queries, keys, and values ​​are generated based on image tagging features. The formula is as follows:

[0092]

[0093]

[0094]

[0095] Where LN represents layer normalization. It is a linear projection. It can be seen that the initial query, key, and value are generated simply based on appearance features, thus easily producing incorrect high attention scores for regions with similar appearance distractors. Therefore, this invention uses foreground-background distribution as the query and key instead of appearance features. Specifically, firstly... and This is achieved by linearly mapping to a channel dimension of C. The foreground-background distribution of each image feature label can be modeled by calculating the similarity between each image feature label and these foreground-background proxies. Finally, the foreground-background distribution of the initial query label is used as the basis for new queries. The formula is as follows:

[0096]

[0097]

[0098] in These are linear mapping parameters, [·] T This is a matrix transpose operation. Similarly, we can obtain a new key. The formula is as follows:

[0099]

[0100]

[0101] By updating the query and key, the proposed distributed awareness attention module can achieve foreground-background awareness feature interaction and learning. The output labeled features of the l-th transformer block... It can be represented as follows:

[0102]

[0103]

[0104] MLP stands for Multilayer Perceptron. Output labeled features. This will be input into the next transformer block. Finally, the labeled features are processed L times through our proposed foreground-background distribution modeling deformer block, and we can obtain the final labeled features.

[0105] Target bounding box regression header. The target bounding box regression header is as follows: Figure 4 As shown, the final search region is first labeled with features. The features are rearranged into a two-dimensional spatial feature map and then fed into a fully convolutional network, which consists of N stacked convolutional layers, batch regularization layers, and ReLU function layers for each output. The output of the fully convolutional network contains the object classification score map. Local offset And the normalized bounding box size (i.e., width and height). Local offsets are used to compensate for discretization errors caused by reduced resolution. Finally, the location with the highest score in the classification score image is considered the target location, i.e., (x... d ,y d ) = argmax (x,y) P xy , where argmax is the function that takes the maximum value. The final target bounding box is:

[0106] (x,y)=(x d +O(0,x d ,y d ),y d +O(1,x d ,y d ))

[0107] (w,h)=(S(0,x d ,y d ),S(1,x d ,y d ))

[0108] The above method can be applied to visual target tracking systems in real-world scenarios such as video surveillance, anti-drone tracking, and autonomous driving to locate any specified target in a video.

[0109] The hardware device of the present application embodiment is illustrated below.

[0110] This application provides an image recognition device. In practice, it can be embedded in a mobile device as software to provide real-time target location results; or it can be installed on a backend server to provide large-scale video location results. Furthermore, the method can be deployed to general-purpose hardware systems using ONNX tools.

[0111] The image recognition device includes a processor and a memory.

[0112] A memory for storing computer programs, the computer programs including program instructions;

[0113] The processor is used to invoke the computer program to implement the visual target tracking method described above.

[0114] Optionally, the image recognition device also includes a communication bus and a communication interface.

[0115] The processor includes one or more processing cores, and it executes various functional applications and data processing by running computer programs.

[0116] The memory can be used to store computer programs. Optionally, the memory can store the operating system and application program units required for at least one function. The operating system can be a real-time operating system (RTX), Linux, UNIX, Windows, or OS X.

[0117] There can be multiple communication interfaces, which are used to communicate with other storage devices or network devices. Network devices can be switches or routers, etc.

[0118] The memory and communication interface are connected to the processor via a communication bus.

[0119] This application also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the above-described visual target tracking method.

[0120] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0121] In the embodiments of this application, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0122] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0123] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the concept and principles of this application should be included within the protection scope of this application.

Claims

1. A visual target tracking method based on foreground and background distribution recognition, characterized in that, include: The system receives the image cropped from the first frame and the search region image cropped from the target position in the previous frame, wherein the image cropped from the first frame is used as a template and the search region image cropped from the target position in the previous frame is used as the search region feature map. The learned labels are embedded in the small block sequence features corresponding to the template and search region feature maps, respectively, to generate template labels and search region labels. The template marker and search region marker are input into multiple foreground distribution modeling converter blocks in the iterative process, and the last foreground distribution modeling converter block outputs the final search region marker; The final search region markers are then converted into two dimensions. These two-dimensional markers are input into the target bounding box regression head for output. The output includes a classification score map, a local target offset, and the target bounding box size. The location with the highest score in the output classification score map is taken as the target, and the corresponding local target offset and target bounding box size determine the target location. The foreground distribution modeling converter block includes: Receive template markers and search region markers, aggregate the information inside or outside the target box corresponding to the template marker and the pseudo target box corresponding to the search region marker, and generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively; Using the updated template foreground proxy, template background proxy, search region foreground proxy, and search region background proxy as the query and key, and aiming to maximize the similarity between the search region tag and template tag and the query and key, respectively, output the obtained template tag and search region tag.

2. The visual target tracking method according to claim 1, characterized in that, The template and search region feature maps are segmented into blocks to obtain template block sequence features and search region block sequence features. The template block sequence features and search region block sequence features are mapped to one dimension. The learned labels are embedded into the one-dimensional template block sequence features and search region block sequence features respectively to generate template labels and search region labels.

3. The visual target tracking method according to claim 1, characterized in that, The input quantities of the foreground distribution modeling converter block are all normalized. The input quantities include the template marker and search region marker input for the first time, as well as the template marker and search region marker processed by the foreground distribution modeling converter block.

4. The visual target tracking method according to claim 1, characterized in that, The acquisition of the pseudo-target bounding box corresponding to the search region marker includes: Receive template markers and search region markers, and perform two-dimensional transformations on the template markers and search region markers respectively to obtain a first template feature map and a first search region feature map. Use the target box corresponding to the template feature as a reference to obtain the target feature of the first template feature map. For each point of the target feature in the first template feature map, obtain the corresponding matching point in the first search region feature map using a similarity matching method. Obtain a set of matching points from each point in the first template feature map to the first search region feature map. Based on the obtained matching points, obtain the pseudo target box corresponding to the search region marker.

5. The visual target tracking method according to claim 4, characterized in that, The similarity matching method includes obtaining a set of matching points from each point in the first template feature map to the first search region feature map based on probability D. l (k,i) is: probability D l (k,i), Where k and i are respectively and All two-dimensional positions; First template feature map; First search region feature map; s is the stride of the backbone network; Each point in A set of matching points Where K = h t w t h t w represents the height of the first template feature map. t x is the width of the first template feature map. i Let y be the x-coordinate of the matching point from the i-th region of the first template feature map to the first search region feature map. i The ordinate is the vertical coordinate of the matching point from the i-th region of the first template feature map to the feature map of the first search region.

6. The visual target tracking method according to claim 1, characterized in that, Based on the obtained matching points, obtain the pseudo-target boxes corresponding to the search region markers, including: obtaining the pseudo-target boxes of the search region based on the mean and standard deviation of the matching points.

7. The visual target tracking method according to claim 1, characterized in that, The information inside or outside the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker is aggregated separately to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively. This includes: using a decoder with a mask cross-attention mechanism to aggregate the foreground or background information of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker respectively to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively.

8. The visual target tracking method according to claim 1, characterized in that, The target bounding box regression head includes a fully convolutional network, which comprises N stacked convolutional layers per output, batch regularization layers, and ReLU function layers.

9. A visual target tracking system for foreground and background distribution recognition, characterized in that, include: The input module is used to receive the image cropped from the first frame and the search region image cropped from the target position in the previous frame, wherein the image cropped from the first frame is used as the template and the search region image cropped from the target position in the previous frame is used as the search region feature map. The labeling module is used to embed the learned labels into the small block sequence features corresponding to the template and search region feature maps, respectively, to generate template labels and search region labels. The processing module is used to input template tags and search region tags into multiple foreground distribution modeling converter blocks in the iterative processing, and the last foreground distribution modeling converter block outputs the final search region tags; This is used to convert the final search region label into two dimensions. The two-dimensional final search region label is input into the target bounding box regression head for output. The output includes a classification score map, target local offset, and target bounding box size. The position with the highest score in the output classification score map is taken as the target, and the corresponding target local offset and target bounding box size determine the target location; wherein, The foreground distribution modeling converter block includes: The foreground-background proxy learning module receives template markers and search region markers, and aggregates information from the inside or outside of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker, respectively, to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy. The distribution-aware attention module is used to take the updated template foreground proxy, template background proxy, search region foreground proxy, and search region background proxy as queries and keys, and aims to maximize the similarity between the search region tag and the template tag and the query and key, respectively, and output the obtained template tag and search region tag.

10. The visual target tracking system according to claim 9, characterized in that, The template and search region feature maps are segmented into blocks to obtain template block sequence features and search region block sequence features. The template block sequence features and search region block sequence features are mapped to one dimension. The learned labels are embedded into the one-dimensional template block sequence features and search region block sequence features respectively to generate template labels and search region labels.

11. The visual target tracking system according to claim 9, characterized in that, The input quantities of the foreground distribution modeling converter block are all normalized. The input quantities include the template marker and search region marker input for the first time, as well as the template marker and search region marker processed by the foreground distribution modeling converter block.

12. The visual target tracking system according to claim 9, characterized in that, The acquisition of the pseudo-target bounding box corresponding to the search region marker includes: Receive template markers and search region markers, and perform two-dimensional transformations on the template markers and search region markers respectively to obtain a first template feature map and a first search region feature map. Use the target box corresponding to the template feature as a reference to obtain the target feature of the first template feature map. For each point of the target feature in the first template feature map, obtain the corresponding matching point in the first search region feature map using a similarity matching method. Obtain a set of matching points from each point in the first template feature map to the first search region feature map. Based on the obtained matching points, obtain the pseudo target box corresponding to the search region marker.

13. The visual target tracking system according to claim 12, characterized in that, The similarity matching method includes obtaining a set of matching points from each point in the first template feature map to the first search region feature map based on probability D. l (k,i) represents the probability D. l (k,i), Where k and i are respectively and All two-dimensional positions; First template feature map; First search region feature map; s is the stride of the backbone network; Each point in A set of matching points Where K = h t w t h t w represents the height of the first template feature map. t x is the width of the first template feature map. i Let y be the x-coordinate of the matching point from the i-th region of the first template feature map to the first search region feature map. i The ordinate is the vertical coordinate of the matching point from the i-th region of the first template feature map to the feature map of the first search region.

14. The visual target tracking system according to claim 9, characterized in that, Based on the obtained matching points, obtain the pseudo-target boxes corresponding to the search region markers, including: obtaining the pseudo-target boxes of the search region based on the mean and standard deviation of the matching points.

15. The visual target tracking system according to claim 9, characterized in that, The information inside or outside the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker is aggregated separately to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively. This includes: using a decoder with a mask cross-attention mechanism to aggregate the foreground or background information of the target box corresponding to the template marker and the pseudo-target box corresponding to the search region marker respectively to generate template foreground proxy, template background proxy, search region foreground proxy and search region background proxy respectively.

16. The visual target tracking system according to claim 9, characterized in that, The target bounding box regression head includes a fully convolutional network, which comprises N stacked convolutional layers per output, batch regularization layers, and ReLU function layers.

17. An image recognition device, comprising: Processor and memory; The memory is used to store computer programs, the computer programs including program instructions; The processor is used to invoke the computer program to implement the visual target tracking method as described in any one of claims 1 to 9.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a processor, implement the visual target tracking method as described in any one of claims 1 to 9.

Images