A target tracking method for robotic vision

Abstract

This invention discloses a target tracking method for robot vision; belonging to the technical field of target tracking methods; it includes the following steps: (1) manually obtaining the upper left and lower right coordinates of the target to be tracked on the two-dimensional image in the first frame of the image, extracting the target image and its surrounding sample image blocks as templates, and obtaining the template sample feature tensor through a feature extraction deep network; (2) inputting the search region sample into the feature extraction deep network to obtain the search region sample feature tensor; (3) simultaneously inputting the template feature tensor and the search region feature tensor into a feature enhancement and feature fusion network based on an involution-attention model to obtain a fused feature tensor containing template features and search region features, and then passing it through a classification network and a regression network to obtain the tracking result; this invention aims to provide a target tracking method for robot vision with high tracking success rate, small tracking error and real-time tracking capability; for real-time target tracking.

Description

Technical Field

[0001] This invention relates to a target tracking method, and more specifically, to a target tracking method for robot vision. Background Technology

[0002] Video target tracking technology, as an important aspect of machine vision research, has attracted widespread attention from researchers. It aims to track targets in a video using the target state information obtained in the first frame, thus acquiring the target state for each frame. During tracking, situations arise such as changes in target shape, complex backgrounds with varying lighting, and target occlusion. Under these circumstances, the target's feature structure changes accordingly, making it difficult for tracking algorithms to lock onto the target.

[0003] Since the introduction of deep learning technology into visual tracking, convolution has been widely used in frameworks for feature extraction and template and search region fusion. Currently popular deep learning trackers are primarily constructed from convolutional kernels; however, due to the high computational cost, convolutional kernels cannot be designed to be too large. Therefore, convolutional kernels cannot interact with long-distance information in a single pass. This limitation restricts the model's ability to handle complex scenes when similar targets or targets with significant morphological changes are present.

[0004] Long-range dependency problems can be effectively solved by introducing self-attention mechanisms, which have been successfully applied in machine translation, natural language processing, and speech processing. Furthermore, they have achieved excellent experimental results in image processing, such as object tracking and detection. While self-attention mechanisms capture global information well, they do not pay particular attention to local information. For object tracking, local information surrounding the object should have a higher weight. Therefore, it is necessary to develop a model that can handle global information like self-attention mechanisms while also handling local information like convolution to address this problem. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing a target tracking method for robot vision that has a high tracking success rate, small tracking error, and can achieve real-time tracking.

[0006] The technical solution of the present invention is implemented as follows: a target tracking method for robot vision, comprising the following steps:

[0007] (1) Manually obtain the upper left and lower right coordinates of the target to be tracked on the two-dimensional image in the first frame of the image, extract the target image and the sample image blocks around it as templates, and obtain the template sample feature tensor through the feature extraction deep network.

[0008] (2) Input the search region samples into the same feature extraction deep network to obtain the search region sample feature tensor;

[0009] (3) Input the feature tensor of the template and the feature tensor of the search region into the feature enhancement and feature fusion network based on the involution-attention model to obtain a fused feature tensor containing the template features and the search region features. Then, pass the fused feature tensor through the classification network and the regression network to obtain the tracking result.

[0010] In the above-mentioned target tracking method for robot vision, in step (1), the feature extraction deep network specifically adopts the ResNet50 network as the benchmark; the ResNet50 includes one trunk layer and four branch layers, with 3, 4, 6 and 3 bottlenecks respectively;

[0011] In the feature extraction deep network, the fourth layer of ResNet50 was discarded, and the downsampling stride parameter of the Conv2d operator in the third layer was changed from 2 to 1; in the dry layers of ResNet50, 7×7 pairs of inner convolutions were used instead of the previous 7×7 convolution kernels; in other layers, all 3×3 convolution kernels were replaced by 7×7 inner convolution kernels; finally, a 1×1 convolution was added after the third layer.

[0012] The target tracking method for robot vision described above, in step (3), the involution-attention model consists of an involution module, two Add & Norm modules, and one FFN & Relu module;

[0013] The internal injection module takes tensor A and tensor B as input; using and Convolution tensors and inner convolution kernels are constructed separately, where d is the number of channels and w×w is the scale of the image patch;

[0014] To construct the inner convolutional kernel, tensor B is expanded into Given a learnable parameter matrix and The query Q and key K can be obtained as follows:

[0015] Q = B'W Q

[0016] K = B'W K (1)

[0017] in, Then, attention matrix It can be obtained from equation (2);

[0018]

[0019] Then, the attention matrix M is transformed into an inner convolution kernel tensor. Where g is the number of inner convolution kernel groups, w×w is the scale of the convolutional image, and k×k is the size of the inner convolution kernel.

[0020] In the above-mentioned target tracking method for robot vision, transforming the attention matrix M into an inner convolution kernel tensor I depends on different types of B. It is necessary to process two types of input B: search region samples and template set samples. The template set samples consist of four templates and can be updated online.

[0021] When the input B is a search region tensor, M i，j Let Q represent the similarity between the i-th row of Q and the j-th row of K; since each kernel is globally sampled, all inner convolutional kernels are able to capture the long-range dependencies of the search region; this strategy is called involution attention strategy 1;

[0022] When the input B is a template set tensor, four templates are used to connect the template set tensor; the i-th row of M describes the similarity between the i-th element in Q and all elements in the four templates in K; since each kernel is globally sampled, all inner convolutional kernels are able to capture the long dependencies of the template set tensor; this strategy is called involution attention strategy 2.

[0023] In the above-mentioned target tracking method for robot vision, step (3) consists of five modules: an involution-attention template module, an involution-attention search region module, an involution-attention template search module, an involution-attention search template module, and an involution-attention hybrid module; wherein, the involution-attention in the five modules is based on the involution-attention model.

[0024] The specific steps to obtain the fused feature tensor containing template features and search region features are as follows: First, the template set features F T0 and search region features F S0 The enhanced feature F is obtained through the involution-attention template module and the involution-attention search region module, respectively. T1 and F S1 Then, the enhanced template feature F T1 and search region features F S1 Simultaneously, the involution-attention template search module and the involution-attention template search module are cross-inputted to obtain the fused feature F. T2 and F S2 Among them, the involution-attention template module, the involution-attention search region module, the involution-attention template search module, and the involution-attention search template module jointly construct the feature enhancement fusion layer, which is repeated 4 times;

[0025] Following the feature enhancement and fusion layer, the involution-attention hybrid module fuses the features F. T2 and F S2 The input is the feature F, which is then fed into the regression and classification networks.

[0026] In the above-mentioned target tracking method for robot vision, the involution-attention search region module and the involution-attention template search module use involution attention strategy 1 to obtain the internal convolution kernel, and the involution-attention template module, the involution-attention search template module and the involution-attention hybrid module use involution attention strategy 2 to obtain the internal convolution kernel.

[0027] In the above-described target tracking method for robot vision, step (3) involves a classification network comprising three linear layers and two activations, denoted as follows:

[0028] f c (F)=φ2((φ1(F*W1)*W2))*W3, (3)

[0029] in, The output of the feature mixing network is W1, W2, and W3, which are learnable parameter matrices; the output of the classification network is a binary tensor. The classification loss is calculated using the standard binary cross-entropy loss.

[0030]

[0031] Among them, y i p is the ground-truth label of the i-th sample, where 1 indicates a positive sample. i The probability that it is a positive sample;

[0032] By using the softmax function, f c (F) is mapped to a classification rating matrix S. Ideally, in the rating matrix S, the classification rating of the target region is 1, and the classification rating of the background region is 0.

[0033] In the target tracking method for robot vision described above, in some cases, such as similar targets, occlusion, or targets outside the range, S may be contaminated; therefore, in the tracking method, a classification score higher than a predetermined value is considered a high score.

[0034] Assume the number of high scores inside and outside the regression box is N. i and N o The area of ​​the regression box is N. r Define the updated score as s = (N i -N o ) / N rWhen s > τ and the update interval has elapsed, the template is updated; where τ is the template update threshold.

[0035] In the above-described target tracking method for robot vision, step (3) involves establishing a regression network by estimating the probability distribution of the target bounding box. The regression network is a fully convolutional network (FCN) with four Conv-BN-ReLU layers. The output of the regression network has four channels, representing the probability distributions of the left, right, top, and bottom of the target bounding box, respectively. Therefore, the coordinates of the bounding box are...

[0036] x tl =∑(xP left (x))

[0037] y tl =∑(yP top (y))

[0038] x br =∑(xP right (x))

[0039] y br =∑(yP bottom (y)), (5)

[0040] Among them, P left P top P riqht P bottm Let be the probability distributions of the left, right, top, and bottom of the bounding box, respectively; combining IoU loss and L1 loss, the loss function of the regression network is:

[0041]

[0042] Where, λ iou and λ l These are hyperparameters used to adjust the weights of the two terms, b and These are the coordinates of the true bounding box and the coordinates of the predicted bounding box, respectively.

[0043] This invention employs the aforementioned method, firstly by using an improved feature extraction deep network to obtain template sample feature tensors from the target image and surrounding sample image patches, and secondly by obtaining search region sample feature tensors from search region samples. Then, through a unique feature enhancement and feature fusion network based on an involution-attention model, combined with classification and regression networks, the tracking result is obtained. This effectively enhances the target tracking success rate, reduces tracking errors, achieves real-time tracking, and improves robot control. Attached Figure Description

[0044] The present invention will be further described in detail below with reference to the embodiments shown in the accompanying drawings, but this does not constitute any limitation on the present invention.

[0045] Figure 1 This is a schematic diagram of the framework of the tracking method of the present invention;

[0046] Figure 2 This is a schematic diagram of the involution attention model of the present invention;

[0047] Figure 3 This is a schematic diagram of the internal injection module of the present invention;

[0048] Figure 4 This is a schematic diagram demonstrating the two attention matrix variable-dimensional strategies of the present invention;

[0049] Figure 5 This is a schematic diagram illustrating the relationship between the score heatmap and the regression bounding box of this invention;

[0050] Figure 6 This is a block diagram of the target tracking device of the present invention. Detailed Implementation

[0051] See Figure 1 As shown, a target tracking method for robot vision according to the present invention includes the following steps:

[0052] (1) Manually obtain the upper left and lower right coordinates of the target to be tracked on the two-dimensional image in the first frame of the image, extract the target image and the sample image blocks around it as templates, and obtain the template sample feature tensor through the feature extraction deep network.

[0053] (2) Input the search region samples into the same feature extraction deep network to obtain the search region sample feature tensor;

[0054] (3) Input the feature tensor of the template and the feature tensor of the search region into the feature enhancement and feature fusion network based on the involution-attention model to obtain a fused feature tensor containing the template features and the search region features. Then, pass the fused feature tensor through the classification network and the regression network to obtain the tracking result.

[0055] In step (1), in order to extract features more effectively, the existing feature extraction deep network was modified using internal convolution. The feature extraction deep network described in this invention is as follows: it uses the ResNet50 network as the benchmark; the ResNet50 includes one trunk layer and four branch layers, with 3, 4, 6 and 3 bottlenecks respectively.

[0056] In this feature extraction deep network, the fourth layer of ResNet50 is discarded. To obtain a larger feature resolution, the downsampling stride parameter of the Conv2d operator in the third layer is changed from 2 to 1. In the dry layers of ResNet50, 7×7 pairs of inner convolutions are used instead of the previous 7×7 convolution kernels. In other layers, all 3×3 convolution kernels are replaced by 7×7 inner convolution kernels. Therefore, this feature extraction deep network can obtain a larger receptive field in one step. Finally, a 1×1 convolution is added after the third layer to reduce the channel dimension of the feature extraction deep network output.

[0057] Table 1 lists the details of the modified kernel. The second column indicates the size of the replaced convolutional kernel, and the multiplier in parentheses indicates the number of times the convolutional kernel in that layer was replaced. The third column indicates the size of the inner convolutional kernel. The fourth column is the number of channels of the convolution / kernel. The last column indicates the number of groups of inner convolutional kernels. The input to the backbone is a template image. Search area image Template features output by the backbone network after passing through the backbone network Search area characteristics

[0058] Table 1 Modified ResNet50 kernel

[0059]

[0060]

[0061] See Figure 2 As shown, in this embodiment, the involution-attention model described in step (3) consists of an involution module, two Add&Norm modules, and one FFN&ReLU module.

[0062] See Figure 3 As shown, the internal injection module takes tensor A and tensor B as input; using and Convolution tensors and inner convolution kernels are constructed separately, where d is the number of channels and w×w is the scale of the image patch.

[0063] To construct the inner convolutional kernel, tensor B is expanded into Given a learnable parameter matrix and The query Q and key K can be obtained as follows:

[0064] Q = B'W Q

[0065] K = B'W K (1)

[0066] in, Then, attention matrix It can be obtained from equation (2).

[0067]

[0068] Then, the attention matrix M is transformed into an inner convolution kernel tensor. Where g is the number of inner convolution kernel groups, w×w is the scale of the convolutional image, and k×k is the size of the inner convolution kernel.

[0069] It's important to note that transforming the attention matrix M into an inner convolutional kernel tensor I depends on different types of input B. Two types of input B need to be processed: search region samples and template set samples. The template set samples consist of four templates and can be updated online. See also... Figure 4 As shown, it illustrates two types of input B-dimensionality variation strategies.

[0070] When the input B is a search region tensor, M i，j This represents the similarity between the i-th row of Q and the j-th row of K; for example... Figure 4 As shown in (a), for simplicity, we take a 64×64 matrix M and an inner convolutional kernel of shape 2×2 as an example. The dashed rectangle represents the first row of matrix M, which can be reconstructed into an 8×8 matrix. By extracting one element from every 4 rows and 4 columns, we can construct 16 sets of inner convolutional kernels of shape 2×2. Since each kernel is globally sampled, all inner convolutional kernels are able to capture the long-range dependencies of the search region. This strategy is called Involution Attention Strategy 1.

[0071] When input B is a template set tensor, this embodiment uses four templates to connect the template set tensor. For example... Figure 4 As shown in (b), blocks 1-1 and 1-2 represent the pairwise similarity between the first template itself and between the first and second templates, respectively. For example, the i-th row of M describes the similarity between the i-th element in Q and all elements in the four templates in K. The dashed rectangles in M ​​can be reshaped into an 8×8 matrix, where the blocks in the red, blue, yellow, and green boxes are associated with the first, second, third, and fourth templates, respectively. Specifically, in the figure, number 2 corresponds to the yellow box, number 3 to the red box, number 4 to the blue box, and number 6 to the green box. In each block, elements are extracted from every two rows and two columns. For all blocks, 16 sets of inner convolutional kernels are obtained. Since each kernel is globally sampled, all inner convolutional kernels are able to capture long dependencies of the template set tensor. This strategy is called Involution Attention Strategy 2.

[0072] More preferably, in step (3), the feature enhancement and feature fusion network based on the involution-attention model consists of five modules: an involution-attention template module, an involution-attention search region module, an involution-attention template search module, an involution-attention search template module, and an involution-attention hybrid module; wherein, the involution-attention in the five modules all represent the involution-attention model.

[0073] The specific steps to obtain the fused feature tensor containing template features and search region features are as follows: First, the template set features F T0 and search region features F S0 The enhanced feature F is obtained through the involution-attention template module and the involution-attention search region module, respectively. T1 and F S1 Then, the enhanced template feature F T1 and search region features F S1 Simultaneously, the involution-attention template search module and the involution-attention template search module are cross-inputted to obtain the fused feature F. T2 and F S2 The involution-attention template module, the involution-attention search region module, the involution-attention template search module, and the involution-attention search template module together construct the feature enhancement fusion layer, which is repeated 4 times.

[0074] Following the feature enhancement and fusion layer, the involution-attention hybrid module fuses the features F. T2 and F S2 The input is the feature F, which is then fed into the regression and classification networks.

[0075] In this embodiment, the involution-attention search region module and the involution-attention template search module use involution attention strategy 1 to obtain the internal convolution kernel, and the involution-attention template module, the involution-attention search template module and the involution-attention hybrid module use involution attention strategy 2 to obtain the internal convolution kernel.

[0076] More preferably, to improve the robustness of tracking, the template set needs to be updated during the tracking process. However, when tracking drift, target occlusion, or target deviation occurs, the current tracking results are unreliable. To ensure the reliability of the tracking results, this invention proposes a classification network containing three linear layers and two activations, which can be represented as follows:

[0077] f c (F)=φ2((φ1(F*W1)*W2))*W3, (3)

[0078] in, The output of the feature mixing network is W1, W2, and W3, which are learnable parameter matrices; the output of the classification network is a binary tensor. The classification loss is calculated using the standard binary cross-entropy loss.

[0079]

[0080] Among them, y i p is the ground-truth label of the i-th sample, where 1 indicates a positive sample. i The probability that it is a positive sample;

[0081] By using the softmax function, f c (F) is mapped to a classification rating matrix S. Ideally, in the rating matrix S, the classification rating of the target region is 1, and the classification rating of the background region is 0.

[0082] However, in certain situations, such as with similar targets, obstructions, or targets outside the range, S may be contaminated.

[0083] like Figure 5 As shown, Figure 5 The red box represents the regression boxes provided by the regression network. In score heatmap A, the regression boxes contain most of the high scores. In score heatmap B, the regression boxes contain the portion of content that received high scores due to similar targets. Therefore, the results of heatmap A are more reliable than those of heatmap B, demonstrating that updating the template set using the tracking results of heatmap A is reasonable. In the tracking method, classification scores above a predetermined value are considered high scores; in this embodiment, the predetermined value is 0.88. Assume that the number of high scores inside and outside the regression boxes is N. i and N o The area of ​​the regression box is N. r Define the updated score as s = (N) i -N o ) / N r When s > τ and the update interval has elapsed, the template is updated. Here, τ is the template update threshold.

[0084] Traditional anchorless regression networks directly learn the target state and follow a Dirac delta distribution, which is limited to situations where the target boundary is not clear, such as occlusion, motion blur, shadows, and complex backgrounds. This invention establishes a regression network by estimating the probability distribution of the target bounding box.

[0085] In this embodiment, the regression network is a fully convolutional network (FCN) with four Conv-BN-ReLU layers; the output of the regression network has four channels, representing the probability distribution of the left, right, top, and bottom of the bounding box, respectively; therefore, the coordinates of the bounding box are...

[0086] xtl =∑(xP left (x))

[0087] y tl =∑(yP top (y))

[0088] x br =∑(xP right (x))

[0089] y br =∑(yP bottom (y)), (5)

[0090] Among them, P left P top P right P bottm These represent the probability distributions for the left, right, top, and bottom of the bounding box, respectively. Compared to other regression networks, the regression network of this invention performs better in handling uncertainty. Combining IoU loss and l1 loss, the loss function of the regression network is:

[0091]

[0092] Where, λ iou and λ l These are hyperparameters used to adjust the weights of the two terms, b and These are the coordinates of the true bounding box and the coordinates of the predicted bounding box, respectively.

[0093] Experimental results

[0094] The tracking method and apparatus of the present invention were compared with the more advanced methods on the current mainstream datasets, namely TrackingNet and GOT-10k datasets.

[0095] Table 2 shows the test results of the tracking method of the present invention and other algorithms on TrackingNet. It can be seen that the method proposed in this invention has achieved the best results in terms of Prec. (%), N.Prec. (%) and Success (AUC).

[0096] Table 2 Comparison results of Prec., N.Prec., and AUC on the TrackingNet test set

[0097]

[0098] Table 3 shows the test results of the tracking method of the present invention and other algorithms on the GOT-10k test set. It can be seen that the method proposed in this invention has achieved the best results in terms of mAO (%), SR0.5 (%) and SR0.75 (%).

[0099] method mAO (%) SR 0.5 (%) SR 0.75 (%) TrSiam 67.3 78.7 58.6 TrDiMP 68.8 80.5 59.7 TransT 72.3 82.4 68.2 TREG 66.8 77.8 57.2 STACK-ST50 68.0 77.7 62.3 SiamFC++ 59.5 69.5 47.9 Method of the present invention 73.2 83.3 68.8

[0100] See Figure 6 As shown, a target tracking device based on the above method according to the present invention includes an image acquisition module, a feature extraction module, a feature enhancement module, a feature fusion module, a classification module, a regression module, a result display module, a template update module, and a robot data interface module.

[0101] The system comprises several modules: an image acquisition module that uses a camera to capture video data; a feature extraction module that extracts depth features from the target template and the search area; a feature enhancement module that enhances the extracted depth features; a feature fusion module that fuses the enhanced features from the target template and the search area; a classification module that classifies and judges the feature regions; a regression module that determines the target state; a result display module that displays the tracking results in the original video image; a template update module that determines whether the tracking results should be used to update the target template set based on the results from the classification and regression modules; and a robot data interface module that transmits the tracking results to the robot controller to help the robot make decisions and plan actions.

[0102] The above-described embodiments are preferred embodiments of the present invention and are only used to facilitate the illustration of the present invention. They are not intended to limit the present invention in any way. Any person skilled in the art who makes local modifications or alterations to the technical content disclosed in the present invention without departing from the scope of the technical features of the present invention shall still fall within the scope of the technical features of the present invention.

Claims

1. A target tracking method for robot vision, characterized by, Includes the following steps: (1) Manually obtain the upper left and lower right coordinates of the target to be tracked on the two-dimensional image in the first frame of the image, extract the target image and the sample image blocks around it as templates, and obtain the template sample feature tensor through the feature extraction deep network. (2) Input the search region samples into the same feature extraction deep network to obtain the search region sample feature tensor; (3) Input the feature tensor of the template and the feature tensor of the search region into the feature enhancement and feature fusion network based on the involution-attention model to obtain a fused feature tensor containing the template features and the search region features. Then, pass the fused feature tensor through the classification network and the regression network to obtain the tracking result. The involution-attention model consists of an involution module, two Add & Norm modules, and one FFN & ReLU module. The injection module uses tensors and tensor For input; use and Construct convolutional tensors and inner convolutional kernels separately, where For the number of channels, The scale of the image patch; To construct the inner convolutional kernel, tensors Expanded into Therefore, given a learnable parameter matrix... and You can get the query s and keys for (1) in, , ; Then, attention matrix It can be obtained from equation (2); （2） Then, the attention matrix Transform the dimension into an internal convolution kernel tensor ,in The number of internal convolutional kernel groups. The scale of the convolutional image. This represents the size of the internal convolution kernel.

2. The target tracking method for robot vision according to claim 1, characterized in that, In step (1), the feature extraction deep network specifically uses the ResNet50 network as a benchmark; the ResNet50 includes one trunk layer and four branch layers, with 3, 4, 6 and 3 bottlenecks respectively; In the feature extraction deep network, the fourth layer of ResNet50 was discarded, and the downsampling stride parameter of the Conv2d operator in the third layer was changed from 2 to 1; in the dry layers of ResNet50, 7×7 pairs of inner convolutions were used instead of the previous 7×7 convolution kernels; in other layers, all 3×3 convolution kernels were replaced by 7×7 inner convolution kernels; finally, a 1×1 convolution was added after the third layer.

3. The target tracking method for robot vision according to claim 1, characterized in that, Attention matrix Transform the dimension into an internal convolution kernel tensor Depends on different types It needs to handle two types of input. Search region samples and template set samples. The template set samples consist of four templates and can be updated online. When input When it is a search region tensor, express The lines and The Row similarity; since each kernel is globally sampled, all inner convolutional kernels are able to capture long-range dependencies in the search region, which is called the involution attention strategy 1; When input When it is a template set tensor, use four templates to connect the template set tensor; The The line describes The first in Each element and The similarity between all elements in the four templates; since each kernel is globally sampled, all inner convolutional kernels are able to capture long dependencies of the template set tensor, which is called the involution attention strategy 2.

4. A target tracking method for robot vision according to claim 3, characterized in that, In step (3), the feature enhancement and feature fusion network based on the involution-attention model consists of five modules: an involution-attention template module, an involution-attention search region module, an involution-attention template search module, an involution-attention search template module, and an involution-attention hybrid module; wherein, the involution-attention in the five modules is based on the involution-attention model. The specific steps to obtain the fused feature tensor containing template features and search region features are as follows: First, the template set features... and search region features Enhanced features are obtained through the involution-attention template module and the involution-attention search region module, respectively. and Then, the enhanced template features and search region features Simultaneously cross-input to the involution-attention template search module and the involution-attention template search module to obtain fused features. and Among them, the involution-attention template module, the involution-attention search region module, the involution-attention template search module, and the involution-attention search template module jointly construct the feature enhancement fusion layer, which is repeated 4 times. Following the feature enhancement and fusion layer, the involution-attention hybrid module fuses the features. and Input, output features They are then fed into regression and classification networks.

5. A target tracking method for robot vision according to claim 4, characterized in that, The involution-attention search region module and the involution-attention template search module use involution attention strategy 1 to obtain the internal convolution kernel, while the involution-attention template module, the involution-attention search template module, and the involution-attention hybrid module use involution attention strategy 2 to obtain the internal convolution kernel.

6. A target tracking method for robot vision according to claim 1, characterized in that, In step (3), the classification network is a classification network containing 3 linear layers and 2 activations, represented as follows: (3) in, The output of the feature mixing network, , , The parameter matrix is ​​learnable; the output of the classification network is a binary tensor. The classification loss is calculated using the standard binary cross-entropy loss. (4) in, For the first The ground-truth label of each sample, with a value of 1 indicating a positive sample. The probability that it is a positive sample; By using the softmax function, Mapped to a classification rating matrix In an ideal state, in the rating matrix In the data, the classification score for the target region is 1, while the classification score for the background region is 0.

7. A target tracking method for robot vision according to claim 6, characterized in that, Similar targets, obscured targets, or targets outside the range. It may be contaminated; therefore, in the tracking method, a classification score higher than a predetermined value is considered a high score. Assume the number of high scores inside and outside the regression frame are respectively and The area of ​​the regression box is Define the updated score as ;when Furthermore, the template is updated when the update interval is reached; where τ is the template update threshold.

8. A target tracking method for robot vision according to claim 1, characterized in that, In step (3), the regression network is established by estimating the probability distribution of the target box; the regression network is a fully convolutional network (FCN) with four Conv-BN-ReLU layers; the output of the regression network has four channels, representing the probability distribution of the left, right, top, and bottom of the target box respectively; therefore, the coordinates of the bounding box are... (5) in, , , , These represent the probability distributions for the left, right, top, and bottom of the bounding box, respectively; combined with IoU loss and The loss function of the regression network is: (6) in, and These are hyperparameters used to adjust the weights of the two terms. and These are the coordinates of the true bounding box and the coordinates of the predicted bounding box, respectively.

Images