A feature matching method of a tracking model based on a twin network

Abstract

The application discloses a feature matching method of a tracking model based on a twin network, wherein a template frame and a detection frame are first sent into a network for convolution feature extraction to generate corresponding template feature maps and detection feature maps, and a matching feature map is obtained; then, a region proposal network is optimized, the matching is enhanced by increasing the number of convolution layers and the number of channels and introducing grouped convolution, feature matching is performed on the template feature maps and the detection feature maps, necessary classification and dimension mapping of a regression branch are performed on the obtained matching feature map, and finally, response maps of two branches are output; the method has certain adaptability and expansibility, can be applied to a tracking algorithm network model based on a twin network structure, and can enhance the tracking precision without sacrificing the speed performance.

Description

Technical Field

[0001] This invention belongs to the field of target tracking technology, and specifically relates to a feature matching method for a tracking model based on Siamese networks. Background Technology

[0002] In the field of target tracking, speed and accuracy are the two core metrics for evaluating algorithm performance. The application of tracking methods requires a balanced improvement in both accuracy and speed; however, most existing tracking methods currently focus on improving tracking accuracy while neglecting or even sacrificing tracking speed.

[0003] The SiamRPN (Siamese region proposal network) algorithm, proposed by Li et al. in 2018, is a real-time target tracking algorithm with excellent balanced performance, and has been favored by both academia and industry. The network structure of SiamRPN can be divided into, for example... Figure 1 The diagram shows two parts: a Siamese network and a region proposal network. The Siamese network is used for image feature extraction from template frames and detection frames. The region proposal network has two branches: a classification branch and a regression branch. The classification branch distinguishes the target from the background, while the regression branch is used to obtain a more accurate target size and location. Siamese networks generally use common feature extraction networks such as AlexNet and VGGNet.

[0004] Both classification and regression branches involve the crucial step of template matching. For example... Figure 2 As shown, in the target tracking model, during inference computation, the target is tracked within a sequence of video frames. Typically, the first frame is manually labeled as a template frame, while subsequent frames serve as detection frames. The template frame is fed into the network for convolutional feature extraction and then passes through different convolutional layers to generate weights for the classification and regression branches, respectively. These weights serve as matching templates for subsequent frames. When subsequent frames are fed into the network as detection frames, feature extraction is also performed on the backbone network to generate corresponding feature maps. The backbone network here shares weights with the backbone network of the template frame. The feature maps obtained from the detection frames are then fed into the classification and regression branches, respectively. After passing through convolutional layers, the matching templates generated from the template frames are used as convolution kernels for convolution operations. Finally, response maps for each branch are generated, and post-processing is performed to obtain the predicted target bounding boxes. Summary of the Invention

[0005] To address the imbalance between tracking speed and tracking accuracy improvements in most current Siamese network tracking models, this invention proposes a feature matching method for tracking models based on Siamese networks.

[0006] The technical solution adopted by this invention to solve its technical problem is: a feature matching method based on a Siamese network tracking model, comprising the following steps:

[0007] Step S1: The first frame in the video frame sequence of the tracked target is used as the template frame, and the subsequent frames are used as the detection frames. They are fed into the backbone network for convolutional feature extraction to generate the corresponding template feature map z and detection feature map x.

[0008] The template feature map z is used as the input to the classification branch of the region proposal network. The template feature map z is mapped through a convolutional layer, and the dimensionality is increased to 2k channels to obtain the output template feature map. Where k is the number of pre-selected anchor boxes with different aspect ratios; simultaneously, the detection feature map x is used as input, and a convolutional layer is used to map the detection feature map x to obtain the output feature map. ;

[0009] The template feature map z is used as input to the regression branch in the region proposal network. The template feature map z is mapped through a convolutional layer, and the dimensionality is increased to 4k channels to obtain the output template feature map. Where k is the number of pre-selected anchor boxes with different aspect ratios; simultaneously, the detection feature map x is used as input, and a convolutional layer is used to map the detection feature map x to obtain the output feature map. ;

[0010] Template matching is performed on the classification and regression branches using cross-correlation operations respectively:

[0011] ,

[0012] ,

[0013] in Indicates the relevant operation. and These represent template feature maps respectively. and The response feature map obtained by performing grouped convolution operations as the convolution kernel;

[0014] Step S2: Based on the SiamRPN network, a deep cross-correlation module is added to the cross-correlation convolution operation. First, the two parallel front-end convolutional layers and one deep cross-correlation module in the region proposal network are used as a matching module to find the response feature map. and The similar features are then mapped to 1×1 convolutional feature maps through the dimension mapping module in the region proposal network, which respectively realizes foreground and background classification and target coordinate regression.

[0015] Step S3: The cross-correlation module and its front-end convolutional layer in the region proposal network are denoted as the matching module, and the 1×1 convolutional layer is denoted as the dimension mapping module. Based on the baseline algorithm, the input template frame features and detection frame features are matched by two parallel convolutional layers and a deep cross-correlation module to find similar features in the two feature maps, so as to facilitate the subsequent network's classification of the presence or absence of targets and target localization. At the same time, the 1×1 dimension mapping network is used to perform dimension mapping on the feature map after matching calculation, and the matching is enhanced by increasing the number of convolutional layers and channels.

[0016] Step S4: After matching enhancement, the ME-SiamRPN containing the backbone network and the region proposal network is obtained. The region proposal network contains an enhanced matching module and a dimension mapping module. The enhanced matching module is connected after the backbone network to perform feature matching on the template feature map and the detection feature map. The resulting matched feature map is input into the region proposal network to perform dimension mapping for the classification and regression branches.

[0017] Furthermore, in step S1, the response feature map output by the classification branch has 2k channels, with k channels each for the target and background, and their values ​​represent the positive and negative activation values ​​at the corresponding positions; the feature map output by the regression branch has 4k channels, corresponding to the four parameters adjusted for the regression of the k candidate boxes: dx , dy , dw , dh This represents the displacement and scaling parameters between the corresponding anchor frame and the actual frame.

[0018] The enhanced process in step S3 further includes: X mod_in Template diagram used as input to the backbone network. X det_in The detection map, used as input to the backbone network, is processed by the network using the formula before enhancement. X mod = f bone ( X mod_in )and X det = f bone ( X det_in ) Conduct twin backbone network f bone Extraction yields template feature maps. X mod and detection feature map X det Then through the formula M = f Xcorr ( fmod ( X mod ), f det ( X det Using the matching module f Xcorr The template feature map is matched with the detection feature map to obtain the matching feature map. M And then the mapping module f c The feature map is mapped, and copies are made for mapping in different dimensions, using the formula... A cls = f 2k ( f c ( M ))and A reg = f 4k ( f c ( M Obtain the classified response map. A cls Response plot with regression A reg .

[0019] Furthermore, the enhanced backbone network convolution is denoted as... f bone n- This indicates the backbone network relative to the baseline network. f bone The number of layers was reduced by n; adding n layers to the convolutional layers of the matching module resulted in two branches. f mod n+ and f det n+ ; From the formula X mod n- = f bone n- (X mod_in ) and X det n- = f bone n- (X det_in ) Calculate feature map X mod and Xdet After being fed into the enhanced 3×3 convolution module, it is then calculated using the formula... M ad = f Xcorr (f mod n+ (X mod n- )，f det n+ (X det n- )) Perform matching to obtain enhanced matching feature maps M ad .

[0020] The beneficial effects of this invention are as follows: Based on the feature matching enhancement module proposed by Siamese networks, this invention optimizes the network structure of the tracking algorithm, allocating network computational resources to key modules, thereby enabling the algorithm model to achieve optimal tracking performance. The method of this invention has a certain degree of adaptability and scalability, and can be applied to tracking algorithm network models based on Siamese network structures, enhancing tracking accuracy without sacrificing speed performance. Attached Figure Description

[0021] Figure 1 This is a framework diagram of the SiamRPN network in the existing technology;

[0022] Figure 2 This is a diagram of the network inference framework of SiamRPN in the existing technology;

[0023] Figure 3 This is a network framework diagram of the baseline algorithm in the existing technology;

[0024] Figure 4 This is a diagram of the SiamRPN region proposal network framework of the present invention;

[0025] Figure 5 This is the baseline network framework diagram of the present invention;

[0026] Figure 6 This is a schematic diagram of the convolution adjustment of the matching module of the present invention;

[0027] Figure 7 This is a schematic diagram of the enhanced matching module based on adaptive grouped convolution of the present invention;

[0028] Figure 8 This is a diagram of the overall network framework of the ME-SiamRPN of this invention;

[0029] Figure 9This is a comparison diagram of the tracking effect of helicopters in an embodiment of the present invention;

[0030] Figure 10 This is a comparison diagram of the tracking effect of the car in an embodiment of the present invention. Detailed Implementation

[0031] To further illustrate the purpose and technical solution of the present invention, a more detailed description is provided below in conjunction with the accompanying drawings and specific embodiments.

[0032] The present invention discloses a feature matching method based on a Siamese network tracking model, comprising the following steps.

[0033] (1) In the region proposal network, the input is set as template feature map z and detection feature map x. In the classification branch, z is first mapped through a convolutional layer to obtain the output feature map. Simultaneously, dimensionality is increased, with the number of channels increased to 2k (k refers to the number of pre-selected anchor boxes with different aspect ratios). Similarly, the detection image x is also processed through a convolutional layer, but the number of channels is not changed, resulting in... Similarly, convolution mapping is also performed in the regression branch. The difference is that when mapping z, the number of channels is increased to 4k, resulting in a feature map. and detection feature maps with a constant number of channels. .

[0034] Both classification and regression branches involve the crucial step of template matching. In region proposal networks, this is achieved using cross-correlation operations, as shown in the diagram. The "" indicates the relevant operation. The specific operation for template matching in the classification and regression branches is as follows:

[0035] ,

[0036] ,

[0037] The above two equations respectively use template feature maps and Use it as a kernel for grouped convolution. "" represents a grouped convolution operation, which yields the corresponding response feature map. and The classification branch outputs a response feature map with 2k channels, k for the target and k for the background, whose values ​​represent the positive and negative activation values ​​at the corresponding locations. The regression branch outputs a feature map with 4k channels, corresponding to the four parameters adjusted for the regression of k candidate boxes. dx , dy , dw , dh This represents the displacement and scaling parameters between the corresponding anchor frame and the actual frame.

[0038] Region proposal networks, during inference computation, such as Figure 2 As shown, for a video frame sequence of a tracked target, the target is typically manually labeled in the first frame and used as a template frame, while subsequent frames are used as detection frames. When the template frame is fed into the network, it first undergoes convolutional feature extraction and is mapped through different convolutional layers to generate weights for the classification and regression branches, respectively. These two weights serve as matching templates for subsequent frames. When subsequent frames are fed into the network as detection frames, they also first undergo feature extraction in the backbone network to generate corresponding feature maps. The backbone network used here shares weights with the backbone network of the template frame. Then, the feature maps obtained from the detection frames are fed into the classification and regression branches, respectively. After passing through convolutional layers, the matching templates generated from the template frame are used as convolution kernels for convolution operations. Finally, response maps for each branch are generated, and post-processing is performed to obtain the predicted target bounding boxes.

[0039] (2) SiamRPN++ proposes a deep cross-correlation module (DW-XCorr) for cross-correlation convolution operations. Deep cross-correlation utilizes channel-wise convolution, reducing the number of parameters in the feature map obtained from the cross-correlation operation and enhancing training stability. Therefore, the redundancy removal method in this invention adopts the deep cross-correlation module, ultimately resulting in the baseline algorithm structure, as follows: Figure 3 As shown.

[0040] In the baseline algorithm-based network architecture, the region proposal network can be mainly divided into two parts: the first part consists of two parallel front-end convolutional layers and a deep cross-correlation module, and the second part is a 1×1 dimension mapping network. The role of the first part in the region proposal network is to match the template frame features and detection frame features input to this module, finding similar features in the two feature maps to facilitate subsequent network classification of target presence and target localization. The second part performs dimension mapping on the feature maps after the matching calculation.

[0041] (3) such as Figure 4 As shown, based on the functions implemented by the modules in the framework, the deep cross-correlation layer and its front-end convolutional layer in the region proposal network are denoted as the matching module, and the 1×1 convolutional layer is denoted as the dimension mapping module. The matching module consists of a single-layer convolution and a deep cross-correlation layer. The convolution operation is used to enhance the feature matching ability of the cross-correlation. The result is as follows: Figure 5 The baseline algorithm shown.

[0042] One way to enhance the matching module is to increase the number of convolutional layers and channels. However, directly increasing the number of convolutional layers or channels increases the required computational power, reduces tracking speed, and limits the actual performance of the tracking algorithm. Therefore, this approach requires joint optimization design of the overall network structure. That is, while increasing the convolutional layers of the matching module, the number of convolutional layers in the network outside the module (the backbone network) should be reduced by an appropriate amount to keep the overall computational cost of the algorithm from increasing.

[0043] The specific enhancement process is explained as follows: First, remember... X mod_in This is a template diagram for the backbone network input. X det_in This is the detection graph input to the backbone network. Before enhancement, the network passes through the twin backbone network. f bone After extraction, a template feature map is obtained. X mod Detection feature map X det The calculation formula is as follows:

[0044] X mod = f bone ( X mod_in ),

[0045] X det = f bone ( X det_in ),

[0046] Then use the matching module f Xcorr The template feature map is matched with the detection feature map to obtain the matching feature map. M Its calculation formula is M = f Xcorr ( f mod ( X mod ), f det ( X det )), and then the mapping module f c The feature map is mapped, and copies are copied and mapped to different dimensions to obtain the classification response map. A cls Response plot with regression A reg The calculation is as follows:

[0047] A cls = f 2k ( f c ( M )),

[0048] A reg = f 4k ( f c ( M )).

[0049] like Figure 6 As shown in the left figure, the structure of the Siamese network and matching module of the baseline algorithm is illustrated. Template frame image blocks and detection frame image blocks are processed by the backbone network for feature extraction, resulting in their respective feature maps. Since the backbone network is a Siamese structure, the parallel network branches share parameters; therefore, only one branch is represented in the figure. The total number of convolutional layers from the input to the cross-correlation is N, including N-1 layers from the Siamese network and 1 layer from the matching module. After enhancement, as shown in the right figure, n more convolutional layers are added to the matching module, while the number of convolutional layers in the backbone network is reduced by the same amount to ensure that the total number of convolutional layers from the input to the cross-correlation remains N. The n convolutional layers marked with dashed boxes outside the matching module in the left figure are part of the feature extraction network convolutional layers in the Siamese network. Since the number of convolutional layers within the module needs to be increased for matching enhancement, and according to the joint enhancement mechanism described above, the total number of convolutional layers must not increase, this is manifested by pruning the network layers within the dashed lines. Simultaneously, convolutional layers with the same number of layers and channels as the pruned convolutional networks are added to each of the two branches of the matching module. The network structure is represented in the right figure as n convolutional layers marked by dashed boxes within the module.

[0050] The enhanced backbone network convolution can then be denoted as... f bone n- This indicates the backbone network relative to the baseline network. f bone The number of convolutional layers was reduced by n. Simultaneously, to maintain the overall convolutional layer count, the mechanism increased the number of convolutional layers in the matching module by n, i.e., the two branches are denoted as follows: f mod n+ and f det n+ The formula for calculating the feature map is:

[0051] X mod n- = f bone n-(X mod_in ) ,

[0052] X det n- = f bone n- (X det_in ) ,

[0053] Calculated feature map X mod and X det The data is fed into an enhanced 3×3 convolutional module and matched to obtain an enhanced matching feature map. M ad Its calculation formula is M ad = f Xcorr (f mod n+ (X mod n- )，f det n+ (X det n- )) .

[0054] From a computational perspective, the computational cost of deep cross-correlation matching remains roughly the same before and after joint optimization, both consisting of N layers with the same number of channels. This improves tracking accuracy without increasing the overall computational power required by the network model. On the other hand, the deep cross-correlation module uses separable convolutions to obtain the relevant features for each channel. However, subsequent networks need to perform feature map dimension mapping, which requires further enhancement of the information interaction between channels. Therefore, this paper introduces Adaptive Grouped Convolution (AGC), and the enhanced matching module structure is as follows: Figure 7 As shown in the figure, the left side is an enhanced matching module based on adaptive grouping convolution, which mainly consists of a matching module plus adaptive grouping convolution.

[0055] The network structure of the adaptive grouped convolutional module is shown on the right side of the figure. When the input features... Figure X First, the number of groups g is adaptively obtained based on the number of channels in the input feature map. Then, the feature maps of each group are convolved with a 1×1 convolution to obtain the enhanced features. Figure X '.

[0056] The number of groups, g, needs to be calculated based on the number of channels, C, which should be positively correlated with g. Let's assume their relationship function is... And assume a simple linear relationship as shown in the following equation:

[0057] C = γ · g + b ,

[0058] in γ and b It is a constant parameter, which is set in this paper. γ =1, b The value is 4.

[0059] Considering that the input feature map channel C is generally a power of 2, it can be transformed into a nonlinear function as shown in the following formula: Considering that g is a power of 2, we use the "power" operator to take the power of 2 closest to the final solution.

[0060] The number of groups, g, needs to be calculated based on the number of channels, C, which should be positively correlated with g. Let's assume their relationship function is... And assume a simple linear relationship as shown in equation (11), where γ and b It is a constant parameter, which is set in this paper. γ =1, b The value is 4. Considering that the input feature map channel C is generally a power of 2, it is transformed into a nonlinear function as shown in equation (12). Solving (12) for g yields equation (13). Considering that g is a power of 2, the "power" operator is used to take the power of 2 closest to the final solution.

[0061] (11)

[0062] (12)

[0063] (13)

[0064] Based on the adaptive grouped convolution proposed in this paper, let the input feature map be the enhanced matching feature map. M ad The resulting matching feature map after grouped convolution is M ad ′.

[0065] Then it is calculated using the following formula:

[0066]

[0067]

[0068] They are then fed into the classification and regression branches respectively to perform mappings along different dimensions. f 2k and f 4k .

[0069] (4) After completing the matching enhancement module through the above design, ME-SiamRPN is obtained, and its overall network structure is as follows: Figure 8 As shown, the network mainly consists of two sub-networks: a backbone network and a region proposal network. The region proposal network includes an enhancement matching module and a dimension mapping module. The enhancement matching module, connected after the backbone network, performs feature matching on the template feature map and the detection feature map. The matched feature map obtained from the matching module undergoes necessary parallel branching and is input into the region proposal network for necessary dimension mapping of the classification and regression branches.

[0070] After completing the matching enhancement module as described above, we obtain ME-SiamRPN, whose overall network structure is as follows: Figure 8 As shown, the network mainly consists of two sub-networks: a backbone network and a region proposal network. The region proposal network includes an enhancement matching module and a dimension mapping module. The enhancement matching module, connected after the backbone network, performs feature matching on the template feature map and the detection feature map. The matched feature map obtained from the matching module undergoes necessary parallel branching and is input into the region proposal network for necessary dimension mapping of the classification and regression branches.

[0071] The specific network model structure of ME-SiamRPN is shown in the table below (comparison of the network layers of the baseline tracking algorithm and tiny-SiamRPN).

[0072] .

[0073] The table on the left shows the network structure of the baseline tracking algorithm, while the table on the right shows the network structure of the tracking algorithm designed in this paper. The table mainly represents the convolutional and pooling layers in the neural network. The convolutional layer is represented as "conv<kernel size>-<number of kernels>", and the pooling layer is represented as "maxpool". Other special modules include the deep cross-correlation module represented as "DW-Xcorr" and the adaptive grouped convolution module represented as "AGC". The adjusted structures of the algorithms on both sides are indicated in bold.

[0074] The tracking network has two input branches: the template branch takes the initial frame target image patch as input (112×112×3), and the detection branch takes the search region image from subsequent frames (224×224×3). The following section mainly describes the adjustments made due to network structure improvements. To enhance the matching module without significantly increasing computational power and parameters, two "conv3-32" convolutional layers were added. Simultaneously, to achieve channel interaction while minimizing computational resources, the two "conv1-64" convolutions in the baseline network were replaced with adaptive grouped convolutions, corresponding to the enhancement of the matching module. The final output response map of the network contains the number of preselected boxes k; in this paper, k is set to 3.

[0075] This invention presents comparative experiments on the design of the aforementioned matching enhancement module to verify the effectiveness of the design method. The experiments were conducted on a unified dataset and verified the tracking improvement effect of ME-SiamRPN in multiple scenarios.

[0076] This experiment uses success rate, normal precision, and frame rate (FPS) as the evaluation metrics for tracking performance. Among the precision evaluation metrics, intersection-over-union (IoU) and center pixel error (PixelError) are two fundamental metrics. Success rate and normal precision are defined around these two metrics.

[0077] Center pixel error refers to the Euclidean distance between the center point of the predicted bounding box of the target and the center point of the actual bounding box of the target. Its calculation formula is as follows:

[0078] ,

[0079] In the above formula x , y These are the x and y coordinates of the center pixel of the prediction bounding box, respectively. x G , y G These represent the horizontal and vertical coordinates of the center pixel of the bounding box. The center pixel error can roughly reflect the accuracy of the tracking algorithm's prediction of the target position. The smaller the distance between the predicted position and the actual position of the target, the more accurate the algorithm's prediction; conversely, the larger the center pixel error, the lower the accuracy of the algorithm's prediction.

[0080] The intersection-union ratio (IUU) is the ratio of the intersection to the union of the area predicted by the algorithm and the actual area of ​​the target area. The formula is as follows:

[0081] ,

[0082] In the above formula AG Represents the predicted area. A T This represents the actual area of ​​the target region. We can also determine the intersection and union ratio. IoU ∊[0, 1], close to 1 represents a state where the predicted box and the actual box are close to overlapping, that is, the prediction accuracy of the algorithm in this frame is high; conversely, when the value of the cross-union ratio is close to 0, it means that the predicted box and the actual target box have almost no overlapping area, that is, the prediction accuracy of the algorithm in this frame is low.

[0083] Accuracy refers to the proportion of frames in which the algorithm accurately predicts the target position out of the total number of video frames during the entire tracking process. The criterion for accurate prediction is that the error of the center pixel in that frame calculated by the algorithm is less than a set error threshold. The specific calculation process for accuracy is shown in the following formula:

[0084] ,

[0085] In the above formula bool The value is 1 if the condition in parentheses is met, and 0 otherwise. (In the total number of frames) n In the middle, the first i Center pixel error in frame calculation PixelError i If it is less than the set error threshold T 1 (This article sets the resolution to 40 pixels) then considers the first... i If the frame prediction is accurate, then bool The function count is 1. This statistical method is used to find the proportion of frames with accurate predictions out of the total number of frames; this proportion represents the calculated accuracy. precision Average accuracy refers to the accuracy calculated for tracking video frames excluding cases where the image is completely occluded or moves out of the field of view. The calculation formula is as follows:

[0086] ,

[0087] In the above formula n 1 This refers to the number of valid frames remaining after removing scenes with complete occlusion from the total number of frames. The remaining calculation and statistical methods are consistent with those for accuracy; the higher this value, the higher the algorithm's accuracy. Success rate refers to the ratio of successfully predicted frames to the total number of video frames during tracking. The criterion for determining if a frame was predicted successfully is whether the intersection-over-union ratio (IoU) calculated by the tracking algorithm for that frame is greater than a set threshold.

[0088] The specific calculation process for the success rate is shown in the following formula:

[0089] ,

[0090] In the above formula boolThe value is 1 if the condition in parentheses is met, and 0 otherwise. (The total number of frames is...) n In the middle, the first i Frame crossover ratio IoU i If it exceeds the set threshold T 2 (This is set to 0.4) Then this frame is counted as a successfully predicted frame. The higher the success rate obtained by comparing the number of successfully predicted frames to the total number of frames, the higher the overall accuracy of the algorithm.

[0091] Under the same tracking test set, the improvement effects of ME-SiamRPN are shown in the improvement effect table of ME-SiamRPN network joint design and the improvement effect table of ME-SiamRPN network AGC.

[0092] The improvement effect of ME-SiamRPN network AGC is shown in the table below:

[0093]

[0094] In the table above, the overall optimization design of the backbone network and matching module improved the tracking success rate and accuracy by 1.3%, but also reduced the frame rate by 16.2 FPS.

[0095] The improvement effect of ME-SiamRPN network AGC is shown in the table below:

[0096] .

[0097] As shown in the table above, the success rate improved by 0.9%, and the average accuracy improved by 1.8%, indicating that the AGC module has a certain optimization capability for feature maps. There was also a slight improvement of 16.6 FPS in frame rate, which is the result of AGC replacing a portion of the network in the dimension mapping module and reducing parameters after performing grouped convolution calculations.

[0098] The final improvement results of the ME-SiamRPN network are shown in the table below:

[0099] .

[0100] Ultimately, the performance improvement of ME-SiamRPN compared to the baseline is shown in the table above. Its speed remains essentially unchanged, but it improves the success rate and average accuracy by 2.2% and 3.1%, respectively. The module of this invention enables the baseline algorithm to improve accuracy without decreasing speed, validating the effectiveness of the design.

[0101] Furthermore, this paper compares the tracking performance of the proposed algorithm with baseline algorithms for some real-world scenarios and targets. A portion of these comparisons is shown here:

[0102] Figure 9 In the case of rapid helicopter movement and changes in attitude and scale, the target prediction bounding box of the proposed algorithm exhibits higher localization accuracy. Furthermore, the proposed algorithm demonstrates stronger robustness against interference from the helicopter rotor compared to the baseline algorithm.

[0103] exist Figure 10 In the case of interference from similar targets (sedans with similar shapes) around the target car or background interference similar to the target, the algorithm in this paper shows stronger stability.

[0104] The dataset and platform environment used in this experiment are as follows: the training dataset is ILSVRC, and the test dataset is a self-built dataset, which includes 257 videos of 16 target categories such as airplanes, ships, cars, and pedestrians, with a total of 580,453 images. The hardware and software platform includes: Intel(R) Core(TM) i7-8700 CPU@3.20GHz*12; GPU: Force GTX1080Ti; operating system: ubuntu14.04LTS; deep learning framework: pytorch.

[0105] The learning rate for training weights in this experiment was set to 0.005, the weight decay rate was 0.0001, each batch contained 256 images, the total number of iterations was 17550, the number of epochs was set to 40, and the number of iterations per epoch was 390.

[0106] Those skilled in the art will readily understand that the above description is merely a preferred use case of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A feature matching method based on a twin network-based tracking model, characterized in that: Comprising the following steps Step S1, taking the first frame in the video frame sequence of the tracking target as a template frame and the subsequent frames as detection frames, inputting the network for convolution feature extraction to generate weights for the classification branch and the regression branch, and generating corresponding template feature maps z and detection feature maps x; The template feature map z is taken as an input of a classification branch in a region proposal network, the template feature map z is mapped through a convolutional layer, and meanwhile, dimension is increased to increase the number of channels to 2k, to obtain an output template feature map where k is the number of preselected anchor boxes of different aspect ratios; meanwhile, the detection feature map x is taken as an input, the detection feature map x is mapped through a convolutional layer, to obtain an output feature map ; The template feature map z is taken as an input of a regression branch in a region proposal network, the template feature map z is mapped through a convolutional layer, and meanwhile, the channel number is increased to 4k through dimension increasing, so that an output template feature map is obtained wherein k is the number of anchor boxes of different aspect ratios selected in advance; meanwhile, the detection feature map x is taken as an input, the detection feature map x is mapped through a convolutional layer, and an output feature map is obtained ​ The classification branch and the regression branch are matched respectively by using the cross-correlation operation: ， ， wherein denotes a correlation operation, and denotes a response feature map obtained by a grouped convolution operation with the template feature map and as a convolution kernel, respectively. Step S2, based on the SiamRPN network, a deep cross-correlation module is added in the cross-correlation convolution operation, first, two front-end convolution layers and a deep cross-correlation module in the region proposal network are taken as a matching module, and similar features in the response feature map and are found, and then the dimension mapping module in the region proposal network is used for 1×1 convolution dimension feature map mapping, so as to realize foreground and background classification and target coordinate regression respectively. Step S3, the cross-correlation module and the front-end convolution layer in the region proposal network are recorded as a matching module, and the 1x1 convolution layer is recorded as a dimension mapping module, based on the baseline algorithm, the template frame features and the detection frame features are matched by two convolution layers and a deep cross-correlation module arranged in parallel from top to bottom, similar features in the two feature maps are found, and the dimension mapping of the feature map after the matching calculation is performed through the 1x1 dimension mapping network, and the matching is enhanced by increasing the number of convolution layers and the number of channels; Step S4, the ME-SiamRPN containing the backbone network and the region proposal network is obtained after the matching is enhanced, wherein the region proposal network contains the enhanced matching module and the dimension mapping module, the enhanced matching module is connected after the backbone network, the feature matching of the template feature map and the detection feature map is performed, the matching feature map obtained is input into the region proposal network, and the dimension mapping of the classification branch and the regression branch is performed. 2.The feature matching method of a twin network-based tracking model according to claim 1, wherein, The response feature map output by the classification branch in step S1 has 2k channels, wherein k channels are for the target and k channels are for the background, and the values represent positive and negative activation values at corresponding positions; the feature map output by the regression branch has 4k channels, corresponding to 4 parameters for adjusting k candidate boxes: dx , dy , dw , dh represent displacement and scaling parameters of the corresponding anchor box and the real box. 3.The feature matching method of a twin network based tracking model according to claim 2, wherein, The enhancement process in step S3 is as follows: [The process is described in the original text, which is not translated here.] X mod_in and X det_in The template graph and detection graph, which serve as inputs to the backbone network, are derived from the formula... X mod = f bone ( X mod_in )and X det = f bone ( X det_in ) Conduct twin backbone network f bone Extraction yields template feature maps. X mod and detection feature map X det Then through the formula M = f Xcorr ( f mod ( X mod ), f det ( X det Using the matching module f Xcorr The template feature map is matched with the detection feature map to obtain the matching feature map. M And then the mapping module f c The feature map is mapped, and copies are made for mapping in different dimensions, using the formula... A cls = f 2k ( f c ( M ))and A reg = f 4k ( f c ( M Obtain the classified response map. A cls Response plot with regression A reg .

4. The feature matching method of claim 3, wherein, The enhanced backbone network convolution is denoted as f bone n- , which represents the backbone network of the enhanced network relative to the baseline network f bone n layers are reduced; meanwhile, n layers are added to the convolutional layers of the matching module to obtain two branches f mod n+ and f det n+ ; From the formula X mod n- f bone n- (X mod_in ) and X det n- f bone n- (X det_in ) The feature map is calculated X mod and X det , and after being input into an enhanced 3x3 convolution module, the calculation formula M ad f Xcorr (f mod n+ (X mod n- )，f det n+ (X det n- )) The enhanced matching feature map is obtained by matching M ad .​​​

Images