Robust online learning ship tracking method based on twin network

Abstract

The application discloses a robust online learning ship tracking method based on a twin network, which firstly extracts a series of candidate target frames by using an anchor frame-based method, then extracts features of templates and detection frames respectively by using a twin network, uses a region selection network for classification and anchor frame regression, introduces an adaptive online update network UpdateNet, fuses SiamRPN classification scores and online classifier scores, and establishes a shielding judgment mechanism according to the classification fusion scores, finally tests on an inland ship dataset and evaluates tracking performance. The application improves the SiamRPN tracking model, so that it is suitable for inland water scenarios, thereby improving the target tracking drift problem under the mutual shielding condition of inland ships. The application applies the twin method to inland ship tracking, and the performance is better than that of existing methods, and the application shows strong robustness under the shielding scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of inland ship target tracking, and particularly relates to a robust online learning ship tracking method based on a twin network. BACKGROUND

[0002] At present, the technical means applied to inland ship monitoring at home and abroad mainly include an automatic identification system (AIS) and radar tracking. However, the AIS has problems such as not fully playing the function and not fully utilizing the information. The radar tracking is easily disturbed by waves, coastal scenery and other factors, and has a certain blind area. Therefore, video monitoring as an auxiliary technical means helps to improve the intelligent level of inland supervision and has a positive role in the safe operation of inland shipping.

[0003] In recent years, many researchers have used computer vision techniques to analyze target information in videos to achieve ship detection and tracking. Xiao et al. (Xiao L, Xu M, Hu Z. Real-time inland CCTV ship tracking[J]. Mathematical Problems in Engineering, 2018, 2018:1-10.) designed a short-term tracker based on random projection under the TLD framework, which can significantly alleviate the tracking drift of ships when occluded. Chen et al. (Chen X, Wang S, Shi C, et al. Robust ship tracking via multi-view learning and sparse representation[J]. The Journal of Navigation, 2019, 72(1):176-192.) proposed a framework integrating multi-view learning algorithms and sparse representation methods to extract highly coupled and robust ship descriptors from multiple different ship features. Since target ships are often partially or completely occluded by maritime obstacles, Chen et al. (Chen X, Xu X, Yang Y, et al. Augmented ship tracking under occlusion conditions from maritime surveillance videos[J]. IEEE Access, 2020, 8:42884-42897.) proposed an augmented ship tracking framework using KCF and curve fitting algorithms. However, the above methods usually use manually designed features and lack generalization ability for challenging application scenarios. Recently, deep learning-based methods have shown good performance in ship tracking. Shan et al. (Shan Y, Zhou X, Liu S, et al. SiamFPN: A deep learning method for accurate and real-time maritime ship tracking[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 31(1):315-325.) used an improved twin network combined with multiple RPNs to construct a maritime ship tracking framework.Yang et al. (Yang X, Wang Y, Wang N, et al. An Enhanced SiamMask Network for Coastal Ship Tracking [J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-11.) proposed an enhanced SiamMask network for coastal ship tracking to address the lack of contour and edge information extracted by twin networks. Although twin network-based methods are widely used in ground ship tracking, there are few studies on ship tracking, especially for inland river scenes.

[0004] However, there are some obvious shortcomings in directly applying twin networks to inland ship tracking. First, twin networks have poor discrimination ability in distinguishing targets from backgrounds when facing similar object interference. Second, twin networks cannot handle target occlusion well, which may cause target tracking drift or even loss. Finally, twin networks usually update the template every frame, but too frequent updates may introduce too much background information into the target model, leading to template degradation and increasing the probability of tracking drift. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art and provide a robust online learning ship tracking method based on twin networks.

[0006] The present application is achieved by the following technical solution: a robust online learning ship tracking method based on twin networks, comprising the following steps:

[0007] (1) Fixing k different aspect ratios and a fixed scale, use anchor-based method to extract k x 1 different size candidate target boxes from the initial frame;

[0008] (2) Use twin networks to extract features of the template z and the detection frame x, respectively. The twin network architecture uses the first 5 convolutional layers of AlexNet, and discards the edge padding and fully connected layers. The online training classifier also uses the twin network architecture, and generates multiple initial training samples by data augmentation on the initial frame;

[0009] (3) Send the features of the template z and the detection frame x obtained in step (2) into the classification branch and the regression branch of the region selection network RPN, respectively, and then perform cross-correlation operation on the features of the detection frame x by taking the features of the template z as the convolution kernel, to generate the classification prediction value f cls and the position regression value f reg; the training samples obtained in step (2) are compressed, double attention mechanism and filter operation respectively, so as to generate classification confidence f C ;

[0010] (4) Introducing adaptive online updating network UpdateNet for improving the template degradation problem in tracking process; under the condition of given initial template, accumulated template and current frame template, the offline trained UpdateNet model is used to estimate the optimal accumulated template of next frame; wherein, the initial template and the current frame template are the features extracted from the template z and the detection frame x in step (2);

[0011] (5) The classification prediction value f cls obtained in step (3) is fused with the online classifier score f C , and the anchors extracted in step (1) are combined to find the target position loc predicted by the current frame, which is the left upper corner coordinate, width and height of the target; in addition, a shielding judgment mechanism is established according to the classification fusion score; when it is judged that the target is in the shielding state, the target template is not updated, that is, step (4) is not executed; at the same time, the global search mechanism is started to reposition the target, so as to avoid target tracking drift;

[0012] (6) Steps (1)-(5) are performed for each frame image in the inland ship data set to realize robust online learning ship tracking based on twin network.

[0013] Further, in step (1), k=5, ratios=[0.33, 0.5, 1, 2, 3], and scale=[8]; in step (2), the template with a size of 127x127x3 and the detection frame with a size of 271x271x3 are input into the twin network for feature extraction, that is, the same transformation is performed on the two inputs z and x to generate feature maps with sizes of 6x6x512 and 24x24x512 respectively and In step (3), the feature maps and are sent into the classification branch and the regression branch respectively, and 3x3 convolution operation is performed to obtain wherein the subscript "cls" represents the classification branch, and "reg" represents the regression branch.

[0014] Further, step (3) includes the following sub-steps:

[0015] (3.1) For the classification branch, it is used to predict the target-background classification score of each candidate region; according to step (3), can be regarded as a convolution kernel, and The similarity score is obtained by convolution, thus generating a classification response map

[0016] (3.2) For the regression branch, it gives the position regression value of each candidate region; according to step (3), It can be regarded as a convolution kernel, which is convolved on to obtain the position regression value

[0017] Further, in step (4), the UpdateNet model is updated by using a learning function Φ, which is actually a convolutional neural network, and the formula of the Φ function is as follows:

[0018]

[0019] The updated template is calculated by giving the initial frame Ground-Truth (GT) template The historical frame cumulative template and the template T extracted from the predicted target position in the current frame i

[0020] Further, step (5) includes the following sub-steps:

[0021] (5.1) The online classifier mainly includes a compression module, an attention module and a filter module; first, the compression module is dimensionally compressed by using a random projection method; then, the double attention mechanism is composed of a channel attention mechanism and a spatial attention mechanism; finally, the filter module is used to generate a classification confidence; therefore, according to the above, the online classifier is defined as follows:

[0022]

[0023] Wherein, w represents the online learning network parameter, C represents the compression module, A represents the attention module, and F represents the filter module; (5.2) The objective function of online learning in step (5.1) is defined in a similar form to the L2 classification error:

[0024]

[0025] Each training sample x i is affected by the weight γ i control; y i ∈R W*H is the labeled classification confidence, which is labeled by a Gaussian function centered on the predicted target position; the regularization penalty on w k is set by λ k ;

[0026] ​(5.3) Linearly weighted fusion is adopted to fuse the offline Siamese network classification score obtained in step (3.1) and the score generated by the online classifier in step (5.1) to obtain an adaptive classification score: Figure Two

[0027] f fusion (x;w)=λf C (x;w)+(1-λ)f cls (z,x;w)

[0028] Wherein, λ is an influence factor of the online confidence score;

[0029] (5.4) According to the change degree of the adaptive classification score obtained in step (5.3), a corresponding occlusion judgment mechanism is proposed; and the current tracking state is evaluated according to the adaptive classification score to determine whether the template should be updated and whether the search area should be expanded;

[0030] (5.5) The global search strategy adopts the traditional sliding window method, and template matching operation is performed on the current window through sliding; if the similarity score obtained by the current window is higher than the threshold thl, it is considered that the target position loc is detected.

[0031] Further, in step (5.3), λ=0.9.

[0032] Further, in step (5.4), t h =0.93, t l =0.92, and tol=0.02.

[0033] Further, in step (5.5), thl=0.5.

[0034] The method has the advantages that the application proposes a robust online learning ship tracking method based on a Siamese network, which is used to improve the target tracking drift problem under the occlusion of ships. The method is based on the SiamRPN framework, fuses the offline Siamese network classification score and the online classifier score for discriminative learning, and establishes an occlusion judgment mechanism according to the classification fusion score. When it is judged that the target is in the occlusion state, the target template is not updated, and the global search mechanism is started to reposition the target, so as to avoid the target tracking drift. In addition, an efficient adaptive online update strategy UpdateNet is introduced to improve the template degradation problem in the tracking process. The application has strong robustness in the occlusion scene of inland ships and has good value and prospect in practical application. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure One is the overall framework diagram of the method of the application; ​

[0036] Figure Two is the precision curve of the method of the present application;

[0037] Figure Three is the success rate curve of the method of the present application. DETAILED DESCRIPTION

[0038] The core technology of the present application is to linearly weight and fuse the offline twin network classification score and the online classifier score, apply it to the inland ship tracking, and establish a shielding judgment mechanism according to the classification fusion score, so as to realize the high-precision inland ship tracking shielding problem.

[0039] The present application proposes a robust online learning ship tracking method based on twin network, including the following steps:

[0040] (1) Motion model: fixedly set k different height-width ratios and 1 fixed scale, use anchor-based method to extract k*1 different size candidate target boxes from the initial frame, predict the area where the target in the current frame may appear according to the position of the target in the initial frame, and solve the scale problem; specifically including the following sub-steps:

[0041] (1.1) In the initial frame, use Anchor generator to generate k different size candidate target boxes anchors, wherein k=5, ratios=[0.33, 0.5, 1, 2, 3], scale=[8].

[0042] According to the above method, 5 different size anchors are obtained.

[0043] (2) Feature extraction: use twin network to extract the features of the template z and the detection frame x respectively, the twin network architecture adopts the first 5 convolutional layers of AlexNet, and discards the edge padding and full connection layer; specifically including the following sub-steps:

[0044] (2.1) input the template with a size of 127*127*3 and the detection frame with a size of 271*271*3 into the twin network for feature extraction, that is, the same transformation is performed on the two inputs z and x 6*6*512 and 24*24*512 feature maps are generated respectively and

[0045] (2.2) the online training classifier also adopts the twin network architecture, and generates 30 initial training samples by data augmentation on the initial frame.

[0046] In this way, the template feature Detect frame features and 30 initial training samples.

[0047] (3) Observation model: the template features and the detection frame features are respectively sent to the classification branch and the regression branch of the region proposal network RPN, and then the template features are regarded as a convolution kernel to perform cross-correlation operation on the detection frame features, thereby generating the classification prediction value f cls and the position regression value f reg , the training samples obtained in step (2.2) are input into the online classifier, thereby generating the classification confidence f C ; specifically including the following sub-steps:

[0048] (3.1) the feature maps and are respectively sent to the classification branch and the regression branch, and 3x3 convolution operation is performed to obtain wherein the subscript "cls" represents the classification branch, and "reg" represents the regression branch.

[0049] (3.2) for the classification branch, it is used to predict the target-background classification score of each candidate region; can be regarded as a convolution kernel, and the similarity score is obtained by convolution on , thereby generating the classification response map:

[0050]

[0051] (3.3) for the regression branch, it gives the position regression value of each candidate region; and is regarded as a convolution kernel, and the position regression value is obtained by convolution on :

[0052]

[0053] through the SiamRPN model trained offline, the offline twin classification score map f cls (z, x) and the position regression value f reg (z, x) can be obtained.

[0054] (3.4) The online classifier mainly includes a compression module, an attention module, and a filter module. The compression module is dimensionally compressed using the random projection method, which can achieve efficient real-time calculation. In addition, in order to solve the problem of foreground and background imbalance, the attention mechanism is used to enrich the capture of semantic information, and a double attention mechanism is introduced to fully extract the features of specific targets. The double attention mechanism is composed of spatial attention composed of two fully connected layers after global average pooling, and channel attention composed of Softmax after channel average. Finally, the filter module is used to generate the classification confidence f C . Therefore, according to the above, the online classifier can be defined as follows:

[0055]

[0056] where w represents the online learning network parameter, C represents the compression module, A represents the attention module, and F represents the filter module.

[0057] (3.5) The objective function of online learning in step (3.4) is defined in a similar form to the L2 classification error:

[0058]

[0059] Each training sample x i is affected by the weight γ i . y i ∈R W*H is the labeled classification confidence, which is labeled with a Gaussian function centered on the predicted target position. The regularization penalty on w k is set by λ k . The online classifier optimization part uses the Gauss-Newton descent method to solve the quadratic problem.

[0060] (4) Model update: an efficient adaptive online update network UpdateNet is introduced to improve the template degradation problem in the tracking process; an offline trained update network model is used to estimate the optimal accumulated template of the next frame online under the condition of given initial template, accumulated template and current frame template, wherein the initial template and the current frame template are both extracted from the template and the detection frame in step (2.1); specifically including the following sub-steps:

[0061] (4.1) A general learning function Φ is used to update the template, which is actually a two-layer convolutional neural network with strong feature expression ability and the ability to learn from a large amount of data. The Φ function formula is as follows:

[0062]

[0063] By providing an initial frame ground-truth (GT) template Historical frame accumulation template And the template T extracted from the target location predicted in the current frame. i To calculate the updated template Essentially, this function integrates the current frame T i New information to update previously accumulated templates Therefore, Φ can adapt to the specific update requirements of the current tracking state based on the differences between the current template and the historically accumulated templates. Furthermore, since the initial frame contains highly reliable target information, an initial frame template is considered. It improves robustness to model drift.

[0064] (4.2) Extract from the true target position GT in the initial frame To obtain the T of the current frame i Use the accumulated template from all previous frames. To predict the target's position in the i-th frame, features are extracted from that region. The extracted features... Current frame feature T i and characteristics of historical accumulation The new input is formed, then processed through a series of convolutional layers, and outputs a new accumulated template for prediction. For the first frame, T i and All set to This is because there are no previous frames. The only real information the model uses to update the network is the position of the given target in the initial frame; all other inputs are based on predictions.

[0065] The new template for the current frame can be obtained by using the offline-trained UpdateNet model.

[0066] (5) Method Integration: The classification prediction value obtained in step (3.2) is fused with the online classifier score obtained in step (3.4), and combined with the anchors generated in step (1.1), the predicted target position loc in the current frame is found, which is the upper left corner coordinate, width and height of the target; in addition, an occlusion determination mechanism is established based on the classification fusion score. When the target is determined to be in an occluded state, the target template is not updated, that is, step (4) is not executed; at the same time, a global search mechanism is started to relocate the target, thereby avoiding target tracking drift; specifically including the following sub-steps:

[0067] (5.1) A linear weighted fusion method is used to combine the offline Siamese network classification score obtained in step (3.2) with the score generated by the online classifier in step (3.4). Figure TwoThe online classifier's score map is converted to the same spatial size as the twin classifier's score map by bicubic interpolation, and then fused with a certain weight to obtain the adaptive classification score:

[0068] f fusion (x; w) = λf C (x; w) + (1 - λ)f cls (z, x; w)

[0069] where λ is the influence factor of the online confidence score. When λ = 0.9, the optimal tracking performance can be obtained.

[0070] (5.2) A corresponding occlusion decision mechanism is proposed by the variation degree of the adaptive classification score obtained in step (5.1). Then, the current tracking state can be evaluated according to the adaptive classification score to decide whether the model should be updated and whether the search region should be enlarged.

[0071] Assume Z k represents the tracking state in the k-th frame. Here, Z k = 0 indicates a stable state, which represents that the tracking performance is stable. Z k = 1 indicates an occlusion state, which represents that the tracking performance is deteriorated and the tracker tries to recover the performance. In addition, assume three thresholds (a high threshold t h , a low threshold t l and a tolerance tol). For the first frame, Z k is set to zero, S k represents the highest classification score in the k-th frame, and ΔS k = S k - S k-1 . For the k-th frame, the state transition of the occlusion decision mechanism is shown in Table 1.

[0072] Table 1: State transition table of the occlusion decision mechanism

[0073] Serial Number Z k+1 ]]> kth Frame Condition 1 0 [CAT k > 0, ΔS k > 0, Z k = 0]]> 2 0 S k t h ,ΔS k >0,Z k =1]]> 3 0 S k >0, ΔS k <0, Z k =0, |ΔS k | < tol 4 1 S k <t h ,ΔS k >0,Z k =1]]> 5 1 ΔS k <0,Z k =1]]> 6 1 AS k <0, AS k |> tol]]> 7 1 S k <t l ,ΔS k >0,Z k =1 ]]>

[0074] In Table 1 above, when in conditions 1-3, the tracker will update the model in the next frame. The meanings of these conditions can be summarized as follows: if the current tracker state remains stable, we should consider the current classification score S k and the classification score difference ΔS k . If they are both greater than zero, it is considered that the tracker performance is good. Even if ΔS k<0, the model does not need to be updated either. If the current frame is in the occlusion state, the model can be updated in the next frame until the classification score is large enough, otherwise the model still cannot be updated, as shown in condition 4. For conditions 5-7, the model should stop updating in the next frame. The meanings of these conditions are: if the current frame is in the occlusion state, but the degree of change of the classification score is greater than tol, the model cannot be updated in the next frame; if the degree of change of the classification score is greater than tol, the model also cannot be updated; if the classification score decreases significantly, we should also prevent it from updating in the next frame. When t k = 0.93, t h = 0.92, tol = 0.02, the performance of the occlusion determination mechanism is optimal. l

[0075] (5.3) When the target is long-time occluded, causing the tracking to fail, the global search strategy can detect the target again, making the tracker recover from the error. In this way, even in the case of severe occlusion of the target, it can be tracked for a long time. The global search strategy uses the traditional sliding window method, and performs template matching on the current window each time the window is slid. If the similarity score of the current window is higher than the threshold thl, it is considered that the target position loc is detected. After detecting each different window size, there will be a part with high repetition, and finally the detected target is obtained after non-maximum suppression screening. When thl = 0.5, the optimal similarity matching performance can be obtained.

[0076] According to the above method, strong robustness can be exhibited in the occlusion scene of inland ships.

[0077] (6) Steps (1)-(5) are performed for each image in the inland ship data set, and accurate tracking and evaluation results are output; specifically including the following sub-steps:

[0078] (6.1) The models trained offline in steps (2)-(4) are tested on the inland ship data set to realize the target tracking task of the inland ship. The success rate (Success rate) and precision (Precision) are used to evaluate the tracking performance. The success rate curve is obtained by calculating the center location error (Center Location Error, CLE), and the precision curve is obtained by calculating the overlap score (Overlap Score, OS).

[0079] In pixels, let (x t ,y t ) represent the predicted target center position, and (x g ,y g ) represent the manually labeled target center position, then the center location error calculation formula is as follows:​

[0080]

[0081] The bounding box b predicted by the tracking algorithm t and the GT bounding box b annotated by human g Then the overlap ratio can be expressed as:

[0082]

[0083] The overall performance of the whole sequence can be evaluated by the average center position error and the average overlap ratio of each frame in the sequence. The greater the values of the two indicators, the better the tracking performance.

[0084] (1) The present application linearly weights and fuses the offline twin network classification score and the online classifier score to achieve the purpose of discriminative learning, so as to effectively distinguish the target and the background.

[0085] (2) The present application establishes a shielding judgment mechanism according to the classification fusion score, evaluates the current tracking state to determine whether the model should be updated and whether the search area should be expanded. When it is judged that the target is in a shielding state, the target template is not updated, and a global search mechanism is started to reposition the target, so as to avoid target tracking drift.

[0086] (3) The present application introduces an efficient adaptive online update strategy UpdateNet. Under the condition of given initial template, accumulated template and current frame template, the UpdateNet model trained offline is used to estimate the optimal template of the next frame online, so as to effectively improve the template degradation problem in the tracking process.

[0087] Embodiment

[0088] The beneficial effects of the method of the present application are demonstrated by a specific example below.

[0089] An embodiment of the present application is implemented on a machine equipped with an Intel Core i5-9400 CPU processor and 48 GB of memory. Using the parameter values listed in the specific embodiments, the experimental data used by the present application is a river ship dataset established by collecting or field collecting some port ship videos. The video sequence is divided into 8 common attributes, and the video attributes are: illumination variation (IV), out-of-plane rotation (OPR), scale variation (SV), occlusion (OCC), in-plane rotation (IPR), out-of-view (OV), background clutter (BC), and low resolution (LR). The specific information of the dataset is shown in Table 2.

[0090] Table 2: Detailed description of the river ship dataset (Y: Yes, N: No)

[0091]

[0092]

[0093] Based on the river ship dataset, the overall framework diagram, the success rate / precision curve diagram, and the tracking screenshots of some video sequences shown in the accompanying drawings are obtained.

[0094] As shown in Figure One , the present application proposes an overall framework diagram of a robust online learning ship tracking method based on a twin network. It mainly includes the following five modules:

[0095] Module a: the feature maps of the template and the detection frame are extracted using a twin subnetwork, respectively.

[0096] Module b: the RPN subnetwork is composed of two branches: one is a classification branch for target-background classification, and the other is a regression branch for obtaining more accurate bounding box prediction results.

[0097] Module c: the online classifier mainly includes a compression module, an attention mechanism module, and a filter module. This classifier is completely learned during online tracking, and can predict the confidence of the target from the backbone features (extracted by the twin subnetwork) extracted from the current frame, so as to distinguish the target from other objects in the scene.

[0098] Module d: directly using the offline trained UpdateNet model, in the case of the given initial template, cumulative template and the template of the current frame, online estimating the optimal template of the next frame.

[0099] Module e: evaluating the tracking state through the occlusion judgment mechanism, not considering the tracking performance in the occlusion process, and repositioning the target through the global search strategy after leaving the occlusion. The global search strategy adopts the traditional sliding window method, and performs template matching operation on the current window every time sliding.

[0100] The present application is based on the SiamRPN method, so the already trained Siamese network model is directly used, and the whole tracking framework does not need additional offline training. The online training classifier also uses the Siamese network model, generates 30 initial training samples by data augmentation on the initial frame. The model updating part directly uses the existing update network model, and starts online updating the template from the second frame.

[0101] The online classifier updates the filter by discarding the frames with interference or target missing. The classifier is updated every 10 frames, and the learning rate is set to 0.01. When the interference is detected, the learning rate will be doubled. For classification fusion, when the value of lambda is 0.9, the tracking effect is optimal. The current tracking state is evaluated through the occlusion judgment mechanism to determine whether to update the template. The hyperparameters t h 、t l and tol are set to 0.93, 0.92 and 0.02 respectively, and the sliding window template matching threshold thl is set to 0.5.

[0102] In order to prove that the present application has good advancement and robustness, the success rate and precision are used to evaluate the tracking performance. As can be seen from Table 3, for the inland ship data set, the present application method achieves the optimal accuracy, which is 2% higher than the suboptimal tracking method in accuracy, and has competitive success rate. As Figure Two and Figure Three The precision graph and success rate graph corresponding to the present application method are shown in Figs. 1 and 2, which show the average tracking performance under each video attribute. Since the present application is tested on the ship data set with occlusion attribute, it can be seen that the method performs best under the occlusion condition.

[0103] Table 3: Comparison with the most advanced tracking method on the inland ship data set

[0104]

[0105] Compared with the most advanced tracking methods, only the method of the present application can continue to accurately track the target after the occlusion is removed in the case of large-area occlusion, while other tracking methods will lose tracking. In the case of small-area occlusion, only the method of the present application can continue to accurately track the target, while other methods will cause the target tracking to "drift" due to the interference of the occlusion and continue to track the occlusion. The method of the present application evaluates the tracking state through the occlusion judgment mechanism and does not consider the tracking performance in the occlusion process, and repositions the target by using a global search strategy after the occlusion is removed. Therefore, the method of the present application exhibits the best performance in the case of occlusion.

Claims

1. A robust online learning ship tracking method based on twin networks, characterized in that, The method comprises the following steps: (1) fixedly setting k different high-width ratio ratios and 1 fixed scale, and extracting k x 1 candidate target boxes of different sizes from an initial frame by using an anchor-based method; (2) features of a template z and a detection frame x are extracted respectively by using a Siamese network, the Siamese network architecture adopts the first five convolutional layers of AlexNet, and edge padding and full connection layers are discarded; an online training classifier also adopts the Siamese network architecture, and a plurality of initial training samples are generated by performing data enhancement on the initial frame; (3) the features of the template z and the features of the detection frame x obtained in step (2) are respectively sent into the classification branch and the regression branch of the region selection network RPN, and then the features of the template z are taken as a convolution kernel to perform cross-correlation operation on the features of the detection frame x, thereby generating a classification prediction value f cls and a position regression value f reg ; the training sample obtained in step (2) is respectively compressed, double-attention mechanism is performed, and filter operation is performed, thereby generating a classification confidence f C ; (4) an adaptive online updating network UpdateNet is introduced, which is used to improve the template degradation problem in the tracking process; In the case that an initial template, an accumulated template and a current frame template are given, an offline trained UpdateNet model is used to estimate an optimal accumulated template of a next frame; wherein the initial template and the current frame template are features extracted from the template z and the detection frame x in step (2); (5) The classification prediction value f obtained in step (3) is fused with the online classifier score f cls C The anchors extracted in step (1) are combined to find the predicted target position loc of the current frame, i.e. the upper left corner coordinates, width and height of the target; in addition, a shielding determination mechanism is established according to the classification fusion score; when it is determined that the target is in a shielding state, the target template is not updated, i.e. step (4) is not performed; at the same time, a global search mechanism is started to reposition the target, thereby avoiding target tracking drift.​ (6) steps (1)-(5) are performed for each frame image in an inland ship dataset to realize robust online learning ship tracking based on the Siamese network.

2. The robust online learning ship tracking method based on twin networks according to claim 1, characterized in that: In the step (1), k=5, ratios=[0.33, 0.5, 1, 2, 3], scale=[8]; in the step (2), a template with a size of 127*127*3 and a detection frame with a size of 271*271*3 are input into a twin network for feature extraction, that is, the same transformation is performed on two inputs z and x Feature maps with sizes of 6*6*512 and 24*24*512 are respectively generated and In the step (3), the feature maps and are respectively sent into a classification branch and a regression branch, and 3*3 convolution operations are performed to obtain wherein subscript "cls" represents the classification branch, and "reg" represents the regression branch.

3. The robust online learning ship tracking method based on twin networks according to claim 2, characterized in that: The step (3) comprises the following sub-steps: (3.1) For the classification branch, it is used to predict the target-background classification score for each candidate region; according to step (3), can be seen as a convolution kernel, in convolved to get the similarity score, thus producing a classification response map (3.2) for the regression branch, a position regression value of each candidate region is given; According to step (3), may be considered as a convolution kernel, in convolving to get the position regression value 4. The robust online learning ship tracking method based on twin networks according to claim 3, characterized in that: In the step (4), the UpdateNet model is used to update the template by using a learning function Φ, the function Φ is actually a convolutional neural network, and a formula of the function Φ is as follows: By giving an initial frame Ground-Truth (GT) template history frame accumulated template and a template T extracted from the current frame by predicting the target position i to calculate the updated template 5. The robust online learning ship tracking method based on twin networks according to claim 4, characterized in that: The step (5) comprises the following sub-steps: (5.1) the online classifier mainly comprises a compression module, an attention module and a filter module; first, the compression module is compressed by using a random projection method; then, a double attention mechanism is composed of a channel attention mechanism and a spatial attention mechanism; finally, the filter module is used to generate a classification confidence; therefore, the online classifier is defined as follows: Wherein, w represents an online learning network parameter, C represents the compression module, A represents the attention module, and F represents the filter module; (5.2) a target function of online learning in step (5.1) is defined in a form similar to an L2 classification error: Each training sample x i Subject to a weight γ i Controlled by the impact; y i ∈ R W*H Is the labeled classification confidence, which is annotated with a Gaussian function centered on the predicted target location; the regularization penalty on w k Is set by λ k ; (5.3) a linear weighted fusion method is adopted to fuse the classification scores of the offline Siamese network obtained in step (3.1) and the score map generated by the online classifier in step (5.1), and adaptive classification scores are obtained by fusing with a certain weight: f fusion (x; w) = λf C (x; w) + (1 - λ)f cls (z, x; w) Wherein, λ is an influence factor of the online confidence score; (5.4) an adaptive occlusion judgment mechanism is proposed according to the change degree of the adaptive classification scores obtained in step (5.3); the current tracking state is evaluated according to the adaptive classification scores, so as to determine whether the template should be updated and whether the search region should be expanded; (5.5) a global search strategy adopts a traditional sliding window method, and a template matching operation is performed on the current window by sliding; if a similarity score obtained by the current window is higher than a threshold thl, it is considered that the target position loc is detected.

6. The robust online learning ship tracking method based on twin networks according to claim 2, characterized in that: In the step (5.3), λ = 0.

9.

7. The robust online learning ship tracking method based on twin networks according to claim 2, characterized in that: In the step (5.4), t h = 0.93, t l = 0.92, tol = 0.

02.

8. The robust online learning ship tracking method based on twin networks according to claim 2, characterized in that: In the step (5.5), thl = 0.5.

Images