A method and system for tracking a drone target in a pasture based on a Siamese network

Abstract

The application discloses a kind of based on Siamese network's pasture unmanned aerial vehicle to cow target tracking method and system, it is related to target tracking technical field, to solve the problem that only using initial template in the existing target tracking method based on Siamese network causes poor robustness in tracking process.The technical points of the present application include: in tracking, for subsequent video frames: whether the current frame tracking result needs to update template according to anchor frame ratio change judgment;If template is to be updated, whether similar target and real target are approaching each other in the current frame is judged using response peak euclidean distance detection mechanism;If it is judged that there is not, high confidence template updating mechanism is used to judge the quality of the current frame;If it is judged that the template is of high quality, the corresponding features are integrated into the cumulative template, and the next frame tracking template is formed by connecting the residual error with the initial template.The present application greatly improves the robustness of unmanned aerial vehicle target tracking, and provides a better information collection tool for subsequent livestock behavior analysis.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, specifically to a method and system for tracking cattle targets using unmanned aerial vehicles (UAVs) in a pasture based on Siamese networks. Background Technology

[0002] In UAV target tracking methods, existing methods are dominated by correlation filtering algorithms. While offering high real-time performance, these methods suffer from insufficient accuracy and robustness to meet practical requirements. Correlation filtering algorithms utilize manually extracted features, such as HOG and CN features, which are not robust enough in representing object characteristics. Currently, these algorithms primarily employ image search strategies for tracking, and the size of the search region significantly impacts model performance. Excessively large search regions lead to the model learning much useless background information, and background interference can even degrade performance and cause tracking drift. Furthermore, expanding the search region increases algorithm complexity. In 2016, Bertinetto et al. proposed the SiamFC algorithm, which truly broke the monopoly of correlation filtering in tracking. It utilizes a fully convolutional Siamese neural network—the Siamese network—for end-to-end training of tracking data, resulting in a simple structure and strong real-time performance. This achieves a good balance between real-time performance and accuracy in target tracking algorithms.

[0003] However, the Siamese network algorithm has poor robustness during tracking because it only uses an initial template, and it is easy to lose track of the target or drift. In addition, the computational cost of the feature extraction backbone network used is too large for embedded terminals and is not suitable for deployment on drones. These factors have severely restricted the application of the algorithm in real-world scenarios. Summary of the Invention

[0004] To overcome the shortcomings and deficiencies of existing technologies in tracking cattle in pastures, this invention proposes a method and system for tracking cattle targets using unmanned aerial vehicles (UAVs) in pastures based on Siamese networks.

[0005] According to one aspect of the present invention, a method for tracking cattle targets by a drone in a pasture based on a Siamese network is provided. The method includes: for video captured in real time by a video acquisition device on a drone in a pasture, using a target detection algorithm to detect and obtain an initial frame containing the target; using an improved Siamese network to track the target cattle, wherein the improvement is that the feature extraction backbone network of the Siamese network is replaced by a MobileNet network; during tracking, for subsequent frames: judging the change of the anchor frame ratio according to multiple preset anchor frame ratios, and then judging whether the current frame tracking result needs to update the template; if the template needs to be updated, using the peak response Euclidean distance detection mechanism to judge whether there is a similar target approaching the real target in the current frame; if it is judged that no similar target is approaching, using a high-confidence template update mechanism to judge the quality of the current frame; if the current frame is judged to be a high-quality template, its corresponding features are integrated into the cumulative template and residually connected with the initial template to form the tracking template for the next frame.

[0006] Furthermore, after acquiring the initial frame containing the target, features are extracted using the initial template and the search image through the feature extraction backbone network. After unifying the dimension of the feature maps, they are input into the classification and regression branches of the RPN network respectively. The template feature map and the search feature map are cross-correlated, and multiple anchor box ratios are preset to obtain classification response maps and regression response maps with multiple tracking boxes.

[0007] Furthermore, the process of determining whether a similar target and a real target are approaching each other in the current frame using the response peak Euclidean distance detection mechanism includes: acquiring the classification response map of the current frame, obtaining the current binary response peak map, and calculating the peak distance closest to the tracking result response peak; when the peak distance is less than a preset threshold, it is determined that a similar target is approaching the real target; wherein, the peak distance closest to the tracking result response peak is calculated according to the following formula:

[0008] distance min =min((ww) n ) 2 *(hh n ) 2 )

[0009] In the formula, (w, h) represents the position of the peak response of the tracking result on the peak response map, (w n h n () represents the location of other similar targets on the peak response map; distance min It is the Euclidean distance between the peak value of the smallest similar target and the peak value of the tracking result.

[0010] Furthermore, the process of using a high-confidence template update mechanism to determine the quality of the current frame includes:

[0011] A template is considered high-quality when both the average peak correlation energy (APCE) and response value of the current frame tracking result box are greater than a certain percentage of its historical average. The formula for calculating the average peak correlation energy (APCE) is as follows:

[0012]

[0013] In the formula, F max F min F w，h represents the highest response value, the lowest response value, and the response value at the (w, h) position, respectively; mean represents the average value.

[0014] Furthermore, the cumulative template is represented as:

[0015] T i '=(1-r)T i-1 +rT i

[0016] Among them, T i ′ represents the cumulative template, which is a weighted fusion of the template from the previous frame and the template from the current frame; T i-1 T is the tracking template for the previous frame. i The template generated from the tracking results of the previous frame, where r represents the ratio.

[0017] According to another aspect of the present invention, a UAV-based cattle tracking system for a pasture based on a Siamese network is provided, the system comprising:

[0018] The target detection module is configured to use a target detection algorithm to detect and obtain initial frames containing targets from the video captured in real time by the video acquisition device on the drone in the ranch.

[0019] The target tracking module is configured to track the target cow using an improved Siamese network. The improvement lies in replacing the feature extraction backbone network of the Siamese network with a MobileNet network. During tracking, for subsequent frames: the module judges the change in anchor box ratio based on multiple preset anchor box ratios, and then determines whether the current frame tracking result needs to update the template. If the template needs to be updated, the module uses the peak response Euclidean distance detection mechanism to determine whether there is a similar target approaching the real target in the current frame. If no similar target is determined to be approaching, the module uses a high-confidence template update mechanism to perform a quality judgment on the current frame. If the current frame is judged to be a high-quality template, its corresponding features are integrated into the cumulative template and residually connected with the initial template to form the tracking template for the next frame.

[0020] Furthermore, in the target tracking module, after acquiring the initial frame containing the target, features are extracted using the initial template and the search image through the feature extraction backbone network. After unifying the dimension of the feature maps, they are input into the classification and regression branches of the RPN network respectively. The template feature map and the search feature map are cross-correlated, and multiple anchor box ratios are preset to obtain classification response maps and regression response maps with multiple tracking boxes.

[0021] Furthermore, the process in the target tracking module that uses the response peak Euclidean distance detection mechanism to determine whether a similar target and a real target are approaching each other in the current frame includes: acquiring the classification response map of the current frame, obtaining the current binary response peak map, and calculating the peak distance closest to the tracking result response peak; when the peak distance is less than a preset threshold, it is determined that a similar target is approaching the real target; wherein, the peak distance closest to the tracking result response peak is calculated according to the following formula:

[0022] distance min =min((ww) n ) 2 *(hh n ) 2 )

[0023] In the formula, (w, h) represents the position of the peak response of the tracking result on the peak response map, (w n h n () represents the location of other similar targets on the peak response map; distance min It is the Euclidean distance between the peak value of the smallest similar target and the peak value of the tracking result.

[0024] Furthermore, the process of judging the quality of the current frame using the high-confidence template update mechanism in the target tracking module includes:

[0025] A template is considered high-quality when both the average peak correlation energy (APCE) and response value of the current frame tracking result box are greater than a certain percentage of its historical average. The formula for calculating the average peak correlation energy (APCE) is as follows:

[0026]

[0027] In the formula, F max F min F w，h represents the highest response value, the lowest response value, and the response value at the (w, h) position, respectively; mean represents the average value.

[0028] Furthermore, the cumulative template in the target tracking module is represented as:

[0029] T i'=(1-r)T i-1 +rT i

[0030] Among them, T' i For cumulative templates; T i-1 T is the tracking template for the previous frame. i The template generated from the tracking results of the previous frame, where r represents the ratio.

[0031] The beneficial technical effects of this invention are:

[0032] This invention utilizes the ratio change of the tracking box as an update opportunity based on the Siamese network, and improves the robustness of the original tracking algorithm by using the cumulative template of residual connections and the interference peak detection module, making it suitable for UAV target tracking of cattle in pastures. This greatly improves the robustness of UAV tracking of cattle and provides a better information collection tool for subsequent livestock behavior analysis. Attached Figure Description

[0033] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, which together with the following detailed description are included in and form part of this specification, and are used to further illustrate preferred embodiments of the invention and explain the principles and advantages of the invention.

[0034] Figure 1 This is a schematic diagram of the Siamese network tracking algorithm framework.

[0035] Figure 2 This is a schematic diagram of the depth-separable convolution structure in this invention.

[0036] Figure 3 These are examples of tracking drift before updates; where (a) and (b) are the search image and corresponding response image of the 283rd frame of the actual cattle farm tracking; (c) and (d) are the search image and corresponding response image of the 297th frame of the actual cattle farm tracking.

[0037] Figure 4 This is an example of the overlapping and drifting phenomenon of similar individuals.

[0038] Figure 5 This is a visual schematic diagram of peak detection in this invention.

[0039] Figure 6 This is a flowchart of peak distance detection in this invention.

[0040] Figure 7 This is a schematic diagram of the overall model structure design of the present invention.

[0041] Figure 8 This is a schematic diagram of the residual connection accumulation template structure in this invention.

[0042] Figure 9 This is a flowchart of the high-confidence template update judgment module in this invention.

[0043] Figure 10 This is a comparison chart of tracking results on three challenging video sequences in the experiment of this invention. Detailed Implementation

[0044] To enable those skilled in the art to better understand the present invention, exemplary embodiments or examples of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments or examples are merely some, not all, of the embodiments or examples of the present invention. All other embodiments or examples obtained by those skilled in the art based on the embodiments or examples of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0045] This invention proposes a method for tracking cattle targets using a drone in a pasture based on a Siamese network. The method includes: using a target detection algorithm to detect and obtain an initial frame containing the target from the video captured in real time by a video acquisition device on a drone in the pasture.

[0046] Real-time target tracking is achieved using an improved Siamese network. The improvements are as follows: the feature extraction backbone network of the Siamese network is replaced with a MobileNet network; during tracking, for subsequent frames: the changes in anchor box ratios are judged based on multiple preset anchor box ratios, and then it is determined whether the current frame tracking result needs to be updated; if the template needs to be updated, the peak response Euclidean distance detection mechanism is used to determine whether there are similar targets in the current frame approaching the real target; if it is determined that no similar targets are approaching, a high-confidence template update mechanism is used to judge the quality of the current frame; if the current frame is judged to be a high-quality template, the features corresponding to the current frame are integrated into the cumulative template and residually connected with the initial template to form the tracking template for the next frame.

[0047] In this embodiment, preferably, after obtaining the initial frame containing the target, the initial template and the search image are used to extract features through the feature extraction backbone network. After unifying the dimension of the feature map, they are input into the classification branch and regression branch of the RPN network respectively. The template feature map and the search feature map are cross-correlated, and multiple anchor box ratios are preset to obtain the classification response map and regression response map with multiple tracking boxes.

[0048] In this embodiment, preferably, the process of determining whether a similar target and a real target are approaching each other in the current frame using the response peak Euclidean distance detection mechanism includes: acquiring the classification response map of the current frame, obtaining the current binary response peak map, and calculating the peak distance closest to the tracking result response peak; when the peak distance is less than a preset threshold, it is determined that a similar target is approaching the real target; wherein, the peak distance closest to the tracking result response peak is calculated according to the following formula:

[0049] distance min =min((ww) n ) 2 *(hh n ) 2 )

[0050] In the formula, (w, h) represents the position of the peak response of the tracking result on the peak response map, (w n h n () represents the location of other similar targets on the peak response map; distance min It is the Euclidean distance between the peak value of the smallest similar target and the peak value of the tracking result.

[0051] In this embodiment, preferably, the process of using a high-confidence template update mechanism to determine the quality of the current frame includes:

[0052] A template is considered high-quality when both the average peak correlation energy (APCE) and response value of the current frame tracking result box are greater than a certain percentage of its historical average. The formula for calculating the average peak correlation energy (APCE) is as follows:

[0053]

[0054] In the formula, F max F min F w，h represents the highest response value, the lowest response value, and the response value at the (w, h) position, respectively; mean represents the average value.

[0055] In this embodiment, preferably, the accumulation template is represented as:

[0056] T i '=(1-r)T i-1 +rT i

[0057] Among them, T i For cumulative templates; T i-1 T is the tracking template for the previous frame. i The template generated from the tracking results of the previous frame, where r represents the ratio.

[0058] The embodiments of the present invention will be described in detail below.

[0059] The Siamese network-based visual target tracking algorithm achieves a very good balance between accuracy and efficiency. Compared with correlation filtering-based tracking algorithms, the Siamese network-based target tracking algorithm describes visual target tracking as a problem of cross-correlation between template features and search features, which can effectively utilize the advantages of end-to-end learning in deep networks. The relevant network structure of the Siamese network tracking algorithm is as follows: Figure 1 As shown, the Siamese network mainly consists of two network branches with shared weights: a template branch for extracting features from the template image and a search branch for extracting features from the search image. The template is typically obtained from the bounding box of the target to be tracked in the first frame of the video. In practice, this initial bounding box is usually provided by the object detection algorithm, denoted as z. Each subsequent frame obtains the search region based on the tracking results of the previous frame, denoted as x. The Siamese network feeds the initial template and the search template into the template network branch and the search network branch, respectively, through an offline-trained backbone network with shared weights. Feature extraction is performed, with the backbone network having weight parameters θ. Then, the template branch output feature map is used to perform a cross-correlation operation on the search feature map to obtain the tracking response map of the current frame. The values ​​on the response map represent the score of the current target at that position, and its calculation formula is as follows:

[0060]

[0061] This means that a template feature map is used to perform a recurrent convolution operation on the search feature map to obtain a confidence score at each position.

[0062] Training tracking algorithm networks typically involves offline training using a large number of template and search image pairs (z,x) and their corresponding ground truth labels y, collected from numerous training videos. However, the SiamFC algorithm cannot accurately estimate the target scale and can only determine it through multi-scale testing, which significantly increases computational cost and compromises real-time performance. Furthermore, since obtaining the target position after related operations requires translation invariance, only shallow fully convolutional networks without padding layers can be used.

[0063] Previously, Siamese networks used fully convolutional AlexNet networks with padding layers removed as the feature extraction backbone, failing to utilize the advantages of deep neural networks. However, subsequent improvements using networks such as ResNet have too many parameters, making the algorithms unsuitable for deployment on drones. A comparison of commonly used neural networks is shown in Table 1 below.

[0064] Table 1 Comparison of Commonly Used Neural Networks

[0065]

[0066]

[0067] Therefore, this invention replaces the backbone network used for feature extraction with MobileNetV2, which is very friendly to mobile devices. MobileNetV2 is a lightweight convolutional neural network designed mainly for mobile platforms such as embedded devices, and has the characteristics of few parameters and small performance loss.

[0068] like Figure 2 As shown, it mainly reduces computation by improving ordinary convolution into a depthwise separable convolution structure composed of depthwise convolution and pointwise convolution. If the input feature map shape is D F ×D F If the output feature map has N channels (×M), then the computational cost of standard convolution is:

[0069] D K ×D K ×M×N×D F ×D F (2)

[0070] The computational cost of depthwise separable convolution is:

[0071] D K ×D K ×M×D F ×D F +M×N×D F ×D F (3)

[0072] The ratio of the computational load of the two is:

[0073]

[0074] MobileNet uses 3×3 kernels, so the computational cost of depthwise separable convolutions is only 1 / 9 that of standard convolutions. This structure allows MobileNet to significantly reduce computational cost while maintaining model performance. Furthermore, it embeds the inverse residual module in MobileNetV2 using a lightweight and efficient ECA attention module. This module replaces the previously fully connected layers with a local cross-channel interaction strategy that does not reduce dimensionality, resulting in lower computational cost compared to other attention modules such as SE attention, while achieving performance comparable to SE attention. This improves the network's sensitivity to important target features without significant tracking latency.

[0075] This invention uses the SiamRPN++ algorithm as a baseline algorithm, analyzes the tracking failures of this algorithm in actual tracking scenarios, and combines the actual operation mechanism of the algorithm to find that the reason for insufficient robustness is that the algorithm only uses the initial template of the first frame as the template for the entire tracking process. However, when tracking the target, the features of the target may change with scale or shape, causing the initial template to fail to match the subsequent target features well. Other existing technologies have also proposed corresponding solutions to this problem, namely updating the template. However, fixing the update time and simple updates cannot effectively improve the performance of the algorithm.

[0076] This invention starts with the actual tracking phenomena of baseline algorithms in tracking scenarios. Through the analysis of drift phenomena in three different situations, it analyzes the reasons for frequent tracking drift in the Siamese network algorithm and provides corresponding solutions:

[0077] 1) Because the essence of the Siamese network tracking algorithm is to determine the final tracking target by comparing the similarity between the search image and the template image feature z, in the subsequent tracking process, if the target features in the search image frame x change, it will cause the response value f of the real target part to change. θ As the template decreases, the actual target features will differ more significantly from the initial template features. For example... Figure 3 As shown, at this point, as long as the similarity between the surrounding objects and the template features is higher than the changed response value f, θ Higher values ​​will cause the tracking box to drift onto other objects, resulting in tracking drift.

[0078] This invention analyzes multiple video clips of cattle tracked by drones. The cattle's posture is relatively uniform and their movements are slow, without dramatic changes in shape. When the cattle turn or perform other actions, their features change accordingly, and the ratio of the tracking frame encompassing the cattle also changes. Therefore, the change in the anchor frame ratio can be used as an opportunity to update the template. This not only avoids the real-time loss caused by frequent updates but also allows for timely acquisition of the current template features of the real target after minor changes in characteristics. This enables the template to better represent the changing features of the tracked target during the tracking process.

[0079] 2) After implementing a simple update of the cow's feature template z based on scale, it was found that when the template was updated due to changes in the tracking box ratio, a lot of noise information around the target was added. As the tracking time increased, the template even completely lost most of the real features of the target, ultimately leading to tracking failure. Therefore, it is necessary to design a template update strategy that can retain the real features of the tracked target without introducing excessive noise features into the template.

[0080] This invention designs a cumulative template update strategy that combines APCE peak correlation energy and response value. The response value is used to represent the similarity between the corresponding part in the search image and the template image. However, in some cases, such as complex background interference, the response value may be high even when the final tracking result is not the tracked target. Therefore, this invention introduces both APCE peak correlation energy and response value to determine the quality of the current template.

[0081] The formula for calculating the peak-related energy of APCE is as follows:

[0082]

[0083] In the formula F max F min F w,h These represent the responses at the highest, lowest, and (w,h) positions, respectively. This criterion reflects the degree of oscillation in the response map. When the APCE value decreases, it indicates the presence of many interfering targets in the search image. In this case, it is not convenient to use the tracking result of this frame as a template to update the cumulative template. This ensures that the template retains the most essential and realistic features of the target during tracking, and also provides a high-quality updated template when features change during tracking, reducing the risk of introducing contaminated templates.

[0084] 3) When there are only a very few similar individuals in the search image during tracking, and the real target partially overlaps with similar targets during tracking, the response value and APCE value will be high. This makes the high-confidence cumulative template update strategy improved for the second scenario lack robustness in this situation. For example... Figure 4 As shown, at this time, the response peaks of the two targets gradually converge on the response map corresponding to the search map, and the conditions for updating the template are also met at the same time, which leads to erroneous updates. The features of similar targets are updated into the template, which increases the probability that the tracking box will drift to another similar object.

[0085] To address this phenomenon, this invention designs a response peak Euclidean distance detection mechanism to determine whether there are similar targets approaching the real target in the current frame. This provides reasonable judgment information for subsequent update judgments.

[0086] To address the issue of erroneous template updates caused by occlusion of similar targets in the search image, leading to tracking drift onto similar targets, a peak Euclidean distance detection module is proposed to solve this problem. The peak detection function formula is:

[0087]

[0088] Peak detection visualization, such as Figure 5As shown. The minimum distance between the coordinates of other response peaks and the coordinates of the response value of the tracking result is obtained by calculating the distance between them. The formula is as follows:

[0089] distance min =min((ww) n ) 2 *(hh n ) 2 (7)

[0090] Where (w,h) is the position of the peak value of the tracking result on the response graph, (w n ,h n () represents the coordinates of other peak values ​​where the response value of other non-tracked results is greater than a certain proportion of the response value, distance. min It is the Euclidean distance between the minimum interference peak and the result peak, when distance min When the distance is less than a threshold, the system sets a status variable to indicate that the frame template is in a contaminated state, and the contaminated state is canceled only when a distance greater than the threshold is detected. The process of this module is as follows: Figure 6 As shown.

[0091] In summary, this invention uses SiamRPN++ as the baseline algorithm. To address the aforementioned tracking drift issues and improve the robustness of Siam network-based tracking algorithms, a template update mechanism based on peak distance detection and high-confidence residual accumulation templates is proposed. This mechanism uses the first frame's bounding box as the most essential feature of the tracked target, and subsequent target bounding box images obtained during the tracking process as backup templates for updates. Similar to existing Siam network tracking algorithms, the initial template remains fixed throughout the tracking process after the first frame. Then, during the tracking process, subsequent high-quality tracking results are used as update templates and fused with the initial template. This fully utilizes both the initial features and the features resulting from changes in the target, and improves robustness against similar interfering targets, thereby enhancing the algorithm's overall robustness.

[0092] like Figure 7As shown, the Siam-RAT algorithm framework proposed in this invention mainly consists of the SiamRPN++ network architecture plus the three improved modules proposed in this invention. In the initial frame, the initial template and the search image are used to extract features through the MobileNet feature extraction backbone network, and then the feature map dimensions are unified through the intermediate neck layer. Then, they are input into the RPN network and anchor boxes of 5 preset scales are used for foreground and background classification and regression operations in 4 directions of the tracking box to finally obtain the target tracking box. First, the tracking result of this frame is determined based on the change of the anchor box scale ratio to determine whether the template needs to be updated. Then, it is input into the peak distance detection module to determine whether it is in a "contaminated" state. Next, the high confidence template update judgment module performs template quality judgment. The template that meets the quality standard is input into the residual accumulation template to generate a high-quality tracking template for the next frame, thus completing one round of tracking process.

[0093] Previous tracking algorithms based on Siamese networks either directly tracked using only the initial frame template or updated the tracking template using templates extracted from subsequent frames. The former method, relying solely on the initial template, may fail to adapt well to changing target features during subsequent tracking, while the latter is prone to template contamination due to target occlusion or other factors, potentially leading to complete tracking failure. Therefore, this invention combines these two approaches by proposing a residual accumulation template. This template uses initial template features as residual connections, ensuring the most realistic features are retained throughout the tracking process, preventing complete template contamination. Furthermore, high-confidence features from subsequent frames are incorporated into the template to provide robustness to continuously changing features.

[0094] The quality of the template determines the tracking performance of the Siamese network-based tracking algorithm. A good template can produce stable results, while an unsuitable template will seriously reduce the algorithm's performance. Figure 8 The overall architecture of the residual accumulation template is shown. The high-confidence template update judgment process is as follows: Figure 9 As shown. Based on the conclusions drawn from the first phenomenon, the template is updated by taking the ratio change of the tracking box during the cow's movement as the trigger. The template update process is only initiated when the state is not contaminated. Then, the quality of the template at this moment is judged. If the APCE average peak correlation energy and response value of the tracking result box of the frame are greater than a certain proportion of the historical average, it is judged as a high-quality template. Finally, it is integrated into the cumulative template and residually connected with the initial template to form a new template.

[0095] The subsequent frame template incorporated into the cumulative template is a new template z′ clipped from the region covered by the highest confidence bounding box, which is represented as:

[0096] z′=crop(argmaxf θ (z,x)) (8)

[0097] In the formula, crop(·) is a cropping operation that crops a new template z′ from the tracking results of the current frame, f θ (z,x) is the response score for the classification branch. The cumulative template calculation formula is as follows:

[0098] T i ′=(1-r)T i-1 +rT i (9)

[0099] Where T i ' is the final tracking template, T i-1 T is the tracking template for the previous frame. i The template generated from the tracking results of the previous frame.

[0100] It should be noted that the improved target tracking algorithm proposed in this invention is not only applicable to real-time tracking of cattle in pastures, but also applicable to real-time tracking of other animals in pastures or other scenarios.

[0101] The technical effects of the present invention were further verified through experiments.

[0102] The experimental process includes four parts: model training settings, experimental evaluation indicators, comparative experiments, and ablation experiments, in order to demonstrate the effectiveness of the method of the present invention in the application scenario of tracking cattle in pastures.

[0103] The experiments were conducted offline on the GOT10K dataset. The GOT10K dataset contains over 10,000 real-world moving object video clips and over 1.5 million manually labeled bounding boxes, covering more than 560 categories. The validation and test sets each contain 180 video sequences. The template image size is 127×127×3, and the search image size is 255×255×3, both being RGB three-channel images. MobileNet was pre-trained on a large amount of data. During network training, the initial learning rate was set to 0.01, and the learning rate was increased from 10... -2 Decays exponentially to 10 -5 The batch size was 8, training lasted 50 epochs, and the SGD optimizer was used with a momentum value of 0.9 and an L2 penalty term (weight_decay) of 5E-4. The search images were scaled at five different scales ([0.33, 0.5, 1, 2, 3]). The hyperparameter a was initially set to 0.0001.

[0104] The main evaluation metrics for single-target tracking algorithms include: Accuracy, Robustness, Expected Average Overlap, and FPS (frame rate).

[0105] Accuracy is used to evaluate how accurately a tracker tracks a target; the higher the value, the higher the accuracy. The accuracy of frame t in a sequence is defined as the Interchange of Units (IOU) value for each frame, as shown in the following formula:

[0106]

[0107] The average precision is the average of all valid frames, calculated using the following formula:

[0108]

[0109] Robustness is used to evaluate the stability of the tracker in tracking the target; the higher the value, the worse the stability. F represents the number of failures in repeated tests N_rep, as shown in the formula below:

[0110]

[0111] Expected Average Overlap (EAO): Proposed at VOT2015, this new evaluation metric, EAO (Expected Average Overlap), addresses the shortcomings of ARrank-based evaluation methods in fully utilizing the raw data for accuracy and robustness. As the name suggests, this metric only considers accuracy defined based on overlap.

[0112] FPS: This refers to the number of images processed per second by the algorithm deployed on the device. The formula is as follows:

[0113]

[0114] To verify the performance of the Siam-RAT tracking method, it was compared with current mainstream and advanced tracking algorithms. Several latest tracking algorithms based on Siamese networks were selected for comparative experiments with the method of this invention.

[0115] The comparative experiment selected a test dataset made from videos of cattle collected by drones in pastures and videos with similar conditions from the VOT2018 dataset to compare several different tracking algorithms with the method of this invention. The VOT2018 dataset contains more than 60 test videos. It is an authoritative performance evaluation dataset in the field of target tracking, including various conditions such as rotation, deformation, and occlusion. The evaluation of this public dataset mainly consists of three important metrics: Accuracy (A), Robustness (R), and Expected Average Overlap (EAO). In the experiment, the method of this invention was compared with SiamRPN++. [1] (Li et al., 2019), SiamDW [2] (Zhang and Peng, 2019), DaSiamRPN [3] (Zhu et al., 2018), SiamRPN [4] (Li et al., 2018), ECO-HC [5] (Wang et al.), SiamFC [6] A comparison was made between six state-of-the-art tracking algorithms, including those developed by Bertinetto et al. Table 2 shows the specific performance metrics of different methods on a self-made test dataset for tracking cattle in a pasture setting.

[0116] Table 2 Performance comparison of different methods on a self-made test dataset

[0117]

[0118]

[0119] Note: Bold text indicates the best result for each column.

[0120] Table 3 shows the comparison results of specific performance metrics of different methods on the VOT2018 test dataset containing videos with the same attributes as the application scenario of this invention.

[0121] Table 3 shows the performance comparison of different methods on the VOT2018 dataset.

[0122]

[0123] Note: Bold text indicates the best result for each column.

[0124] As can be seen from Tables 2 and 3, compared with SiamRPN++, SiamDW, etc., the method of this invention, Siam-RAT, shows very competitive performance, achieves the best results in robustness, and has a significant improvement in real-time performance compared with SiamRPN++. Although the accuracy is slightly reduced, it still ranks second among all trackers, making it more suitable for the actual tracking needs in ranches.

[0125] Figure 10 The qualitative comparison results of the proposed method with tracking algorithms such as SiamRPN, SiamRPN++, SiamDW, and Siam-RAT in the video sequences of the application scenario of this invention and three challenging video sequences with similar attributes in VOT are further presented. In the first video, there are significant feature changes between the initial template and subsequent images. In the second and third rows of video frames, there is mutual interference between similar individuals. The proposed method did not experience drift, while other algorithms drifted to similar targets. This demonstrates that the proposed Siam-RAT method still maintains good tracking performance even with significant appearance changes, similar interference objects, and partial occlusion. This proves that the tracking algorithm proposed in this invention can significantly improve the robustness of Siamese network tracking algorithms.

[0126] The effectiveness of the improved method of the present invention can be verified by using a self-made test dataset for ablation experiments. Under the premise of using MobileNet as the backbone network, the impact of different improved modules on the experimental results is explored. The experimental results are shown in Table 4.

[0127] Table 4 Comparison of ablation experiments

[0128]

[0129]

[0130] As can be seen from the comparison table of ablation experiments, the frame rate was greatly improved and the robustness was also significantly enhanced after the lightweight network improvement and the addition of the residual accumulation template. However, the accuracy decreased slightly. After performing high-confidence updates on the residual accumulation template and multi-peak Euclidean distance judgment, the robustness of the tracking algorithm was greatly improved, and the drift phenomenon during tracking was greatly reduced, making the method of this invention more suitable for the application scenario of this invention.

[0131] Another embodiment of the present invention proposes a UAV-based cattle target tracking system for a pasture based on a Siamese network, the system comprising:

[0132] The target detection module is configured to use a target detection algorithm to detect and obtain initial frames containing targets from the video captured in real time by the video acquisition device on the drone in the ranch.

[0133] The target tracking module is configured to achieve real-time target tracking using an improved Siamese network. The improvement lies in replacing the feature extraction backbone network of the Siamese network with a MobileNet network. During tracking, for subsequent frames: the changes in anchor box ratios are judged based on multiple preset anchor box ratios, thereby determining whether the current frame tracking result needs to update the template; if the template needs to be updated, the peak response Euclidean distance detection mechanism is used to determine whether there are similar targets in the current frame approaching the real target; if it is determined that no similar targets are approaching, a high-confidence template update mechanism is used to perform quality judgment on the current frame; if the current frame is judged to be a high-quality template, the features corresponding to the current frame are integrated into the accumulated template and residually connected with the initial template to form the tracking template for the next frame.

[0134] In this embodiment, preferably, after acquiring the initial frame containing the target, the target tracking module uses the initial template and the search image to extract features through the feature extraction backbone network. After unifying the dimension of the feature map, it inputs them into the classification branch and regression branch of the RPN network respectively. The template feature map and the search feature map are cross-correlated, and multiple anchor box ratios are preset to obtain classification response maps and regression response maps with multiple tracking boxes.

[0135] In this embodiment, preferably, the process of determining whether a similar target and a real target are approaching each other in the current frame using the response peak Euclidean distance detection mechanism in the target tracking module includes: acquiring the classification response map of the current frame, obtaining the current binary response peak map, and calculating the peak distance closest to the tracking result response peak; when the peak distance is less than a preset threshold, it is determined that a similar target is approaching the real target; wherein, the peak distance closest to the tracking result response peak is calculated according to the following formula:

[0136] distance min =min((ww) n ) 2 *(hh n ) 2 )

[0137] In the formula, (w, h) represents the position of the peak response of the tracking result on the peak response map, (w n h n () represents the location of other similar targets on the peak response map; distance min It is the Euclidean distance between the peak value of the smallest similar target and the peak value of the tracking result.

[0138] In this embodiment, preferably, the process of judging the quality of the current frame using the high-confidence template update mechanism in the target tracking module includes:

[0139] A template is considered high-quality when both the average peak correlation energy (APCE) and response value of the current frame tracking result box are greater than a certain percentage of its historical average. The formula for calculating the average peak correlation energy (APCE) is as follows:

[0140]

[0141] In the formula, F max F min F w，h represents the highest response value, the lowest response value, and the response value at the (w, h) position, respectively; mean represents the average value.

[0142] In this embodiment, preferably, the cumulative template in the target tracking module is represented as:

[0143] T i '=(1-r)T i-1 +rT i

[0144] In the formula, T i " is a cumulative template; T" i-1 T is the tracking template for the previous frame. i The template generated from the tracking results of the previous frame, where r represents the ratio.

[0145] The function of the cattle target tracking system for a farm based on a Siamese network described in this embodiment can be explained by the aforementioned method for cattle target tracking in a farm based on a Siamese network. Therefore, for the parts not described in detail in this embodiment, please refer to the above method embodiments, and they will not be repeated here.

[0146] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. The disclosure of the invention is illustrative and not restrictive, and the scope of the invention is defined by the appended claims.

[0147] The references cited in this invention are as follows:

[0148] [1]LI B,WU W,WANG Q,et al.Siamrpn++:Evolution of siamese visualtracking with very deep networks[C].USA:IEEE,2019:4282-4291.

[0149] [2]ZHANG Z,PENG H.Deeper and wider siamese networks for real-timevisual tracking[C].USA:IEEE,2019:4591-4600.

[0150] [3]ZHU Z,WANG Q,LI B,et al.Distractor-aware siamese networks forvisual object tracking[C].German:Springer,2018:101-117.

[0151] [4]LI B,YAN J,WU W,et al.High performance visual tracking withSiamese region proposal network[C].USA:

[0152] IEEE,2018:8971-8980.

[0153] [5]WANG Y,HUANG H,HUANG X,et al.ECO-HC based tracking for groundmoving target using single UAV[C].China:IEEE,2020:6414-6419.

[0154] [6]BERTINETTO L,VALMADRE J,HENRIQUES J F,et al.Fully-convolutionalsiamese networks for object tracking[C].German:Springer,2016:850-865。

Claims

1. A method for tracking a drone to a cattle target in a pasture based on a Siamese network, characterized in that, The method comprises the following steps: For the video collected in real time by the video acquisition device on the unmanned aerial vehicle in the pasture, an initial frame containing a target is obtained by using a target detection algorithm; Real-time tracking of the target is realized by using an improved twin network, and the improvement is that the feature extraction backbone network of the twin network is replaced by a MobileNet network; when tracking the subsequent frame: according to a preset plurality of anchor box ratios, the anchor box ratio change is judged, and then it is judged whether the tracking result of the current frame needs to update the template; if the template needs to be updated, it is judged whether similar targets and the real target are approaching each other in the current frame by using a response peak Euclidean distance detection mechanism; if it is judged that no similar target is approaching, a high-confidence template updating mechanism is used to judge the quality of the current frame; if the current frame is judged to be a high-quality template, the features corresponding to the current frame are integrated into the cumulative template, and the residual connection is formed with the initial template to form the tracking template of the next frame; The process of judging whether similar targets and the real target are approaching each other in the current frame by using the response peak Euclidean distance detection mechanism comprises: obtaining the classification response map of the current frame, and obtaining the current binary response peak map, calculating the peak distance closest to the tracking result response peak; when the peak distance is less than a preset threshold, it is judged that similar targets are approaching the real target; wherein the peak distance closest to the tracking result response peak is calculated according to the following formula: wherein is the position of the response peak of the tracking result on the response peak map, is the position of other similar targets on the response peak map; is the smallest peak value of the similar target and the peak value of the tracking result. The cumulative template is represented as: wherein, is a cumulative template; is a previous frame tracking template, is a template generated according to a previous frame tracking result, and r represents a ratio.

2. The method of claim 1, wherein the method is based on a Siamese network. After obtaining the initial frame containing the target, the initial template and the search image are used to extract features by the feature extraction backbone network, and after the feature map dimensions are unified, they are respectively input into the classification branch and the regression branch of the RPN network, the cross-correlation operation is performed between the template feature map and the search feature map, and a plurality of anchor box ratios are preset, the classification response map and the regression response map with a plurality of tracking boxes are obtained. 3.The method of claim 1, wherein: The process of using the high-confidence template updating mechanism to judge the quality of the current frame comprises: When the APCE average peak correlation energy and the response value of the current frame tracking result box are greater than a certain proportion of the historical average value, it is determined that the template is of high quality, wherein the calculation formula of the APCE average peak correlation energy is as follows: wherein, , , respectively represent the response value highest, the response value lowest and the response value in the position; mean represents the average value.

4. A system for tracking a drone target cow in a pasture based on a Siamese network, characterized in that, The system comprises: A target detection module configured to, for the video collected in real time by the video acquisition device on the unmanned aerial vehicle in the pasture, obtain an initial frame containing a target by using a target detection algorithm; The target tracking module is configured to realize real-time tracking of the target by using an improved twin network, and the improvement is that a feature extraction backbone network of the twin network is replaced by a MobileNet network; when tracking a subsequent frame: according to a preset plurality of anchor box ratios, the anchor box ratio change is judged, and then it is judged whether the tracking result of the current frame needs to update a template; if the template needs to be updated, a response peak Euclidean distance detection mechanism is used to judge whether similar targets and the real target are approaching each other in the current frame; if it is judged that no similar target is approaching, a high-confidence template updating mechanism is used to judge the quality of the current frame; if the current frame is judged to be a high-quality template, the features corresponding to the current frame are integrated into the cumulative template, and the residual connection is formed with the initial template to form the next frame tracking template; wherein the process of using the response peak Euclidean distance detection mechanism to judge whether similar targets and the real target are approaching each other in the current frame includes: obtaining a classification response map of the current frame, and obtaining a current binary response peak map, calculating the peak distance closest to the tracking result response peak; when the peak distance is less than a preset threshold, it is judged that similar targets are approaching the real target; wherein the peak distance closest to the tracking result response peak is calculated according to the following formula: wherein is the position of the response peak of the tracking result on the response peak map, is the position of other similar targets on the response peak map; is the smallest peak value of the similar target and the peak value of the tracking result. The cumulative template is represented as: wherein is the accumulated template; is the previous frame tracking template, is the template generated from the previous frame tracking result, r denotes the ratio.

5. A cattle target tracking system for a pasture based on a Siamese network according to claim 4, characterized in that, After obtaining the initial frame containing the target in the target tracking module, the initial template and the search image are used to extract features by the feature extraction backbone network, and after the feature map dimension is unified, the classification branch and the regression branch of the RPN network are input respectively, the cross-correlation operation is performed between the template feature map and the search feature map, and a plurality of anchor box ratios are preset, the classification response map and the regression response map with a plurality of tracking boxes are obtained.

6. A cattle target tracking system for a pasture based on a Siamese network according to claim 4, characterized in that, The process of using the high-confidence template updating mechanism to judge the quality of the current frame in the target tracking module includes: When the APCE average peak correlation energy and the response value of the current frame tracking result box are greater than a certain proportion of the historical average value, it is determined that the template is of high quality, wherein the calculation formula of the APCE average peak correlation energy is as follows: wherein, , , respectively represent the response value in the highest, the response value in the lowest and the response value in the position; mean represents the average value.

Images