A multi-target endangered animal tracking method based on small sample learning
By adopting a small sample learning method in multi-objective tracking technology, combining feature reweighting and Kalman filtering algorithms, the tracking efficiency problem in the absence of sufficient training data is solved, and efficient and accurate tracking in sparse scenarios is achieved.
Patent Information
- Application Number
- CN202111479215.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-06
- Publication Date
- 2025-04-22
- Estimated Expiration
- 2041-12-06
AI Technical Summary
Existing multi-objective tracking technologies are difficult to achieve efficient tracking results without sufficient training data, especially in sparse scenarios such as endangered animals.
A multi-objective tracking method based on small sample learning is adopted, and a feature reweighted small sample object detector and Kalman filtering algorithm are combined with self-supervised learning and manifold mixing algorithm to train a feature extraction network that can effectively track the target under a small amount of data.
It realizes efficient and accurate multi-target tracking without a large number of training samples, especially in sparse scenarios such as endangered animals, improving tracking efficiency and accuracy.
Smart Images
Figure CN114283355B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer vision, and in particular relates to a multi-target endangered animal tracking method based on small sample learning. Background Art
[0002] The main task of multiple object tracking (MOT) is to locate multiple objects of interest in a given video at the same time and maintain the unique ID of each object. In real life, multi-object tracking is closely related to video surveillance, autonomous driving and other fields. These practical needs have aroused great interest in MOT. Zhongdao Wang et al. proposed a multi-object tracking paradigm named JDE (Jointly learns the Detedctor and Embedding model). The JDE paradigm embeds the target detection stage and data association stage in multi-object tracking into a network, which enables the network to learn end-to-end, thereby speeding up the detection speed. Although this method can achieve SOTA at present, in the presence of class interference, the appearance features cannot obtain discriminative and accurate matching confidence. It also named the corresponding learning paradigm as SDE paradigm, that is, detection and data association are two independent models. This structure can have higher performance but relatively poor real-time performance. Based on the above two paradigms, there are rich research in the field of multi-object tracking. SiamMOT proposed by Bing Shuai et al. adds a region-based twin tracker to the standard object detection algorithm Faster R-CNN to model instance-level motion. This is similar to the SORT algorithm based on Kalman filtering and Hungarian algorithm proposed by Alex Bewley et al., which builds a motion model to associate previous and next frames. CorrTracker proposed by Qiang Wang et al. uses CenterNet as a detector, combines the features of time and space for data association, and proposes a local structure perception network to improve the identifiability of similar objects using self-supervised learning. This includes both motion models and appearance models. By learning the appearance features of each target, the occlusion problem in the field of multi-target tracking can be better handled. This is similar to DeepSort, an improvement on the SORT algorithm by Wojke et al., which comprehensively considers the motion trend and appearance features of the target in the video frame. However, there is a common point in all these studies, that is, both the detector module and the feature learning module require a large amount of data for training. This is also the place where deep learning is currently limited. The current multi-target tracking technology research objects are non-rigid targets, the most typical of which is multi-target tracking of pedestrians. This research trend is due to the fact that there are a lot of videos of pedestrians in real life and they are easy to collect, so they can be used for neural network learning, and secondly, they have extremely high commercial value. However, current research has not taken into account the serious lack of training data in some application scenarios. When the network cannot obtain enough data, it is difficult for the multi-target tracking network to reach the current level. Summary of the invention
[0003] In order to solve the problems existing in the above prior art, the present invention proposes a multi-target endangered animal tracking method based on small sample learning. For endangered animals in sparse scenes, a multi-target tracking method based on small sample learning is used to track multiple targets, including obtaining video data to be detected, inputting the video data into a multi-target tracking model based on a small sample, and obtaining a multi-target tracking result;
[0004] The process of using a multi-target tracking model based on small sample learning to track multiple targets in the video data to be detected includes:
[0005] S1: Fully sample the video to be detected to obtain a video frame sequence, and preprocess each frame;
[0006] S2: Use a small sample target detector based on feature reweighting to detect targets in the preprocessed video frames to obtain the location information and category information of the targets in the video frames;
[0007] S3: Using the Kalman filter algorithm to process the position information of each target, obtain the position information of the target at the next moment, and obtain the motion characteristics of the target according to the position information of the target at the next moment;
[0008] S4: cutting out the target from the video frame according to the position information of the target in the video frame; extracting features of the cut-out target using a feature extraction network trained based on a small sample learning method to obtain more distinguishing appearance features of the target;
[0009] S5: Perform data association on the motion features and appearance features of the targets in the previous and next frames to achieve multi-target tracking.
[0010] Preferably, the video is fully sampled to obtain a video frame sequence, each frame of the image is filtered through a Wiener filter to eliminate a series of motion blurs generated by the rapid movement of endangered animals, and each frame of the image after filtering is cropped to obtain a 448*448 size video frame after eliminating motion blur.
[0011] Preferably, the process of using a small sample target detector based on feature reweighting to detect the preprocessed video frame includes: the small sample target detector based on feature reweighting includes a meta-feature extraction module, a reweighting module and a one-stage detection network; using the meta-feature extraction module to extract features from image i to obtain meta-features F=D(i); using the reweighting module to obtain a global representation ω of each target in the support set i =M(li,M i), in the process of obtaining the global representation of the target, the mask channel is used to mark the target position in the image, so that each image in the support set becomes a four-channel (R, G, B, Mask); the meta-feature and the global representation are multiplied in the channel domain to obtain the re-weighted meta-feature A one-stage detection network is used to regress the position of the target in the video frame and extract the category information of the reweighted meta-features to obtain the position information and category information of the target in the video frame. i represents the global representation of the i-th target; M represents the reweighted module; li represents the support set data; M i Indicates the four-channel data corresponding to each image in the support set; RGB indicates the full-color image mode of the image, with three channels R\G\B, R represents red, G represents green, and B represents blue; F represents meta-feature; N represents the total number of global representations of the target.
[0012] Furthermore, the expressions of the position information and category information of the target in the video to be detected are:
[0013] {o i ,x i ,y i ,h i ,ω i ,c i =P (F),i＝1,…,N
[0014] Among them, i Indicates the probability of the existence of an object in the detection box, x i Indicates the horizontal coordinate of the upper left corner of the detection box, y i Indicates the vertical coordinate of the upper left corner of the detection box, h i Indicates the height of the detection box, w i Indicates the height of the detection box, c i Indicates the category label of the target in the detection box.
[0015] Preferably, the process of using the Kalman filter algorithm to process the position information of each target includes: after passing through the small sample target detector, the target position information and category information are obtained as the target detection frame, and the Kalman filter is used to predict the position of each detection frame in the subsequent frame; the Mahalanobis distance is used to measure the distance between the detection frame and the prediction frame, and the Mahalanobis distance is thresholded; the threshold t is set (1) , will be less than the specified threshold t (1) The Mahalanobis distance is set to successful association.
[0016] Furthermore, the formula for obtaining the target's motion characteristics is:
[0017]
[0018]
[0019] Among them, d j represents the i-th newly obtained detection box, y i represents the i-th track, is the covariance of d and y, Indicates d j With y i The Mahalanobis distance between represents a vector of all 1s, t (1) represents the threshold value, Indicates the part where the Mahalanobis distance is less than the threshold.
[0020] Preferably, the process of training the feature extraction network based on a small sample learning method includes:
[0021] Step 1: Use the self-supervised learning method to add a pre-task Rotation to the network, so that each image in the training set is flipped 90°, 180°, and 270° respectively, to obtain images with four flipping angles, namely 0°, 90°, 180°, and 270°, and create corresponding angle labels;
[0022] Step 2: Use elastic deformation to process the data for creating angle labels to simulate the complexity of the apparent features of endangered animals during movement. For example, the apparent features of birds are very different when they are walking on the ground and flapping their wings. Elastic deformation can increase the robustness of the feature extraction network;
[0023] Step 3: The processed image is passed through the feature extraction network to extract features and obtain the feature information of the image. In the process of feature extraction, when the image passes through the k-th layer (k is the number of middle layers of the feature extraction network) of the neural network, the manifold mixing algorithm is used to fuse the middle hidden layer vectors of the image and create a new category label.
[0024] Step 4: Predict the flip angle and image category of the image based on the features output by the feature extraction network;
[0025] Step 5: Compare the angle results and category results predicted by the neural network with the true values to calculate the loss, and adjust the parameters of the network to optimize the model.
[0026] Preferably, the formula for obtaining the apparent characteristics of the target is:
[0027]
[0028]
[0029] Among them, r j It represents the apparent features of the current target extracted by the small sample feature extraction network. Indicates that the target is stored in the feature representation of the apparent feature list, R i Represents the apparent feature storage list, t (2) represents the threshold and T represents the transpose.
[0030] Preferably, the IOU matching and cascade matching mechanism are used to perform data association on the motion features and appearance features of the target.
[0031] Furthermore, in the process of data association between the target's motion features and appearance features, the hyperparameters of the association results of the detection frame intersection and union ratio and the appearance feature association results are set to deal with the phenomenon of large inter-class differences and small intra-class differences in endangered animal scenes; the formula for associating the representation information and motion features is as follows:
[0032] c i,j =λd 1 (i,j)+(1-λ)d 2 (i,j)
[0033] Among them, c i,j represents the joint cost matrix of the fusion motion distance and appearance distance, λ represents the hyperparameter, and d 1 (i, j) represents the distance matrix between the detection box and the prediction box, d 2 (i,j) represents the target apparent similarity distance matrix
[0034] Furthermore, the value range of the hyperparameter λ is (0.5, 1). When the value of the hyperparameter λ is between 0.5 and 1, the network focuses on the intersection-over-union ratio of the target positions of the previous and next frames during data association, and eliminates the phenomenon of large inter-class differences and small intra-class differences in endangered animal scenes.
[0035] The present invention provides a new method for multi-target tracking, which realizes accurate and fast multi-target tracking without using a large number of training samples; the present invention designs a multi-target endangered animal tracking model based on small sample learning, which can be trained with a small number of samples, so that the trained model can accurately track the target, thereby improving the tracking efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a network structure diagram of the present invention;
[0037] Figure 2 A flow chart of data processing by the cascade matching module of the present invention;
[0038] Figure 3This is a flow chart of data processing by the IOU matching module of the present invention. DETAILED DESCRIPTION
[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] The present invention selects the multi-target tracking paradigm SDE, uses a small sample target detection algorithm based on feature reweighting in the target detection stage, uses the S2M2 method to train a feature extraction network in the tracking stage, and then combines the target's appearance features and motion features for data association. Different from the traditional multi-target tracking algorithm, both the learning of the target detection stage and the learning of the appearance features are implemented in a small sample manner.
[0041] A multi-target endangered animal tracking method based on small sample learning, the method comprising:
[0042] First, train an object detection algorithm for endangered animals that can detect endangered animals when only 1, 2, 3, 5, or 10 images are used for training. The network is trained in two steps: the first step is training based on the base class (a large amount of labeled data); the second step is few-shot fine-tuning based on base+new (only a small amount of labeled data). Use the base class data to train a feature adjustment module, through which the underlying features of the base class can be used to automatically adjust the features of the image to be detected (which may be the base class or the new class).
[0043] Then, a feature extraction network is trained using the S2M2 method to extract the apparent features of each target. This method uses self-supervision loss during training to achieve semantic feature learning, and also adds a manifold mixup algorithm, which can not only enhance data but also increase the generalization ability of the model and expand the low confidence space. In addition, the training data is elastically deformed to simulate the complex morphological transformation of animals during movement, so that more effective feature representation can be extracted for categories with only a small amount of data.
[0044] Finally, multi-target tracking is achieved based on the position information detected by the detector combined with the tracker's prediction matching and update algorithm. The tracker here eliminates the Reid network of DeepSort and instead uses a feature extraction network trained by a small sample learning method to extract the apparent features of the target and put them into the tracker for data association.
[0045] A multi-target endangered animal tracking method based on small sample learning comprises: acquiring video data to be detected, inputting the video data into a multi-target tracking model based on small samples, and obtaining a multi-target tracking result.
[0046] like Figure 1 As shown in the figure, the process of using the multi-target tracking model based on small sample learning to track multiple targets in the video data to be detected includes:
[0047] S1: Fully sample the video to be detected to obtain a video frame sequence, and preprocess each frame;
[0048] S2: Use a small sample target detector based on feature reweighting to detect targets in the preprocessed video frames to obtain the location information and category information of the targets in the video frames;
[0049] S3: Using the Kalman filter algorithm to process the position information of each target, obtain the position information of the target at the next moment, and obtain the motion characteristics of the target according to the position information of the target at the next moment;
[0050] S4: cutting out the target from the video frame according to the position information of the target in the video frame; extracting features of the cut-out target using a feature extraction network trained based on a small sample learning method to obtain more distinguishing appearance features of the target;
[0051] S5: Perform data association on the motion features and appearance features of the targets in the previous and next frames to achieve multi-target tracking.
[0052] The preprocessing process of the video frame includes: fully sampling the video to obtain a video frame sequence, passing each frame image through the Wiener filter to eliminate a series of motion blurs caused by the rapid movement of endangered animals, and cropping each frame image after filtering to obtain a 448*448 size video frame after eliminating motion blur.
[0053] The process of using the detector to detect the preprocessed video frame includes: using a small sample target detector based on feature reweighting to detect the preprocessed video frame includes: the small sample target detector based on feature reweighting includes a meta-feature extraction module, a reweighting module and a one-stage detection network; using the meta-feature extraction module to extract features of image i to obtain meta-features F=D(i); using the reweighting module to obtain the global representation ω of each target in the support set i =M(li,M i ), in the process of obtaining the global representation of the target, the mask channel is used to mark the target position in the image, so that each image in the support set becomes a four-channel (R, G, B, Mask); the meta-feature and the global representation are multiplied in the channel domain to obtain the re-weighted meta-feature A one-stage detection network is used to regress the position of the target in the video frame and extract the category information of the reweighted meta-features to obtain the position information and category information of the target in the video frame. i represents the global representation of the i-th target; M represents the reweighted module; li represents the support set data; M i Indicates the four-channel data corresponding to each image in the support set; RGB indicates the full-color image mode of the image, with three channels R\G\B, R represents red, G represents green, and B represents blue; F represents the meta-feature; N represents the total number of global representations of the target. The expressions for the position information and category information of the target in the video to be detected are:
[0054] {o i ,x i ,y i ,h i ,ω i ,c i =P (F),i＝1,…,N
[0055] Among them, i Indicates the probability of the existence of an object in the detection box, x i Indicates the horizontal coordinate of the upper left corner of the detection box, y i Indicates the vertical coordinate of the upper left corner of the detection box, h i Indicates the height of the detection box, w i Indicates the height of the detection box, c iIndicates the category label of the target in the detection box. The target score o indicates the confidence that there is a target in the bounding box, the category information c indicates what category the target in the bounding box belongs to, and the location information box is the location coordinates x, y, w, h of the bounding box in the image. After the above steps, a target will generate a lot of bounding boxes. At this time, non-maximum suppression is needed to suppress those redundant boxes and keep the box with the highest score.
[0056] After obtaining the location and category information of the target in the video frame from the target detection stage, each bounding box is cropped from the original image and passed into the feature extraction network to extract the appearance features of each target. The tracker uses the Hungarian algorithm for data association. The cost matrix of this process contains two parts, one is the IOU between the position of the original target in the current frame predicted by the Kalman filter and the target detection box in the current frame, and the other is the similarity between the appearance features of the detected object in the current frame and the previously stored appearance features.
[0057] The Kalman filter algorithm is used to process the position information of each target to obtain the position information of the target at the next moment; the processing process includes: if x is the state of the trajectory at time t-1, then the state at time t is x ′ =F·x, where F is the state transfer matrix, x ′ is the state of the trajectory at time t.
[0058] The formula for obtaining the target's motion characteristics based on the target's position information at the next moment is:
[0059]
[0060]
[0061] Among them, d j represents the newly obtained detection box, (y i ,S i ) represents the projection of the ith tracking distribution on the measurement space, y i is the mean, S i is the covariance, Indicates the part where the Mahalanobis distance is less than the threshold, represents a vector of all 1s, t (1) Indicates the threshold value.
[0062] The process of training the feature extraction network based on the small sample learning method includes:
[0063] Step 1: Use the self-supervised learning method to add a pre-task Rotation to the network, so that each image in the training set is flipped 90°, 180°, and 270° respectively, to obtain images with four flipping angles, namely 0°, 90°, 180°, and 270°, and create corresponding angle labels;
[0064] Step 2: Use elastic deformation to process the data for creating angle labels to simulate the complexity of the apparent features of endangered animals during movement. For example, the apparent features of birds are very different when they are walking on the ground and flapping their wings. Elastic deformation can increase the robustness of the feature extraction network;
[0065] Step 3: The processed image is passed through the feature extraction network to extract features and obtain the feature information of the image. In the process of feature extraction, when the image passes through the k-th layer (k is the number of middle layers of the feature extraction network) of the neural network, the manifold mixing algorithm is used to fuse the middle hidden layer vectors of the image and create a new category label.
[0066] Step 4: Predict the flip angle and image category of the image based on the features output by the feature extraction network;
[0067] Step 5: Compare the angle results and category results predicted by the neural network with the true values to calculate the loss, and adjust the parameters of the network to optimize the model.
[0068] In the process of obtaining the apparent features of the target, the minimum cosine distance between the feature sets of the most recent 100 successfully associated features of the ith tracker and the current feature vector is calculated. The situation in which the target is blocked for a long time and causes ID switch in the multi-target tracking task is processed according to the minimum cosine distance to obtain the apparent features of the target. The formula for obtaining the apparent features of the target is:
[0069]
[0070]
[0071] Among them, r j It represents the apparent features of the current target extracted by the small sample feature extraction network. Indicates that the target is stored in the feature representation of the apparent feature list, R i Represents the apparent feature storage list, t (2) represents the threshold and T represents the transpose.
[0072] The IOU matching module and the cascade matching module are used to associate the target’s motion features and appearance features. Figure 3As shown in the figure, the process of processing data using the IOU matching module includes: taking the tracking target track that has only one frame matching as the candidate tracking target track candidate of the IOU, and taking the tracking target that exceeds one frame as the unmatched tracking target unmatched track; respectively calculating the IOU distance distance between the candidate tracking target track candidate and the unmatched tracking target unmatched track; setting the IOU distance threshold; marking the target whose IOU distance is greater than the set IOU distance threshold; using the Hungarian algorithm Hungarian to match the marked tracking target with the calculated IOU distance, and filtering the matching results, deleting the area with a smaller IOU, and obtaining the matched data.
[0073] Preferably, the IOU distance threshold is set to 0.7.
[0074] like Figure 2 As shown in the figure, the process of processing data using the cascade matching module includes: assigning track indices to motion features and appearance features; setting detection indices; setting the number of frames for cyclic detection; calculating the cosine distance of motion features and appearance features, and constructing a cost matrix based on the cosine similarity; calculating the squared Mahalanobis distance between the average track position and the actual position of each tracked target predicted by the Kalman filter; setting the threshold and threshold The squared Mahalanobis distance between the tracking target in the cost matrix and the average track position of the tracking target and the actual position is greater than the set threshold The corresponding value is set to infinity; the tracking target in the cost matrix and the cosine similarity distance are greater than the set threshold The corresponding value is set to a number greater than 10, that is, the value of the cosion distance of the saliency detection result is greater than 10; the Hungarian algorithm is used to match the tracking target and the detection index to obtain the matching result; the matching results are screened and the matching results with saliency detection results greater than 10 are deleted; the above process is repeated until the set number of loop detection frames is met to obtain the preliminary matching result.
[0075] In the process of data association between the target's motion features and appearance features, the hyperparameters of the association results of the detection frame intersection and union ratio and the appearance feature association results are set to deal with the phenomenon of large inter-class differences and small intra-class differences in endangered animal scenes; the formula for fusing the representation information and motion features is:
[0076] c i,j =λd(i,j)+(1-λ)d2 (i,j)
[0077] Among them, c i,j represents the joint cost matrix of the fusion motion distance and appearance distance, λ represents the hyperparameter, and d 1 (i,j) represents the distance matrix, d 2 (i,j) represents the appearance distance matrix.
[0078] The value range of the hyperparameter λ is (0.5, 1). When the value of the hyperparameter λ is between 0.5 and 1, the network focuses on the intersection-over-union ratio of the target positions of the previous and next frames during data association, and eliminates the phenomenon of large inter-class differences and small intra-class differences in endangered animal scenes.
[0079] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A multi-target endangered animal tracking method based on small sample learning, characterized in that: For endangered animals in sparse scenes, a multi-target tracking method based on small sample learning is used to track multiple targets, including: obtaining video data to be detected, inputting the video data into a multi-target tracking model based on small samples, and obtaining multi-target tracking results; The process of using a multi-target tracking model based on small sample learning to track multiple targets in the video data to be detected includes: S1: Fully sample the video to be detected to obtain a video frame sequence, and preprocess each frame; S2: Use a small sample target detector based on feature reweighting to detect targets in the preprocessed video frames to obtain the location information and category information of the targets in the video frames; S3: Using the Kalman filter algorithm to process the position information of each target, obtain the position information of the target at the next moment, and obtain the motion characteristics of the target according to the position information of the target at the next moment; S4: cutting out the target from the video frame according to the position information of the target in the video frame; extracting features of the cut target using a feature extraction network trained based on a small sample learning method to obtain the apparent features of the target; the process of training the feature extraction network based on a small sample learning method includes: Step 1: Use the self-supervised learning method to add a pre-task Rotation to the network, so that each image in the training set is flipped 90°, 180°, and 270° respectively, to obtain images with four flipping angles, namely 0°, 90°, 180°, and 270°, and create corresponding angle labels; Step 2: Process the data for creating angle labels using elastic deformation; Step 3: The processed image is passed through the feature extraction network to extract features and obtain the feature information of the image. In the process of feature extraction, when the image passes through the k-th layer of the neural network, the manifold mixing algorithm is used to fuse the intermediate hidden layer vectors of the image and create a new category label; Step 4: Predict the flip angle and image category of the image based on the features output by the feature extraction network; Step 5: Compare the angle results and category results predicted by the neural network with the true values to calculate the loss, and adjust the parameters of the network to optimize the model; S5: Perform data association on the motion features and appearance features of the targets in the previous and next frames to achieve multi-target tracking.
2. The multi-target endangered animal tracking method based on small sample learning according to claim 1 is characterized in that: The preprocessing process of the video frame includes: fully sampling the video to obtain a video frame sequence, passing each frame image through a Wiener filter to eliminate motion blur caused by the rapid movement of endangered animals, and cropping each frame image after filtering to obtain a 448*448 size video frame after eliminating motion blur.
3. The multi-target endangered animal tracking method based on small sample learning according to claim 1 is characterized in that: The process of using a small sample target detector based on feature reweighting to detect the preprocessed video frame includes: the small sample target detector based on feature reweighting includes a meta-feature extraction module, a reweighting module and a one-stage detection network; the meta-feature extraction module is used to extract features of image i to obtain meta-features F=D(i); the reweighting module is used to obtain the global representation ω of each target in the support set i =M(li,M i ), in the process of obtaining the global representation of the target, the mask channel is used to mark the target position in the image, so that each image in the support set becomes a four-channel (R, G, B, Mask); the meta-feature and the global representation are multiplied in the channel domain to obtain the re-weighted meta-feature A one-stage detection network is used to regress the position of the target in the video frame and extract the category information of the reweighted meta-features to obtain the position information and category information of the target in the video frame. i represents the global representation of the i-th target; M represents the reweighted module; li represents the support set data; M i Indicates the four-channel data corresponding to each image in the support set; RGB indicates the full-color image mode of the image, with three channels R\G\B, R represents red, G represents green, and B represents blue; F represents meta-feature; N represents the total number of global representations of the target.
4. The multi-target endangered animal tracking method based on small sample learning according to claim 1 is characterized in that: The process of using the Kalman filter algorithm to process the position information of each target includes: after passing through the small sample target detector, the target position information and category information are obtained as the target detection frame, and the Kalman filter is used to predict the position of each detection frame in the subsequent frame; the Mahalanobis distance is used to measure the distance between the detection frame and the prediction frame, and the Mahalanobis distance is thresholded; the threshold t is set. (1) , will be less than the specified threshold t (1) The Mahalanobis distance is set to successful association.
5. The multi-target endangered animal tracking method based on small sample learning according to claim 4 is characterized in that: The formula for obtaining the target's motion characteristics is: Among them, d j represents the i-th detection box, y i represents the i-th track, is the covariance of d and y, Indicates d j With y i The Mahalanobis distance between represents a vector of all 1s, t (1) represents the threshold value, Indicates the part where the Mahalanobis distance is less than the threshold.
6. The multi-target endangered animal tracking method based on small sample learning according to claim 1 is characterized in that: The formula for obtaining the apparent characteristics of the target is: Among them, r j It represents the apparent features of the current target extracted by the small sample feature extraction network. Indicates that the target is stored in the feature representation of the apparent feature list, R i Represents the apparent feature storage list, t (2) represents the threshold and T represents the transpose.
7. The multi-target endangered animal tracking method based on small sample learning according to claim 1 is characterized in that: The IOU matching algorithm and cascade matching mechanism are used to associate the target's motion features and appearance features.
8. The multi-target endangered animal tracking method based on small sample learning according to claim 7 is characterized in that: In the process of data association between the target's motion features and appearance features, the hyperparameters of the association results of the detection box intersection and union ratio and the appearance feature association results are set to deal with the phenomenon of large inter-class differences and small intra-class differences in endangered animal scenes. The formula for associating the representation information with the motion features is: c i,j =λd 1 (i,j)+(1-λ)d 2 (i,j) Among them, c i,j represents the joint cost matrix of the fused motion distance and appearance distance; λ represents a hyperparameter; d 1 (i,j) represents the distance matrix between the detection box and the prediction box, d 2 (i,j) represents the target appearance similarity distance matrix.
9. The multi-target endangered animal tracking method based on small sample learning according to claim 8, characterized in that: The value range of the hyperparameter λ is (0.5, 1). When the value of the hyperparameter λ is between 0.5 and 1, the network focuses on the intersection-over-union ratio of the target positions of the previous and next frames during data association, and eliminates the phenomenon of large inter-class differences and small intra-class differences in endangered animal scenes.