Target tracking method, system, terminal and medium based on probability label

Abstract

The present invention provides a target tracking method, system, terminal and medium based on probability labels, including: constructing a template tracker, and using traditional labels to train the template tracker; using the trained template tracker to generate probability labels; based on the template tracker, constructing a target tracker, and using the probability labels to train the target tracker; using the trained target tracker to track a given target and output the target tracking result. The probability label is the output of the template tracker after softmax normalization, and each value in it represents the probability of the target being at the center of the corresponding candidate area. The target tracker is a lightweight network designed on the basis of a classical neural network. Its training target is no longer a binary label but a probability label. Through training, the output result of the target tracker approaches the template tracker, so the tracking performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target tracking, and in particular to a target tracking method, system, terminal and medium based on probability labels. Background Art

[0002] Object tracking is one of the most important research areas in computer vision, and has many wide applications, such as video surveillance, human-computer interaction, medical imaging, etc. In the object tracking process, given the position of the target in the first frame of the video sequence, the target tracker is required to output the position of the target in the subsequent frames. The target tracker usually consists of five parts: motion model, feature extractor, prediction model, template updater and subsequent integrator, among which the motion model generates several candidate regions that may contain the target in the current frame based on the tracking results of the previous frame; the feature extractor extracts features from the generated candidate regions; the prediction model uses the trained classification model to identify the target; the template updater uses the tracking results of the current frame to update the target template; the subsequent integrator integrates multiple possible tracking results and outputs the final tracking result.

[0003] Among non-deep learning algorithms, the correlation filter algorithm has become the most widely used target tracking algorithm because of its simplicity and efficiency. This algorithm calculates the similarity between images through correlation filtering and selects the candidate area with the greatest similarity to the template as the new position of the template. The correlation filter operation in the time domain can be transformed into a point multiplication operation in the frequency domain through discrete Fourier transform, thereby reducing the time complexity of the algorithm. The kernelized correlation filter is the most classic correlation filter algorithm in the field of target tracking. This algorithm uses a circulant matrix to encode all possible position movements of the target. The biggest feature of the circulant matrix is ​​that it can be diagonalized by discrete Fourier. The kernelized correlation filter trains ridge regression to identify the target, and introduces a kernel function to convert the nonlinear problem of input and output into a linear problem. After discrete Fourier transform optimization, the kernelized correlation filter only needs to perform linear operations to track the target, so the running speed on the CPU can exceed 100PFS. The correlation filter algorithm uses HOG features. HOG is based on grayscale features and cannot extract the color information of the image; the correlation filter algorithm predicts the target only by simply predicting the center point of the target, without predicting the scale change of the target.

[0004] Algorithms based on deep neural networks and convolutional neural networks have achieved great success in many fields of computer vision, such as image recognition, target detection, and semantic segmentation. However, for trackers, the lack of target category information makes it difficult to use similar recognition and detection algorithms to locate targets. The twin neural network abandons the classification of targets, converts the tracking task into a template matching problem, and trains the convolutional neural network end-to-end. The training sample is not a single image, but a set of image pairs, including a target template image and an image to be searched. Each set of training sample pairs is selected from different frames of the same video sequence, and is padded to ensure that the target is in the center of the image. The twin neural network contains two branches: the target template branch and the search area branch. The two branches use the same feature extraction network. The target template image size is 127*127, and the corresponding feature map size is 6*6; the image size to be searched is 255*255, and the corresponding feature map size is 22*22. Then the two feature maps are cross-correlated and a 17*17 response matrix is ​​output. Each value in the matrix represents the response value size of the corresponding candidate area.

[0005] The twin neural network uses the back propagation algorithm for end-to-end training, and the size of the training label is also 17*17. The elements on the label whose distance to the center position is less than or equal to a certain threshold are set to 1, and the remaining elements are 0. Because the target is in the center of the target template image and the image to be searched, the response value at the center should be larger, and the response value at the edge should be smaller. The loss function uses the cross entropy loss function. Each response value is normalized by the sigmoid function and the loss between the response value and the label is calculated. All losses are added and averaged before back propagation.

[0006] When tracking, the target template image is usually the image of the target in the first frame, and the image to be searched is usually a sub-region of the frame to be tracked centered on the position of the target in the previous frame. The position of the target in the new frame is determined by the maximum value of the response matrix. In order to cope with the scale change of the target in the video sequence, the twin neural network will examine the images to be searched of different scales, scale them to the same size and input them into the network. The scale of the image with the largest response value is the scale of the target in the next frame.

[0007] Based on the twin neural network, researchers have proposed some improved algorithms. However, most target tracking algorithms based on the twin neural network use binary labels to train the network. This training method has the following two main shortcomings:

[0008] 1. Use the sigmoid function to normalize a single response value. Each candidate region contains the target, which is a mutually exclusive event. If the probability that a candidate region contains the target is high, the response values ​​of other candidate regions should be suppressed; if the sigmoid function is used to normalize a single response value, this effect will not be achieved.

[0009] 2. Binarized training labels contain less supervision information. Binarized labels can only represent positive samples and negative samples, and target tracking is different from target recognition. In addition to identifying the target, target tracking also requires locating the target; therefore, if the target is in different locations in the candidate area, these samples should be assigned different labels.

[0010] After searching, we found:

[0011] The Chinese invention patent application "Video Target Tracking Method Combining Particle Filter and Metric Learning" with publication number CN112085765A first offline trains a convolutional neural network that can effectively obtain high-level abstract features of the target; then, based on the kernel regression metric learning method, a weighted distance metric matrix is ​​learned to minimize the kernel regression prediction error, and the gradient descent method is used to solve the resulting optimization problem to obtain the distance metric matrix that represents the optimal candidate target; further, the reconstruction error is calculated based on the obtained optimal candidate target prediction value to construct a target observation model; finally, an update strategy that combines short-term and long-term stable updates is introduced, and effective target tracking is achieved based on the particle filter tracking framework.

[0012] The Chinese invention patent application with publication number CN113554679A, "An anchor-free target tracking algorithm for computer vision applications", is based on the cross-perception network model SiamCross of the twin architecture. In SiamCross, a new twin cross-perception subnetwork SCAN is designed based on the mutual supervision of the twin network branches. SCAN enables the features extracted by the twin network to effectively perceive and distinguish the foreground and background. The anchor-free region candidate network directly defines the tracking task as the classification and regression of each pixel, and the network branch features can each focus on the local and global spatial information of the target. The algorithm has the following problems:

[0013] The annotation methods of the above two methods are both binary labels, which fail to fully characterize the candidate area information and will affect the deep network's ability to extract and identify features; on the other hand, the algorithm is relatively complex, which affects the tracking efficiency. The present invention cleverly uses a complex template tracker to generate probabilistic labels, uses the probabilistic labels containing rich information to train the target tracker, and uses a target tracker with a small number of parameters for real-time tracking, while giving full play to the advantages of complex networks and ensuring tracking efficiency. Summary of the invention

[0014] In view of the deficiencies in the prior art, the present invention provides a target tracking method, system, terminal and medium based on probability labels, which use probability labels to train twin neural networks, so that performance close to that of a template tracker can be achieved while involving fewer parameters.

[0015] According to one aspect of the present invention, a target tracking method based on probability labels is provided, comprising:

[0016] Building a template tracker and training the template tracker using traditional labels;

[0017] Using the trained template tracker, generating a probability label;

[0018] Based on the template tracker, a target tracker is constructed, and the target tracker is trained using the probability label;

[0019] Tracking a given target using the trained target tracker and outputting a target tracking result;

[0020] in:

[0021] The output result of the trained template tracker is normalized by softmax to obtain the probability label; any value in the probability label is greater than 0 and less than 1; the elements in the probability label represent the label of the corresponding candidate area in the search image, and the label of the corresponding candidate area in the search image represents the probability of the target appearing at the center of the candidate area;

[0022] The output result of the trained target tracker is normalized by softmax to obtain the target tracking result.

[0023] Preferably, the step of constructing a template tracker and training the template tracker using traditional labels comprises:

[0024] Adopting a twin neural network to construct the template tracker;

[0025] The template tracker is trained online or offline using the binary labels.

[0026] Preferably, the online or offline training of the template tracker using the binary label comprises:

[0027] Obtain a set of sample pairs and binary labels, wherein the sample pairs include a target template image and a search range image, and the target template image and the search range image are captured from different frames of the same video sequence;

[0028] The sample pair is input into the template tracker, the deep convolution feature is extracted through the feature extraction branch, and a cross-correlation operation is performed, then:

[0029] f(z,x;θ)=φ(z;θ)*φ(x;θ)+b

[0030] Wherein, z represents the target template image, x represents the search range image, θ represents the parameters of the feature extraction network, φ represents the feature extraction branch, * represents the cross-correlation operation; b represents the bias parameter, and f(z,x;θ) represents the response matrix output after the cross-correlation operation;

[0031] The loss function is defined as the cross entropy loss l between the response matrix and the binary label CE (Y b ,R), then:

[0032]

[0033] Among them, σ(·) represents the sigmoid function, Yb represents the binary label, and R is a brief representation of the response matrix f(z,x;θ).

[0034] Preferably, constructing a target tracker based on the template tracker comprises:

[0035] Based on the template tracker, the template tracker is lightweighted by reducing the number of output channels and / or using depthwise separable convolution to construct the target tracker.

[0036] Preferably, the adopting the probability label to train the target tracker includes:

[0037] Using the probability labels as training samples, constructing training sample pairs;

[0038] The training sample pair is simultaneously input into the template tracker and the target tracker to obtain the output results of the template tracker and the target tracker.

[0039] The output results of the template tracker and the target tracker are normalized by softmax and then the loss value is calculated;

[0040] The gradient back propagation method is adopted to update the parameters of the target tracker based on the loss value to complete the training of the target tracker.

[0041] Preferably, during the training of the target tracker:

[0042] The training sample pairs are simultaneously input into the template tracker and the target tracker, and the output result, i.e., the response matrix, is normalized by softmax and the loss value is calculated, where the loss function is the relative entropy loss l RE (Y p ,R N ):

[0043]

[0044] Among them, Y p represents the response matrix of the template tracker after softmax normalization, that is, the probability label; R N Represents the response matrix of the target tracker after softmax normalization.

[0045] According to another aspect of the present invention, a target tracking system based on probability labels is provided, comprising:

[0046] A template tracker module, which is used to construct a template tracker and train the template tracker using traditional labels; and generate probability labels using the trained template tracker;

[0047] A target tracker module, which constructs a target tracker based on the template tracker and trains the target tracker using the probability label; tracks a given target using the trained target tracker and outputs a target tracking result;

[0048] in:

[0049] The output result of the trained template tracker is normalized by softmax to obtain the probability label; any value in the probability label is greater than 0 and less than 1; the elements in the probability label represent the label of the corresponding candidate area in the search image, and the label of the corresponding candidate area in the search image represents the probability of the target appearing at the center of the candidate area;

[0050] The output result of the trained target tracker is normalized by softmax to obtain the target tracking result.

[0051] Preferably, the template tracker and the target tracker both adopt twin neural networks, and the target tracker is obtained by performing lightweight processing based on the template tracker.

[0052] The parameters of the template tracker and the target tracker are shown in the following table, where ExSN represents the template tracker network structure and EfSN represents the target tracker network structure:

[0053]

[0054] According to a third aspect of the present invention, there is provided a terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute any of the methods described above, or to run any of the systems described above when executing the program.

[0055] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any of the methods described above, or to run any of the systems described above.

[0056] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0057] The probability label-based target tracking method, system, terminal and medium provided by the present invention overcome the shortcomings of the conventional training twin neural network algorithm in the prior art, and can still achieve performance close to that of a template tracker with fewer parameters.

[0058] The target tracking method, system, terminal and medium based on probability labels provided by the present invention have a training target that is no longer a binary label but a probability label; through training, the output result of the target tracker approaches that of the template tracker, thereby improving the tracking performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0060] Figure 1 The flowchart of the target tracking method based on probability label in one embodiment of the present invention.

[0061] Figure 2 Schematic diagram of the operation of a target tracking method based on probability labels in a preferred embodiment of the present invention.

[0062] Figure 3 The figure is a schematic diagram of the composition structure of a target tracking system based on probability labels in one embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0064] Figure 1A flowchart of a target tracking method based on probability labels provided by an embodiment of the present invention.

[0065] like Figure 1 As shown, the target tracking method based on probability labels provided in this embodiment may include the following steps:

[0066] S100, build a template tracker and train the template tracker using traditional labels;

[0067] S200, using the trained template tracker to generate a probability label;

[0068] S300, builds a target tracker based on the template tracker and uses probabilistic labels to train the target tracker;

[0069] S400, tracking a given target using the trained target tracker and outputting a target tracking result;

[0070] in:

[0071] In S200 of this embodiment, the output result of the trained template tracker is normalized by softmax to obtain a probability label; any value in the probability label is greater than 0 and less than 1; the elements in the probability label represent the label of the corresponding candidate area in the search image, and the label of the corresponding candidate area in the search image represents the probability that the target appears at the center of the candidate area;

[0072] In S400 of this embodiment, the output result of the trained target tracker is normalized by softmax to obtain the target tracking result.

[0073] In S100 of this embodiment, as a preferred embodiment, constructing a template tracker and using traditional labels to train the template tracker may include the following steps:

[0074] S101, uses twin neural networks to build template trackers;

[0075] S102, training the template tracker online or offline using the binary label.

[0076] In S102 of this embodiment, as a preferred embodiment, training the template tracker online or offline using the binary label may include the following steps:

[0077] S1021, obtaining a set of sample pairs and binary labels, wherein the sample pair includes a target template image and a search range image, and the target template image and the search range image are captured from different frames of the same video sequence;

[0078] S1022, input the sample pair to the template tracker, extract the deep convolution feature through the feature extraction branch, and perform cross-correlation operation, then:

[0079] f(z,x;θ)=φ(z;θ)*φ(x;θ)+b

[0080] Wherein, z represents the target template image, x represents the search range image, θ represents the parameters of the feature extraction network, φ represents the feature extraction branch, * represents the cross-correlation operation; b represents the bias parameter, and f(z,x;θ) represents the response matrix output after the cross-correlation operation;

[0081] S1023, the loss function is defined as the cross entropy loss l between the response matrix and the binary label CE (Y b ,R), then:

[0082]

[0083] Where σ(·) represents the sigmoid function, Y b represents the binary label, and R is a concise representation of the response matrix f(z,x;θ).

[0084] In S300 of this embodiment, as a preferred embodiment, constructing a target tracker based on a template tracker may include the following steps:

[0085] Based on the template tracker, the template tracker is lightweighted by reducing the number of output channels and / or using depthwise separable convolution to construct a target tracker.

[0086] In S300 of this embodiment, as a preferred embodiment, the target tracker is trained using probability labels, which may include the following steps:

[0087] S301, using the probability labels as training samples to construct training sample pairs;

[0088] S302: Input the training sample pair to the template tracker and the target tracker at the same time to obtain the output results of the template tracker and the target tracker.

[0089] S303, calculating the loss value after performing softmax normalization on the output results of the template tracker and the target tracker;

[0090] S304, using the gradient back propagation method, updating the parameters of the target tracker based on the loss value, and completing the training of the target tracker.

[0091] Further, as a preferred embodiment, in the process of training the target tracker:

[0092] The training sample pairs are simultaneously input into the template tracker and the target tracker, and the output result, i.e., the response matrix, is normalized by softmax and the loss value is calculated, where the loss function is the relative entropy loss l RE (Y p ,R N ):

[0093]

[0094] Among them, Y p represents the response matrix of the template tracker after softmax normalization, that is, the probability label; R N Represents the response matrix of the target tracker after softmax normalization.

[0095] The target tracking method based on probability labels provided by the above embodiment of the present invention first trains a template tracker, the network of which contains more convolutional layers and parameters, and therefore has stronger discrimination ability; the template tracker can generate probability labels for training the target tracker; the network of this target tracker reduces the number of parameters on the basis of the network of the template tracker; when training the target tracker, the training sample pairs are simultaneously input to the template tracker and the target tracker, and the outputs of the two are normalized using the softmax function; then the relative entropy loss function is used to calculate the loss value, the gradient is back-propagated, and the parameters of the target tracker are updated; and the training of the target tracker is completed. The trained target tracker can track the target in the image.

[0096] Figure 2 A schematic diagram of the operation of a target tracking method based on probability labels provided by a preferred embodiment of the present invention.

[0097] like Figure 2 As shown, the target tracking method based on probability labels provided by the preferred embodiment may include the following steps:

[0098] Step 1: Build a template tracker and train it using traditional labels;

[0099] Step 2: Generate probability labels using template tracker;

[0100] Step 3: Based on the template tracker, a lightweight target tracker is constructed, and the lightweight target tracker is trained using probabilistic labels.

[0101] Step 4: Use the target tracker to complete the tracking task of the given target in the image and output the target tracking result.

[0102] In step 1 of the preferred embodiment, the template tracker may be trained online or offline using binary labels.

[0103] In step 2 of the preferred embodiment, using a template tracker to generate a probability label may include the following steps:

[0104] After the output of the template tracker is normalized by softmax, a probability label is obtained, and any value in the probability label is greater than 0 and less than 1.

[0105] In step 2 of the preferred embodiment, the elements in the probability label represent the labels of the corresponding candidate regions in the search image, ie, the probability that the target appears at the center of the candidate region.

[0106] In step 3 of the preferred embodiment, building a lightweight target tracker based on the template tracker may include the following steps:

[0107] The template tracker is lightweighted by reducing the number of output channels and / or using depthwise separable convolution, so as to construct a lightweight target tracker.

[0108] In step 3 of the preferred embodiment, using probabilistic labels to train a lightweight target tracker may include the following steps:

[0109] Step 31, using the probability labels as training samples and inputting them into the template tracker and the lightweight target tracker at the same time;

[0110] Step 32, the outputs of the template tracker and the lightweight target tracker are both normalized by softmax and then the loss value is calculated;

[0111] Step 33, using the gradient back propagation method, updates the network parameters of the lightweight target tracker to complete the training of the lightweight target tracker.

[0112] The target tracking method based on probability labels provided by the above preferred embodiment of the present invention is specifically as follows:

[0113] Construct a template tracker. This network is trained using traditional binary labels. The training sample is a set of sample pairs and binary labels. The input sample pair is a target template image and a search range image. These two images are captured from different frames of the same video sequence. After the feature extraction branch extracts the deep convolution features, a cross-correlation operation is performed and the response matrix is ​​output. This process can be expressed as:

[0114] f(z,x;θ)=φ(z;θ)*φ(x;θ)+b

[0115] Where z represents the target template image, x represents the search range image, θ represents the parameters of the feature extraction network, and φ represents the feature extraction branch. * represents the cross-correlation operation. b represents the bias parameter, and f(z,x;θ) represents the response matrix output after the cross-correlation operation.

[0116] The loss function is defined as the cross entropy loss between the response matrix and the binary labels:

[0117]

[0118] where σ(·) represents the sigmoid function, Y b represents the binary label and R represents the response matrix.

[0119] The template tracker contains more convolutional layers and parameters, and has stronger discrimination and learning capabilities.

[0120] Generate probability labels. Probability labels are the core of this preferred embodiment. Binary labels only contain "1" and "0", which represent positive samples and negative samples respectively. For target tracking, in addition to distinguishing between positive and negative samples, it is also necessary to determine the location of the target. Therefore, for two candidate regions that both contain targets but have different target locations, their labels should be different. Probability labels overcome the shortcomings of binary labels. The label value is greater than 0 and less than 1, indicating the probability of the target being at the center of the candidate region. Unlike binary labels that are manually set, probability labels are generated by template trackers. As a probability distribution, the sum of all label values ​​should be 1. The probability distribution is the output of the template tracker after softmax normalization.

[0121] Build a lightweight target tracker. The lightweight target tracker is a lightweight twin neural network. It is designed based on the template tracker, but has fewer convolutional layers and parameters. There are two ways to construct the target tracker: reducing the number of channels and using depthwise separable convolution. When training the target tracker, the probabilistic labels used need to be generated by the template tracker. The training method is:

[0122] The training sample pairs are simultaneously input into the template tracker and the target tracker, and the output result, i.e., the response matrix, is normalized by softmax and the loss value is calculated, where the loss function is the relative entropy loss l RE (Y p ,R N ):

[0123]

[0124] Among them, Y p represents the response matrix of the template tracker after softmax normalization, that is, the probability label; RN Represents the response matrix of the target tracker after softmax normalization.

[0125] When Y p With R N When the two are the same, the relative entropy is the smallest. Therefore, when the above training algorithm is used to train the target tracker, its output will be close to the output of the template tracker, so it can learn similar discrimination capabilities and obtain better tracking performance.

[0126] In the target tracking method based on probability labels provided in the above embodiment of the present invention:

[0127] The template tracker is used to generate probabilistic labels. This network is trained using traditional training methods. It is a deeper and wider network with more parameters. It is more time-consuming in the offline training phase, but has stronger discrimination ability. Since the role of the template tracker is to output probabilistic labels and is not used for online tracking, it is acceptable to spend more time training the network offline.

[0128] The probability label is obtained by normalizing the output of the template tracker mentioned above by softmax. Any value in the label is greater than 0 and less than 1, which is different from the binary label that only contains 0 and 1. The elements in the probability label represent the label of the corresponding candidate region in the search image, measuring the probability of the target appearing at the center of the candidate region.

[0129] The lightweight target tracker is a lightweight network designed based on the template tracker. There are two main lightweight methods used: reducing the number of output channels and using depthwise separable convolution. The lightweight target tracker contains fewer convolutional layers and fewer parameters. Using the above probabilistic label training, the performance of the lightweight target tracker can be close to that of the template tracker, and it can run in real time.

[0130] In some embodiments of the present invention:

[0131] The labels for training the twin neural network are not traditional binary labels but probabilistic labels.

[0132] The probability label is the result of the template tracker output being globally normalized by softmax. The output of the target tracker is also globally normalized by softmax instead of point-by-point normalization by sigmoid.

[0133] Figure 3 A schematic diagram of the structure of a target tracking system based on probability labels provided by an embodiment of the present invention.

[0134] like Figure 3 As shown, the target tracking system based on probability labels provided by this embodiment may include:

[0135] A template tracker module, which is used to build a template tracker and train the template tracker using traditional labels; using the trained template tracker to generate probabilistic labels;

[0136] The target tracker module builds a target tracker based on the template tracker and uses probability labels to train the target tracker. The trained target tracker is used to track a given target and output the target tracking result.

[0137] in:

[0138] The output of the trained template tracker is normalized by softmax to obtain a probability label; any value in the probability label is greater than 0 and less than 1; the elements in the probability label represent the label of the corresponding candidate area in the search image, and the label of the corresponding candidate area in the search image represents the probability that the target appears in the center of the candidate area;

[0139] The output of the trained target tracker is normalized by softmax to obtain the target tracking result.

[0140] In this embodiment, as a preferred embodiment, both the template tracker and the target tracker use twin neural networks, and the target tracker is obtained by lightweight processing based on the template tracker.

[0141] The parameters of the template tracker and the target tracker are shown in Table 1, where ExSN represents the template tracker network structure and EfSN represents the target tracker network structure:

[0142] Table 1

[0143]

[0144] It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system, that is, the embodiments in the method can be understood as preferred examples for constructing the system, which will not be elaborated here.

[0145] An embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the terminal can be used to execute any method of the above-mentioned embodiments of the present invention, or to run any system of the above-mentioned embodiments of the present invention.

[0146] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any method of the above embodiments of the present invention, or to run any system of the above embodiments of the present invention.

[0147] In the above two embodiments, optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic RandomAccess Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above methods), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0148] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0149] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0150] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0151] The target tracking method, system, terminal and medium based on probability labels provided by the above-mentioned embodiments of the present invention first use the traditional training method to train the template tracker, and then train the target tracker. When training the target tracker, the training sample pairs are simultaneously input into the template tracker and the target tracker, and the outputs of both are simultaneously normalized by softmax to calculate the loss value, and then the gradient back propagation is used to update the parameters of the target tracker so that its output can approach the template tracker. It overcomes the shortcomings of the conventional training twin neural network algorithm in the prior art, and can still achieve performance close to that of the template tracker with fewer parameters; its training target is no longer a binary label, but a probability label; through training, the output result of the target tracker approaches the template tracker, so the tracking performance is improved.

[0152] The above describes the specific implementation of the present invention. It should be understood that the present invention is not limited to the above specific implementation, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention.

Claims

1. A target tracking method based on probability labels, characterized in that: include: Building a template tracker and training the template tracker using traditional labels; Using the trained template tracker, generating a probability label; Based on the template tracker, a target tracker is constructed, and the target tracker is trained using the probability label; Tracking a given target using the trained target tracker and outputting a target tracking result; in: The output result of the trained template tracker is normalized by softmax to obtain the probability label; any value in the probability label is greater than 0 and less than 1; the elements in the probability label represent the label of the corresponding candidate area in the search image, and the label of the corresponding candidate area in the search image represents the probability of the target appearing at the center of the candidate area; The output result of the trained target tracker is normalized by softmax to obtain the target tracking result; The step of constructing a template tracker and training the template tracker using a traditional label includes: Adopting a twin neural network to construct the template tracker; Training the template tracker online or offline using the binary labels; The step of training the template tracker online or offline using the binary label includes: Obtain a set of sample pairs and binary labels, wherein the sample pairs include a target template image and a search range image, and the target template image and the search range image are captured from different frames of the same video sequence; The sample pair is input into the template tracker, the deep convolution feature is extracted through the feature extraction branch, and a cross-correlation operation is performed, then: f(z,x;θ)=φ(z;θ)*φ(x;θ)+b Wherein, z represents the target template image, x represents the search range image, θ represents the parameters of the feature extraction network, φ represents the feature extraction branch, * represents the cross-correlation operation; b represents the bias parameter, and f(z,x;θ) represents the response matrix output after the cross-correlation operation; The loss function is defined as the cross entropy loss l between the response matrix and the binary label CE (Y b ,R), then: Where σ(·) represents the sigmoid function, Y b represents the binary label, R is a brief representation of the response matrix f(z,x;θ); The step of constructing a target tracker based on the template tracker includes: Based on the template tracker, the template tracker is lightweight processed by reducing the number of output channels and / or using depthwise separable convolution to construct the target tracker; The adopting the probability label to train the target tracker includes: Using the probability labels as training samples, constructing training sample pairs; Inputting the training sample pair to the template tracker and the target tracker at the same time to obtain output results of the template tracker and the target tracker; The output results of the template tracker and the target tracker are normalized by softmax and then the loss value is calculated; Adopting a gradient back propagation method, updating the parameters of the target tracker based on the loss value, and completing the training of the target tracker; During the training of the object tracker: The training sample pairs are simultaneously input into the template tracker and the target tracker, and the output result, i.e., the response matrix, is normalized by softmax and the loss value is calculated, where the loss function is the relative entropy loss l RE (Y p ,R N ): Among them, Y p represents the response matrix of the template tracker after softmax normalization, that is, the probability label; R N Represents the response matrix of the target tracker after softmax normalization.

2. A target tracking system based on probability labels, using the target tracking method based on probability labels according to claim 1, characterized in that: include: A template tracker module, the module is used to build a template tracker and train the template tracker using traditional labels; Using the trained template tracker, generating a probability label; A target tracker module, which constructs a target tracker based on the template tracker and trains the target tracker using the probability label; Tracking a given target using the trained target tracker and outputting a target tracking result; in: The output result of the trained template tracker is normalized by softmax to obtain the probability label; any value in the probability label is greater than 0 and less than 1; the elements in the probability label represent the label of the corresponding candidate area in the search image, and the label of the corresponding candidate area in the search image represents the probability of the target appearing at the center of the candidate area; The output result of the trained target tracker is normalized by softmax to obtain the target tracking result.

3. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it can be used to execute the method described in claim 1, or run the system described in claim 2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to execute the method described in claim 1, or to run the system described in claim 2.

Images