Target tracking method and device

Abstract

The present application provides a target tracking method and device, wherein the target tracking method comprises: obtaining an image to be tracked; inputting the image to be tracked into a trained target tracking model to obtain a target tracking result of the image to be tracked; wherein the target tracking model comprises a classification branch and a regression branch, the target tracking model is obtained by training a preset neural network based on regression samples and classification samples, the regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch. It effectively ensures that the classification and regression results are aligned during the training process, thereby improving the robustness and accuracy of the target tracking model, and effectively improving the efficiency and accuracy of target tracking using the target tracking model.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target tracking method. The present application also relates to a target tracking device, a computing device, and a computer-readable storage medium. Background Art

[0002] Object tracking is one of the most important research topics in the field of computer vision. After years of development, from the traditional sparse subspace tracking method to the correlation filter tracking method, and now to the tracking method based on deep twin networks, object tracking technology has made great breakthroughs in accuracy and speed. Thanks to the continuous optimization of tracking algorithms and the upgrading of intelligent hardware, object tracking has been widely used in fields such as intelligent transportation, industrial robots and autonomous driving.

[0003] In the prior art, target tracking is usually performed based on a deep twin network. The process of single target tracking is to automatically estimate the position of any object based on the initial bounding box in the first frame of each frame of a given video sequence, that is, to select the target object in the first frame. The candidate sample is the search area in each subsequent frame, and then the deep twin network is used to calculate the correlation between the target object in the first frame and the search area in the subsequent frames, and the tracking of a target object is achieved by classifying and positioning the candidate samples. However, the deep twin network used in the above method has different sensitivities to the feature space between the classification branch and the regression branch. The classification branch is only sensitive to a specific area, while the regression branch is sensitive to the target as a whole. Therefore, the classification branch and the regression branch need to learn different features, and the classification branch and the regression branch are usually optimized independently during the training process. This also leads to the problem of mismatch between the classification and positioning of the tracking task during target tracking, thereby reducing the accuracy of target tracking. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a target tracking method. The present application also relates to a target tracking device, a computing device, and a computer-readable storage medium to solve the technical defects of low efficiency and poor accuracy of target tracking in the prior art.

[0005] According to a first aspect of an embodiment of the present application, a target tracking method is provided, comprising:

[0006] Acquire the image to be tracked;

[0007] Inputting the image to be tracked into a trained target tracking model to obtain a target tracking result of the image to be tracked;

[0008] Among them, the target tracking model includes a classification branch and a regression branch. The target tracking model is obtained by training a preset neural network based on regression samples and classification samples. The regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch.

[0009] According to a second aspect of an embodiment of the present application, a target tracking device is provided, including:

[0010] An acquisition module, configured to acquire an image to be tracked;

[0011] A tracking module is configured to input the image to be tracked into a trained target tracking model to obtain a target tracking result of the image to be tracked;

[0012] Among them, the target tracking model includes a classification branch and a regression branch. The target tracking model is obtained by training a preset neural network based on regression samples and classification samples. The regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch.

[0013] According to a third aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the target tracking method when executing the computer instructions.

[0014] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed by a processor, the steps of the target tracking model training method or the target tracking model method are implemented.

[0015] The target tracking method provided by the present application obtains an image to be tracked; inputs the image to be tracked into a trained target tracking model, and inputs the image to be tracked into the trained target tracking model to obtain the target tracking result of the image to be tracked; wherein the target tracking model includes a classification branch and a regression branch, and the target tracking model is obtained by training a preset neural network based on regression samples and classification samples, and the regression samples are determined according to the initial classification loss of the initial samples obtained by the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained by the regression branch. Through the cross-guided training method of the initial classification loss and the initial regression loss, the classification branch and the regression branch interact better during training, that is, the regression sample is determined by the initial classification loss for the training of the regression branch, so that the regression branch can pay more attention to the position with high classification weight; the classification sample is determined by the initial regression loss for the training of the classification branch, so that the classification task has a higher classification weight for the high-precision prediction box, which effectively ensures the alignment of the classification and regression results during the training process, thereby improving the robustness and accuracy of the target tracking model. Then, the target tracking model is used to track the image to be tracked, effectively improving the efficiency and accuracy of target tracking using the target tracking model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a target tracking model training method provided by an embodiment of the present application;

[0017] Figure 2 is a structural diagram of a preset neural network provided in one embodiment of the present application;

[0018] Figure 3 It is a flowchart of a target tracking model training method provided by an embodiment of the present application;

[0019] Figure 4 is a flow chart of a target tracking method provided by an embodiment of the present application;

[0020] Figure 5 is a processing flow chart of a target tracking method provided by an embodiment of the present application;

[0021] Figure 6 It is a structural schematic diagram of a target tracking device provided by an embodiment of the present application;

[0022] Figure 7 It is a structural diagram of a target tracking model training module provided by an embodiment of the present application;

[0023] Figure 8 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0025] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to any or all possible combinations of one or more associated listed items.

[0026] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] First, the terms involved in one or more embodiments of the present application are explained.

[0028] Target tracking: also known as target tracking, refers to giving a rectangular box containing the target in the first frame of the video, also known as the target box, which is used to frame the position of the target in the image in subsequent video frames.

[0029] Correlation filtering: used to measure the similarity between two samples. The more similar the regions are, the higher the response value is. Correlation filtering originates from the field of signal processing. Correlation is used to indicate the similarity between two signals. Convolution is usually used to represent the correlation operation. The idea of ​​correlation filtering in target tracking methods is to learn a filter template, perform convolution operation with the next frame image and the filter template, and the region with the largest response is the predicted target.

[0030] Deep Siamese network: a network with two identical structures and shared weights, which receives two inputs and calculates the distance between the two output vectors using some distance metric.

[0031] Then, the target tracking method provided by this application is described.

[0032] As one of the hot topics in computer vision research, target tracking has great commercial value and broad application prospects. After more than 20 years of development, target tracking technology has made great breakthroughs in accuracy and speed, from the traditional sparse subspace tracking method to the correlation filter tracking method, and now the tracking method based on deep twin networks. Thanks to the continuous optimization of tracking algorithms and the upgrading of intelligent hardware, target tracking has been widely used in fields such as intelligent transportation, industrial robots and autonomous driving.

[0033] The current mainstream tracking method uses a deep twin network architecture. In a single target tracking task, the template image used as the benchmark is the target to be tracked. Usually, the image area where the target is located in the first frame of the video is selected as the template image, and the search image is the search area in each subsequent frame. The deep twin network calculates the correlation between the template image, that is, the target in the first frame and the search area in subsequent frames, and tracks a target by classifying and locating the search image.The SINT (Siamese Instance Search for Tracking) method pioneered the application of deep twin networks in the field of target tracking. The SiamFC (Fully-Convolutional Siamese Networks for Object Tracking) method significantly improved the tracking speed and accuracy of deep trackers, but it cannot adapt well to the scale and proportion changes of target boxes. The SiamRPN (High Performance Visual Tracking with Siamese Region Proposal Network) method borrowed the idea of ​​extracting candidate boxes (RPN, Region Proposal Network) from the Faster R-CNN (Towards Real-Rime Object Detection with Region Proposal Networks) method, and introduced a regression (positioning) branch based on SiamFC to achieve accurate prediction of target boxes. The SiamRPN++ (Evolution of Siamese Visual Tracking with Very Deep Networks) method is an upgraded version of SiamRPN. By uniformly sampling samples during training, it solves the position bias problem caused by boundary filling operations in deep networks, making deeper networks, such as ResNet50, Residual Network 50, more accurate. Inspired by the FCOS (Fully Convolutional One-Stage Object Detection) method, the SiamFC++ (Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines) method, the SiamBAN (Siamese Box Adaptive Network for Visual Tracking) method, and the SiamCAR (Siamese Fully Convolutional Classification and Regression for Visual Tracking) network introduced a design without a priori box to avoid the hyperparameters related to the candidate box, making the tracker more flexible and universal.

[0034] However, the above method has the problem of mismatch between tracking task classification and positioning, such as the prediction box corresponding to the position with high classification weight is very poor, and the prediction box corresponding to the position with low classification weight is good. In addition, when the existing deep twin tracking network performs target tracking, it realizes target classification and target box positioning through classification branch and positioning branch, ignoring the prediction quality problem of the target box, thereby reducing the efficiency of target tracking.

[0035] Therefore, the target tracking method provided by the present application obtains the image to be tracked; then the image to be tracked is input into the trained target tracking model to obtain the target tracking result of the image to be tracked, wherein the target tracking model includes a classification branch and a regression branch, and the target tracking model is obtained by training a preset neural network based on regression samples and classification samples, and the regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch. That is, the target tracking method provided by the present application first trains the target tracking model including the classification branch and the regression branch: through the cross-guided training method of the initial classification loss and the initial regression loss, the classification branch and the regression branch interact better during training, that is, the regression sample is determined by the initial classification loss for the training of the regression branch, so that the regression branch can pay more attention to the position with high classification weight; the classification sample is determined by the initial regression loss for the training of the classification branch, so that the classification task has a higher classification weight for the high-precision prediction box, which effectively ensures the alignment of the classification and regression results during the training process, thereby improving the robustness and accuracy of the target tracking model. Then, the target tracking model is used to track the target in the image to be tracked, which effectively improves the efficiency and accuracy of target tracking using the target tracking model.

[0036] In the present application, a target tracking method is provided. The present application also relates to a target tracking device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0037] Figure 1 A flow chart of a target tracking model training method provided according to an embodiment of the present application is shown, which specifically includes the following steps:

[0038] Step 102: Obtain a training image group, input the training image group into a preset neural network, and obtain an initial classification loss and an initial regression loss of each initial sample in the search image, the preset neural network includes a classification branch and a regression branch, and the training image group includes a template image and a search image.

[0039] Specifically, the preset neural network refers to a neural network of a pre-set deep twin network, such as the Siamese region proposal network (Siam-RPN), the fully convolutional Siamese network for object tracking (SiamFC), and the new Siamese network based on SiamFC (SiamFC++, Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines), including a classification branch and a regression branch, wherein the classification branch is used to achieve the classification task, that is, the recognition of the target, and the regression branch is used to achieve the regression of the position, that is, the positioning of the target. The training image group refers to an image group used to train the preset neural network, including a template image and a search image, wherein the template image is the target to be tracked, and the image area where the target is located in the first frame of the video is usually selected as the template image, and the search image can be the search area in the second frame and each subsequent frame of the video. The initial sample refers to the image area centered on a sampling point in the search image, and the search image includes at least one sampling point, and the sampling point corresponds to the initial sample one by one, so the search image contains at least one initial sample. The initial classification loss refers to the loss value directly obtained by the classification branch for the classification task of the search image; the initial regression loss refers to the loss value directly obtained by the regression branch for the positioning task of the search image.

[0040] In practical applications, a training image group can be extracted from the training set, that is, a template image and a search image corresponding to the template image can be extracted from the training set, and the template image and the search image can be input into a preset neural network. The preset neural network processes the template image and the search image accordingly, and then obtains the initial classification loss of each initial sample in the search image based on the focal loss function (FL), and obtains the initial regression loss of each initial sample in the search image based on the evaluation index (IoU, Intersection over Union).

[0041] See also Figure 2 , Figure 2The structure diagram of a preset neural network provided by an embodiment of the present application is shown: the preset neural network includes five units in the horizontal direction, namely, an input unit, a feature extraction unit, a calculation unit, a feature enhancement unit, and an output unit, and includes two branches in the vertical direction, namely, a classification branch and a regression branch. Among them, the input unit is the input end of the preset neural network, including two input branches, namely, a template branch and a regression branch. The input branch on the upper side is a classification branch, which is used to input a template image of a specific size, for example, the input template image size is respectively height=127, width=127, and channels=3; the input branch on the lower side is a regression branch, which is used to input a search image of a specific size, for example, the input search image size is respectively height=255, width=255, and channels=3. The feature extraction unit is used to obtain the abstract features of the image. It can transform the low-dimensional features of the image into high-dimensional features by amplifying the channels layer by layer to realize feature extraction. In order to lightweight the preset neural network, the feature extraction unit uses a lightweight version of the lightweight network (MobileNetV3) to perform 16-fold downsampling on the input image to extract the feature vector, so that the output feature size corresponding to the classification branch is high = 8, width = 8, and channel = 96, and the output feature size corresponding to the regression branch is high = 16, width = 16, and channel = 96. The calculation unit is used to perform related operations for the classification task and the regression task, that is, related filtering or convolution calculation, which can measure the similarity of two output features. More specifically, the features of the template image are used as the convolution kernel and the features of the search image are used as the convolved feature map. The convolution calculation is performed through the features of the two input branches to calculate the correlation of the features. The feature enhancement unit is used to stack multiple convolution blocks, that is, to enhance the learning of the features after the related operations, that is, to convolve the related features after connecting multiple convolution layers in series. The output unit is used to output the prediction results of the classification branch and the regression branch, such as outputting the classification, classification weight and center quality weight according to the characteristics of the classification branch, and outputting the distance from the sampling point corresponding to the feature map point to the prediction box, the positioning weight and the positioning quality weight according to the characteristics of the regression branch.

[0042] Step 104: Based on the search image, determine a regression sample set according to the initial classification loss of each initial sample, and determine a classification sample set according to the initial regression loss of each initial sample, the regression sample set includes regression samples carrying regression labels, and the classification sample set includes classification samples carrying classification labels.

[0043] After obtaining a training image group and inputting the training image group into a preset neural network to obtain an initial classification loss and an initial regression loss of each initial sample in the search image, further, a regression sample set is determined from each initial sample according to the initial classification loss of each initial sample, and a classification sample set is determined from each initial sample according to the initial regression loss of each initial sample.

[0044] Specifically, regression samples refer to samples of each initial sample for the regression branch, and the regression sample set is the set of regression samples. Classification samples refer to samples of each initial sample for the classification branch, and the classification sample set is the set of classification samples. Regression labels are labels corresponding to regression samples, and each regression sample corresponds to a regression label. Classification labels are labels corresponding to classification samples, and each classification sample corresponds to a classification label.

[0045] In practical applications, after obtaining the initial classification loss and initial regression loss of each initial sample, the initial sample whose initial classification loss is greater than the preset classification loss can be determined as a regression sample, and the initial sample whose initial regression loss is greater than the preset regression loss can be determined as a classification sample. The initial sample whose initial classification loss is greater than the preset regression loss can also be determined as a regression sample, and the initial sample whose initial regression loss is greater than the preset classification loss can be determined as a classification sample. In this way, the efficiency of determining regression samples and classification samples can be improved.

[0046] For example, as shown in Table 1, Table 1 shows the initial classification loss and initial regression loss of five initial samples, assuming that the preset classification loss is 1.1 and the preset regression loss is 1.2. If the initial samples whose initial classification loss is greater than the preset classification loss in each initial sample are determined as regression samples, and the initial samples whose initial regression loss is greater than the preset regression loss in each initial sample are determined as classification samples, then the initial samples A3, A4 and A5 are regression samples, and the initial samples A1 and A2 are classification samples; if the initial samples whose initial classification loss is greater than the preset regression loss in each initial sample are determined as regression samples, and the initial samples whose initial regression loss is greater than the preset classification loss in each initial sample are determined as classification samples, then the initial samples A4 and A5 are regression samples, and the initial samples A1, A2 and A3 are classification samples.

[0047] Table 1 Initial classification loss and initial regression loss of five initial samples

[0048]

[0049]

[0050] Since the classification branch and the regression branch have different sensitivities to the feature space, the classification branch is only sensitive to a specific area, while the regression branch is sensitive to the target as a whole, so the classification branch and the regression branch need to learn different features. During the training process, the classification branch and the localization branch are usually optimized independently, which also leads to the problem of mismatch between classification and localization. In this way, by determining the classification samples for the training of the classification branch through the initial regression loss, the classification task can have a higher classification weight for the high-precision prediction box, effectively ensuring the alignment of the classification and regression results during the training process, thereby improving the robustness and accuracy of the target tracking model.

[0051] In order to improve the robustness of the target tracking model, positive and negative samples can be used to train the preset neural network, that is, it is necessary to determine whether each initial sample is a positive initial sample or a negative initial sample. That is, before determining the regression sample set based on the initial classification loss of each initial sample, it is necessary to determine whether each initial sample is a positive initial sample or a negative initial sample: the initial sample can be identified as a positive initial sample or a negative initial sample based on the feature points and the annotation box in the template image, that is, when the template image includes multiple feature points and the search image includes an annotation box, based on the search image, before determining the regression sample set based on the initial classification loss of each initial sample, it also includes:

[0052] For any one of the plurality of feature points, determining whether a sampling point of an initial sample corresponding to the feature point in the search image falls within the annotation frame;

[0053] If yes, then the initial sample is determined to be a positive initial sample;

[0054] If not, it is determined that the initial sample is a negative initial sample.

[0055] Specifically, feature map points refer to points on the template image, that is, pixel points. For example, when the output feature map size corresponding to the template image is 13x13, there are 169 points in total, representing 169 feature map points. Sampling points are points mapped to the search image from feature map points, which are also pixel points in essence. Annotation boxes refer to boxes pre-set for positioning each sampling point.

[0056] In practical applications, for any feature point in the template image, first determine the initial sample in the search image mapped to the feature point and the sampling point in the initial sample, and then determine whether the sampling point is located in the annotation box. If so, the initial sample is a positive initial sample, and the initial label of the positive initial sample is marked as the first label. If not, the initial sample is a negative initial sample, and the initial label of the negative initial sample is marked as the second label.

[0057] Preferably, the first label is 1 and the second label is 0. As shown in Formula 1, if the sampling point of the initial sample corresponding to the feature map point in the search image falls within the annotation box, the initial sample is determined to carry the first label "1" as a positive initial sample, and if the sampling point of the initial sample corresponding to the feature map point in the search image falls outside the annotation box, the initial sample is determined to carry the second label "0" as a negative initial sample.

[0058]

[0059] In Formula 1, Y is the initial sample; 1 is the label of the positive initial sample, that is, the first label; 0 is the label of the negative initial sample, that is, the second label.

[0060] In order to improve the training efficiency of the regression branch and thus improve the robustness of the target tracking model, when determining the regression samples, the regression samples can be distinguished between positive and negative on the basis that each initial sample has a positive initial sample and a negative initial sample, that is, when each initial sample includes at least one positive initial sample and at least one negative initial sample, the regression sample set is determined based on the search image according to the initial classification loss of each initial sample. The specific implementation process can be as follows:

[0061] Based on the search image, comparing the initial classification loss of each positive initial sample with a first preset threshold value;

[0062] Determine the positive initial sample whose initial classification loss is less than the first preset threshold as a positive regression sample;

[0063] The positive initial samples whose initial classification loss is greater than or equal to the first preset threshold and the negative initial samples are determined as negative regression samples.

[0064] Specifically, the first preset threshold refers to a preset value used to measure the initial classification loss.

[0065] In practical applications, since each initial sample includes at least one positive initial sample and at least one negative initial sample, the initial classification loss of each initial sample can be compared with the first preset threshold, and the positive initial sample whose initial classification loss is less than the first preset threshold is determined as a positive regression sample, and the positive initial sample whose initial classification loss is greater than or equal to the first preset threshold is determined as a negative regression sample, and each negative initial sample is determined as a negative regression sample. In addition, the initial classification loss of each initial sample can be sorted from small to large, and the initial samples corresponding to the top K initial classification losses are determined as positive regression samples, where k is an arbitrary positive integer, and the other positive initial samples and negative initial samples are determined as negative regression samples. And the regression label of each positive regression sample is the third label, and the regression label of each negative regression sample is the fourth label. In this way, the positive regression sample and the negative regression sample used to train the regression branch are determined by the initial classification loss and the first preset threshold, which can not only improve the accuracy and speed of determining the positive regression sample and the negative regression sample, but also make the regression branch pay more attention to the position with high classification weight, so that the classification branch and the regression branch are quickly aligned, thereby improving the robustness of the target tracking model and the target tracking efficiency.

[0066] To facilitate training, preferably, the third label is set to 1 and the fourth label is set to 0.

[0067] In order to improve the training efficiency of the classification branch and thus improve the robustness of the target tracking model, when determining the classification samples, the classification samples can be distinguished between positive and negative on the basis that each initial sample has a positive initial sample and a negative initial sample, that is, when each initial sample includes at least one positive initial sample and at least one negative initial sample, the classification sample set is determined based on the search image according to the initial regression loss of each initial sample. The specific implementation process can be as follows:

[0068] Based on the search image, comparing the initial regression loss of each positive initial sample with a second preset threshold value respectively;

[0069] Determine the positive initial sample whose initial regression loss is less than the second preset threshold as a positive classification sample;

[0070] The positive initial samples whose initial regression loss is greater than or equal to the second preset threshold and the negative initial samples are determined as negative classification samples.

[0071] Specifically, the second preset threshold refers to a preset value used to measure the initial regression loss.

[0072] In practical applications, since each initial sample includes at least one positive initial sample and at least one negative initial sample, the initial regression loss of each initial sample can be compared with the second preset threshold, and the positive initial sample whose initial regression loss is less than the first preset threshold is determined as a positive classification sample, the positive initial sample whose initial regression loss is greater than or equal to the first preset threshold is determined as a negative classification sample, and each negative initial sample is determined as a negative classification sample. In addition, the initial regression losses of each initial sample can be sorted from small to large, and the initial samples corresponding to the top K initial regression losses are determined as positive classification samples, where k is an arbitrary positive integer, and the other positive initial samples and negative initial samples are determined as negative classification samples. And the classification label of each positive classification sample is the fifth label, and the classification label of each negative classification sample is the sixth label. In this way, the positive classification samples and negative classification samples used to train the classification branch are determined through the initial regression loss and the second preset threshold, which can not only improve the accuracy and speed of determining the positive classification samples and the negative classification samples, but also enable the classification branch to have a higher classification weight for the high-precision prediction box, so that the classification branch and the regression branch are quickly aligned, thereby improving the robustness of the target tracking model and the target tracking efficiency.

[0073] To facilitate training, preferably, the third label is set to 1 and the fourth label is set to 0.

[0074] It should be noted that both regression samples and classification samples are derived from initial samples, but the labels or sample identifiers of regression samples and classification samples are different. For example, the initial sample a is a positive sample when used as a regression sample, and a negative sample when used as a classification sample, that is, the sample identifiers are different. For example, the initial sample b carries a regression label of 1 when used as a regression sample, and carries a classification label of 2 when used as a classification sample, that is, the labels are different.

[0075] Step 106: Calculate the total training loss based on the classification label of each classification sample and the classification prediction result of the classification branch, as well as the regression label of each regression sample and the positioning prediction result of the regression branch.

[0076] Based on determining the regression sample set from each initial sample according to the initial classification loss of each initial sample, and determining the classification sample set from each initial sample according to the initial regression loss of each initial sample, further, according to the classification label of each classification sample and the classification prediction result of the classification branch, as well as the regression label of each regression sample and the positioning prediction result of the regression branch, the total training loss is calculated.

[0077] Specifically, the classification label refers to the label carried by the classification sample; the classification prediction result is the output of the classification branch for the classification sample; the regression label refers to the label carried by the regression sample; the positioning prediction result is the output of the regression branch for the regression sample; the total training loss is used to evaluate the difference between the initial prediction ratio relationship of the preset neural network and the ideal value.

[0078] In practical applications, the classification loss of the classification branch can be re-determined based on the classification label of each classification sample and the classification prediction result of the classification branch, that is, the updated classification loss is obtained, and then the regression loss of the regression branch can be re-determined based on the regression label of each regression sample and the positioning prediction result of the regression branch, that is, the updated regression loss is obtained, and then the total training loss is calculated based on the updated classification loss and the updated regression loss. That is, according to the classification label of each classification sample and the classification prediction result of the classification branch, as well as the regression label of each regression sample and the positioning prediction result of the regression branch, the specific implementation process of calculating the total training loss is as follows:

[0079] Based on the classification labels of the classification samples and the classification prediction results of the classification branches, an updated classification loss is obtained; based on the regression labels of the regression samples and the positioning prediction results of the regression branches, an updated regression loss is obtained;

[0080] Based on the updated classification loss and the updated regression loss, the total training loss is calculated.

[0081] In the specific implementation, for any one of the classification samples, the classification label of the classification sample is compared with the classification prediction result of the classification branch for the classification branch, so as to determine the sub-update classification loss for the classification sample. After traversing all the classification samples, the total update classification loss is determined according to the sub-update classification loss of each classification sample. Similarly, for any one of the regression samples, the regression label of the regression sample is compared with the positioning prediction result of the regression branch for the regression branch, so as to determine the sub-update regression loss for the regression sample. After traversing all the regression samples, the total update regression loss is determined according to the sub-update regression loss of each regression sample. Then, the total training loss is calculated according to the update classification loss and the update regression loss. In this way, the total training loss can be determined in a more fine-grained manner, and the correlation between the total training loss and the classification branch and the regression branch is improved, which is conducive to improving the efficiency of adjusting the network parameters based on the total training loss, so that the preset neural network converges quickly, and improves the robustness of the target tracking model while improving the model training speed.

[0082] In order to determine the total training loss in a more fine-grained manner and enable the preset neural network to converge quickly, the updated classification loss may include classification loss and center quality loss, and the updated regression loss may include positioning loss and positioning quality loss. Then, the classification loss and center quality loss are determined based on the classification label of each classification sample and the classification prediction result of the classification branch, and the positioning loss and positioning quality loss are determined based on the regression label of each regression sample and the positioning prediction result of the regression branch. That is, in the case where the updated classification loss includes classification loss and center quality loss, and the updated regression loss includes positioning loss and positioning quality loss, the updated classification loss is obtained based on the classification label of each classification sample and the classification prediction result of the classification branch, and the updated regression loss is obtained based on the regression label of each regression sample and the positioning prediction result of the regression branch. The specific implementation process can be as follows:

[0083] Calculating classification loss based on the classification labels of the classification samples and the classification weights in the classification prediction results of the classification branches;

[0084] Based on each classification label, determine the true value of the center mass of each classification sample, and calculate the center mass loss according to the true value of the center mass of each classification sample and the center mass weight in the classification prediction result of the classification branch;

[0085] Calculating positioning loss based on the regression label of each regression sample and the prediction box in the positioning prediction result of the regression branch;

[0086] Based on each regression label, a true value frame of a positive regression sample in each regression sample is determined, and the positioning quality loss is calculated according to the true value frame of the positive regression sample and the positioning weight in the positioning prediction result of the regression branch.

[0087] Specifically, the center mass true value refers to the actual center degree of the classified sample, that is, the numerical value corresponding to the center mass; the true value box refers to the real bounding box corresponding to the positive regression sample, which can be carried in the regression label of the regression sample, or each regression label corresponds to a true value box.

[0088] In practical applications, for any one of the classification samples, the classification label of the classification sample is compared with the classification weight in the classification prediction result of the classification branch for the classification branch, so as to determine the sub-classification loss for the classification sample; after traversing all the classification samples, the total classification loss is determined according to the sub-classification loss of each classification sample. Similarly, for any one of the classification samples, the center quality true value of the regression sample is determined according to the classification label of the classification sample, and the center quality true value is compared with the center quality weight in the classification prediction result of the classification branch for the classification branch, so as to determine the sub-center quality loss for the classification sample; after traversing all the classification samples, the total center quality loss is determined according to the sub-center quality loss of each classification sample. For any one of the regression samples, the regression label of the regression sample is compared with the prediction box in the positioning prediction result of the regression branch for the regression branch, so as to determine the sub-positioning loss for the regression sample; after traversing all the regression samples, the total positioning loss is determined according to the sub-positioning loss of each regression sample. For any positive regression sample in each regression sample, determine the regression label of the positive regression sample, determine the true value box of the positive regression sample, and then compare the true value box of the positive regression sample with the positioning weight corresponding to the positive regression sample in the positioning prediction result of the regression branch, so as to determine the sub-positioning quality loss for the positive regression sample; after traversing all positive regression samples, determine the total positioning quality loss according to the sub-positioning quality loss of each positive regression sample. Then calculate the total training loss according to the classification loss, center quality loss, positioning loss and positioning quality loss. In this way, the total training loss can be determined in a more fine-grained manner, and the correlation between the total training loss and the classification and classification quality in the classification branch, and the positioning and positioning quality in the regression branch can be improved, which is conducive to improving the efficiency of adjusting network parameters based on the total training loss, so that the preset neural network converges quickly, and improves the robustness of the target tracking model while improving the model training speed.

[0089] It should be noted that there are many loss functions for calculating classification loss, center quality loss, positioning loss and positioning quality loss, such as cross entropy loss function (Cross Entropy Loss), L1 norm loss function, maximum loss function, mean square error loss function, logarithmic loss function, etc., which are not limited in this application. Preferably, the classification loss, center quality loss and positioning quality loss are calculated by the cross entropy loss function, and the positioning loss is calculated by the evaluation index loss function (IoU, Intersection over Union). And because only relying on the classification weights to filter the prediction boxes will filter out some prediction boxes with good regression effects but small classification weights, there is a lack of quality supervision on the positioning of the regression branch. Even if the prediction box has a higher classification weight, there will still be errors in the regression branch. In order to improve the accuracy of screening high-quality prediction boxes, positioning quality input is added.

[0090] In addition, the classification loss, center quality loss, localization loss and localization quality loss can be weighted summed to calculate the total training loss, as shown in Equation 2.

[0091] L=λ1S1+λ2S2+λ3S3+λ4S4 (Formula 2)

[0092] In Formula 2, L is the total training loss; S1, S2, S3, and S4 are the classification loss, center quality loss, localization loss, and localization quality loss, respectively; λ1, λ2, λ3, and λ4 are the weights of classification loss, center quality loss, localization loss, and localization quality loss, respectively.

[0093] Step 108: Based on the total training loss, adjust the network parameters of the preset neural network and return to step 102.

[0094] After determining the total training loss, the network parameters of the preset neural network are further adjusted according to the total training loss, and then the training image group is continuously acquired to start the next round of training.

[0095] In practical applications, when adjusting the network parameters of a preset neural network, the total training loss can be back-propagated to sequentially update the network parameters of the output unit, the network parameters of the feature enhancement unit, the network parameters of the calculation unit, the network parameters of the feature extraction unit, and the network parameters of the input unit in the preset neural network. After adjusting all the network parameters, continue to execute step 102, that is, reacquire the training samples and start the next round of training.

[0096] See also Figure 3 , Figure 3A flow chart of a target tracking model training method provided by an embodiment of the present application is shown: first, a regression sample set is determined according to the initial classification loss of each initial sample for the classification branch, and a classification sample set is determined according to the initial regression loss of each initial sample for the regression branch; then, an updated classification loss of the classification branch is determined according to the classification prediction result output by the classification branch and the classification sample set, and an updated regression loss of the regression branch is determined according to the positioning prediction result output by the regression branch and the regression sample set; further, a total training loss is calculated based on the updated classification loss and the updated regression loss, and the total training loss is back-propagated to sequentially update the network parameters of the preset neural network.

[0097] Step 110: When a preset stop condition is reached, stop training to obtain a target tracking model.

[0098] In one or more implementations of this embodiment, the specific implementation process of stopping training when the preset stop condition is reached is: when the total training loss is less than the target value, stop training the preset neural network; when the total training loss stops decreasing, stop training the preset neural network; or when the number of iterations of the preset neural network reaches the target number of iterations, stop training the preset neural network.

[0099] In this embodiment, a target value is set in advance. When the total training loss of the preset neural network is lower than the preset target value, or when the total training loss stops decreasing, or when the number of iterations of the preset neural network reaches the target number of iterations, it means that the preset neural network has reached a certain target tracking level, that is, the training stop condition is reached. Therefore, the training of the preset neural network is stopped to obtain a trained target tracking model.

[0100] The target tracking model training method provided by the present application uses a cross-guided training method of initial classification loss and initial regression loss to enable the classification branch and the regression branch to interact better during training, that is, by using the initial classification loss to determine the regression sample for the training of the regression branch, the regression branch can pay more attention to the position with high classification weight; by using the initial regression loss to determine the classification sample for the training of the classification branch, the classification task can have a higher classification weight for the high-precision prediction box, effectively ensuring the alignment of the classification and regression results during the training process, thereby improving the robustness and accuracy of the target tracking model. Furthermore, using the target tracking model to track the image to be tracked can effectively improve the efficiency and accuracy of using the target tracking model to track the target.

[0101] Figure 4 A flow chart of a target tracking method provided according to an embodiment of the present application is shown, which specifically includes the following steps:

[0102] Step 402: Acquire an image to be tracked.

[0103] Step 404: input the image to be tracked into the trained target tracking model to obtain the target tracking result of the image to be tracked; wherein the target tracking model includes a classification branch and a regression branch, and the target tracking model is obtained by training a preset neural network based on regression samples and classification samples, and the regression samples are determined according to the initial classification loss of the initial samples obtained by the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained by the regression branch.

[0104] Specifically, the image to be tracked refers to an image for which the target needs to be determined; the target tracking model is obtained by training a preset neural network based on regression samples and classification samples; the preset neural network refers to a neural network with a preset deep twin network architecture; the class branch is used to implement the classification task, that is, the recognition of the target, and the regression branch is used to implement the position regression, that is, the positioning of the target; the initial classification loss refers to the loss value directly obtained by the classification branch for the classification task for the initial sample; the initial regression loss refers to the loss value directly obtained by the regression branch for the positioning task for the initial sample. The initial sample refers to the image area centered on a certain sampling point in the search image, and the search image includes at least one sampling point, and the sampling point corresponds to the initial sample one by one.

[0105] In practical applications, the image to be tracked can be obtained first, and then the image to be tracked is input into the target tracking model, and then the target tracking model outputs the target tracking result of the image to be tracked, that is, the target in the image to be tracked is located using the target box in the image to be tracked.

[0106] It should be noted that the target tracking model is trained by the above target tracking model training method.

[0107] After obtaining the trained target tracking model, that is, before inputting the image to be tracked into the trained target tracking model, the target tracking model needs to be tested. A test image can be obtained and input into the target tracking model for testing. The specific implementation process can be as follows:

[0108] Acquire a test image, input the test image into the target tracking model, and obtain a classification prediction result and a positioning prediction result of the target in the test image, wherein the classification prediction result includes a classification weight and a center quality weight, and the positioning prediction result includes a prediction box and a positioning quality weight of the prediction box;

[0109] Calculating the prediction weight of each prediction box according to the classification weight, the center quality weight and the positioning quality weight;

[0110] According to the prediction weights of the prediction boxes, a target box is determined from the prediction boxes.

[0111] Specifically, the test image refers to an image used to test the target tracking model; the prediction weight refers to the score obtained after the target tracking model evaluates a certain prediction box; the classification weight represents the type of the target; and the target box is the box used to locate the target that is finally output by the regression branch of the target tracking model.

[0112] In practical applications, when the training phase is completed and the target tracking model is obtained, the test phase is entered. A test image can be extracted from the test set, and the test image can be input into the target tracking model. The target tracking model processes the test image accordingly, and then the classification branch of the target tracking model outputs the classification weights and center quality weights corresponding to each target in the test image, and the regression branch outputs the prediction box corresponding to each target and the positioning quality weight of each prediction box. On this basis, for any prediction box in each prediction box, the prediction weight of the prediction box is calculated based on the classification weight, center quality weight and positioning quality weight corresponding to the prediction box. Further, based on the prediction weights of each prediction box, a prediction box is selected from each prediction box as the target box, and preferably, the prediction box with the highest prediction weight is determined as the target box. In this way, in the test phase, the prediction box with the highest overall score can be screened out as the target box by comprehensive positioning quality scores.

[0113] Optionally, when calculating the prediction weight of the prediction box based on the classification weight, center quality weight and positioning quality weight corresponding to the prediction box, the weighted sum of the classification weight, center quality weight and positioning quality weight can be used as the prediction weight of the prediction box, and the product of the classification weight, center quality weight and positioning quality weight can be used as the prediction weight of the prediction box.

[0114] Preferably, the product of the classification weight, the center quality weight and the positioning quality weight can be used as the prediction weight of the prediction box, as shown in Formula 3.

[0115] A=a1*a2*a3 (Formula 3)

[0116] In Formula 3, A is the prediction weight, a1, a2, and a3 are the classification weight, center quality weight, and positioning quality weight, respectively.

[0117] When the test image is input into the target tracking model to obtain the classification prediction result and the positioning prediction result of the target in the test image, in order to ensure the accuracy of the positioning prediction result, it is necessary to improve the correct output of the positioning prediction result of the regression branch. Therefore, the positioning quality weight in the positioning prediction result of the regression branch can be calculated according to the prediction box in the positioning prediction result and the true value box corresponding to the target. That is, the specific implementation process of inputting the test image into the target tracking model to obtain the classification prediction result and the positioning prediction result of the target in the test image is as follows:

[0118] Inputting the test image into the classification branch of the target tracking model to obtain the classification weight and center quality weight of the target in the test image;

[0119] Inputting the test image into the regression branch of the target tracking model to obtain a predicted box of the target in the test image;

[0120] Calculating a positioning quality error according to a true value frame of the target and the prediction frame in a positioning prediction result of the target;

[0121] According to the positioning quality error, a positioning quality weight of the prediction box in the positioning prediction result is determined.

[0122] Specifically, the true value box refers to the real bounding box corresponding to the positive regression sample, which can be carried in the regression label of the regression sample, or each regression label corresponds to a true value box.

[0123] In practical applications, after the test image is input into the target tracking model, the classification branch and regression branch of the target tracking model process the test image respectively, and the classification branch outputs the classification prediction result of the target in the test image, that is, the classification weight and center quality weight of the target; the regression branch first outputs the prediction box of the target, and then introduces a boundary quality evaluation mechanism to evaluate the prediction quality of the prediction box, so as to obtain the positioning quality weight: the regression branch outputs the prediction box of the target, that is, the prediction box corresponding to the target, and evaluates the error between the prediction box of each target and the true value box of the target to obtain the positioning quality error, and calculates the positioning quality weight according to the positioning quality error, and the regression branch outputs the positioning quality weight, that is, the positioning quality weight is part of the positioning prediction result. In this way, by comparing the prediction box with the true value box to determine the positioning quality error, the authenticity of the positioning quality error can be improved, and then the accuracy of the positioning quality weight determined based on the positioning quality error can be improved.

[0124] In one or more optional embodiments of the present specification, in order to improve the efficiency of determining the positioning quality error, the positioning quality error can be calculated according to the margins between the specified sampling point in the target and the prediction box and the true value box. That is, the positioning quality error is calculated according to the true value box of the target and the prediction box in the positioning prediction result of the target. The specific implementation process is:

[0125] Determine first margins from the specified sampling point in the target to four boundaries of the true value box, and second margins from the specified sampling point to four boundaries of the prediction box;

[0126] A positioning quality error is determined according to the first margin and the second margin.

[0127] In practical applications, the sub-margins of the distance between the specified sampling point in the target and each boundary of the truth box can be determined first, and then the first margin can be determined based on each sub-margin. At the same time, the sub-margins of the distance between the specified sampling point in the target and each boundary of the truth box can be determined, and then the second margin can be determined. It should be noted that the weighted sum of each sub-margin can be used to determine the first margin or the second margin, and the set of each sub-margin can be used as the first margin or the second margin. Furthermore, the first margin is compared with the second margin, and the positioning quality error is determined according to the difference between the two.

[0128] It should be noted that the first margin may be determined first and then the second margin; the second margin may be determined first and then the first margin; or the first margin and the second margin may be determined at the same time, which is not limited in the present application.

[0129] According to the error between the margins from the specified sampling point to the prediction box and the true value box, that is, according to the difference between the first margin and the second margin, a quality assessment is performed, that is, the positioning quality error is determined, and the positioning quality weight in the positioning prediction result is determined in a finer granularity, thereby improving the accuracy of the positioning quality weight, thereby improving the precision of the prediction weight, so that the target tracking model can determine the target box faster and more accurately, thereby improving the robustness of the target tracking model.

[0130] Preferably, a set of each sub-margin can be used as the first margin or the second margin, that is, determining the first margin from the specified sampling point in the target to the four boundaries of the true value box, and the second margin from the specified sampling point to the four boundaries of the prediction box. The specific implementation process can be as follows:

[0131] Respectively determine a first left margin from a specified sampling point in the target to a left boundary of the truth box, a first upper margin from an upper boundary of the truth box, a first right margin from a right boundary of the truth box, and a first lower margin from a lower boundary of the truth box, wherein the first margin includes the first left margin, the first upper margin, the first right margin, and the first lower margin;

[0132] Determine respectively a second left margin of the specified sampling point from the left boundary of the prediction box, a second upper margin from the upper boundary of the prediction box, a second right margin from the right boundary of the prediction box, and a second lower margin from the lower boundary of the prediction box, wherein the second margin includes the second left margin, the second upper margin, the second right margin, and the second lower margin.

[0133] In practical applications, the positioning output in the regression branch can determine the margins from the specified sampling point to the left boundary, upper boundary, right boundary and lower boundary of the true value box, that is, the first left margin, the first upper margin, the first right margin and the first lower margin; at the same time, the margins from the specified sampling point to the left boundary, upper boundary, right boundary and lower boundary of the prediction box, that is, the second left margin, the second upper margin, the second right margin and the second lower margin, are determined. In this way, by specifying the margins from the sampling point to the left boundary, upper boundary, right boundary and lower boundary of the true value box and the prediction box, the first margin and the second margin are determined, and the first margin and the second margin can also be made more accurate, and at the same time, the accuracy of the positioning quality error determined based on the first margin and the second margin is improved, and the positioning quality weight in the positioning prediction result is determined in a finer granularity, and the accuracy of the positioning quality weight is improved, thereby improving the accuracy of the prediction weight, so that the target tracking model can determine the target box faster and more accurately, thereby improving the robustness of the target tracking model.

[0134] Accordingly, the positioning quality error may be determined based on the first left margin and the second left margin, the first upper margin and the second upper margin, the first right margin and the second right margin, and the first lower margin and the second lower margin. That is, in the case where the positioning quality error includes the left boundary positioning quality error, the upper boundary positioning quality error, the right boundary positioning quality error, and the lower boundary positioning quality error, the specific implementation process of determining the positioning quality error according to the first margin and the second margin may be as follows:

[0135] Determining a left boundary positioning quality error according to the first left margin and the second left margin;

[0136] Determining an upper boundary positioning quality error according to the first upper margin and the second upper margin;

[0137] Determining a right boundary positioning quality error according to the first right margin and the second right margin;

[0138] A lower boundary positioning quality error is determined according to the first lower margin and the second lower margin.

[0139] In practical applications, the first left margin is compared with the second left margin to determine the left boundary positioning quality error; the first upper margin is compared with the second upper margin to determine the upper boundary positioning quality error; the first right margin is compared with the second right margin to determine the right boundary positioning quality error; the first lower margin is compared with the second lower margin to determine the lower boundary positioning quality error. In this way, by specifying the margins of the sampling points to the left boundary of the true value box and the prediction box to determine the left boundary positioning quality error, by specifying the margins of the sampling points to the right boundary of the true value box and the prediction box to determine the right boundary positioning quality error, by specifying the margins of the sampling points to the upper boundary of the true value box and the prediction box to determine the upper boundary positioning quality error, by specifying the margins of the sampling points to the lower boundary of the true value box and the prediction box to determine the lower boundary positioning quality error, the accuracy of the positioning quality error can be improved, thereby improving the accuracy of the prediction weight, so that the target tracking model can determine the target box faster and more accurately, thereby improving the robustness of the target tracking model.

[0140] The normalized error can be calculated using the formula shown in Formula 4, that is, the left boundary positioning quality error, the upper boundary positioning quality error, the right boundary positioning quality error, and the lower boundary positioning quality error are calculated respectively.

[0141]

[0142] In Formula 4, Z, S, Y, and X represent the left boundary positioning quality error, the upper boundary positioning quality error, the right boundary positioning quality error, and the lower boundary positioning quality error, respectively; z1 and z2 represent the first left margin and the second left margin, respectively; s1 and s2 represent the first upper margin and the second upper margin, respectively; y1 and y2 represent the first right margin and the second right margin, respectively; x1 and x2 represent the first lower margin and the second lower margin, respectively.

[0143] Preferably, when the positioning quality error includes a left boundary positioning quality error, an upper boundary positioning quality error, a right boundary positioning quality error, and a lower boundary positioning quality error, the positioning quality weight of the prediction box in the positioning prediction result is determined according to the positioning quality error, and the specific implementation process may be as follows:

[0144] Determining a left boundary positioning quality weight according to the left boundary positioning quality error;

[0145] Determining an upper boundary positioning quality weight according to the upper boundary positioning quality error;

[0146] Determining a right boundary positioning quality weight according to the right boundary positioning quality error;

[0147] Determining a lower boundary positioning quality weight according to the lower boundary positioning quality error;

[0148] The positioning quality weight of the prediction box in the positioning prediction result is determined according to the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight and the lower boundary positioning quality weight.

[0149] In practical applications, when determining the true value of the positioning quality weight, the left boundary positioning quality weight can be first calculated based on the left boundary positioning quality error, the upper boundary positioning quality weight can be calculated based on the upper boundary positioning quality error, the right boundary positioning quality weight can be calculated based on the right boundary positioning quality error, and the lower boundary positioning quality weight can be calculated based on the lower boundary positioning quality error. Then, the weighted sum of the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight, and the lower boundary positioning quality weight is calculated to determine the positioning quality weight in the positioning prediction result of the regression branch. Preferably, the positioning quality weight is the average of the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight, and the lower boundary positioning quality weight. In this way, the accuracy of the positioning quality weight can be improved, thereby improving the accuracy of the prediction weight, so that the target tracking model can determine the target frame faster and more accurately, thereby improving the robustness of the target tracking model.

[0150] Referring to Formula 5, the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight, and the lower boundary positioning quality weight may be calculated respectively using the formula shown in Formula 5.

[0151]

[0152] In formula 5, δ(z), δ(s), δ(y) and δ(x) represent the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight and the lower boundary positioning quality weight respectively. z1 and z2 represent the first left margin and the second left margin respectively; s1 and s2 represent the first upper margin and the second upper margin respectively; y1 and y2 represent the first right margin and the second right margin respectively; x1 and x2 represent the first lower margin and the second lower margin respectively; They respectively represent the left boundary positioning quality error, the upper boundary positioning quality error, the right boundary positioning quality error and the lower boundary positioning quality error;.

[0153] It should be noted that since the four boundaries of the prediction box cannot be accurately aligned with the four boundaries of the true value box at the same time, the effect of filtering the prediction box as a whole by relying solely on the classification weight is poor. Therefore, the four boundaries of the prediction box can be evaluated separately, and each boundary of the prediction box can be scored according to the error between the actual boundary and the predicted boundary. The larger the error, the lower the score. If the error is greater than or equal to the actual boundary value, the score is 0.

[0154] The target tracking method provided by the present application obtains the image to be tracked; then the image to be tracked is input into the trained target tracking model to obtain the target tracking result of the image to be tracked, wherein the target tracking model includes a classification branch and a regression branch, the target tracking model is obtained by training the preset neural network based on the regression sample and the classification sample, the regression sample is determined according to the initial classification loss of the initial sample obtained by the classification branch, and the classification sample is determined according to the initial regression loss of the initial sample obtained by the regression branch. That is, the target tracking provided by the present application first trains the target tracking model including the classification branch and the regression branch: through the cross-guidance training method of the initial classification loss and the initial regression loss, the classification branch and the regression branch interact better during the training, that is, the regression sample is determined by the initial classification loss for the training of the regression branch, so that the regression branch can pay more attention to the position with high classification weight; the classification sample is determined by the initial regression loss for the training of the classification branch, so that the classification task has a higher classification weight for the high-precision prediction box, which effectively ensures the alignment of the classification and regression results during the training process, thereby improving the robustness and accuracy of the target tracking model. Then, the target tracking model is used to track the image to be tracked, effectively improving the efficiency and accuracy of target tracking using the target tracking model.

[0155] The following combination Figure 5 , the target tracking method is further described. Figure 5 A processing flow chart of a target tracking method provided by an embodiment of the present application is shown, which specifically includes the following steps:

[0156] Step 502: Obtain a training image group.

[0157] The preset neural network includes a classification branch and a regression branch, the training image group includes a template image and a search image, the template image includes a plurality of feature points, and the search image includes a labeling box.

[0158] Step 504: Input the training image group into the preset neural network to obtain the initial classification loss and initial regression loss of each initial sample in the search image.

[0159] Step 506: for any feature graph point, determine whether the sampling point of the initial sample corresponding to the feature graph point falls within the marked box.

[0160] If yes, execute step 508; if no, execute step 510.

[0161] Step 508: Determine whether the initial sample is a positive initial sample.

[0162] Step 510: Determine whether the initial sample is a negative initial sample.

[0163] Step 512: Based on the search image, compare the initial classification loss of each positive initial sample with a first preset threshold.

[0164] Step 514: Determine the positive initial samples whose initial classification loss is less than the first preset threshold as positive regression samples.

[0165] Step 516: Determine the positive initial samples whose initial classification loss is greater than or equal to the first preset threshold and the negative initial samples as negative regression samples.

[0166] Step 518: Based on the search image, compare the initial regression loss of each positive initial sample with a second preset threshold.

[0167] Step 520: Determine the positive initial samples whose initial regression loss is less than the second preset threshold as positive classification samples.

[0168] Step 522: Determine the positive initial samples whose initial regression loss is greater than or equal to the second preset threshold and the negative initial samples as negative classification samples.

[0169] Step 524: Calculate the classification loss based on the classification label of each classification sample and the classification weight in the classification prediction result of the classification branch.

[0170] Step 526: Based on each classification label, determine the true value of the center mass of each classification sample, and calculate the center mass loss according to the true value of the center mass of each classification sample and the center mass weight in the classification prediction result of the classification branch.

[0171] Step 528: Calculate the positioning loss based on the regression label of each regression sample and the prediction box in the positioning prediction result of the regression branch.

[0172] Step 530: Based on each regression label, determine the true value box of the positive regression sample in each regression sample, and calculate the positioning quality loss according to the true value box of the positive regression sample and the positioning weight in the positioning prediction result of the regression branch.

[0173] Step 532: Calculate the total training loss based on the classification loss, center quality loss, positioning loss and positioning quality loss.

[0174] Step 534: Determine whether a preset stop condition is reached.

[0175] If not, execute step 536; if so, execute step 538.

[0176] Step 536: Based on the total training loss, adjust the network parameters of the preset neural network.

[0177] Afterwards, return to execute step 502.

[0178] Step 538: Stop training and obtain the target tracking model.

[0179] Step 540: Acquire a test image.

[0180] Step 542: input the test image into the classification branch of the target tracking model to obtain the classification weight and center quality weight of the target in the test image;

[0181] Step 544: Input the test image into the regression branch of the target tracking model to obtain the predicted box of the target in the test image.

[0182] Step 546: Determine the first margins from the specified sampling point in the target to the four boundaries of the true value box, and the second margins from the specified sampling point to the four boundaries of the prediction box.

[0183] Optionally, determining first margins from a specified sampling point in the target to four boundaries of the true value box, and second margins from the specified sampling point to four boundaries of the prediction box, includes:

[0184] Respectively determine a first left margin from a specified sampling point in the target to a left boundary of the truth box, a first upper margin from an upper boundary of the truth box, a first right margin from a right boundary of the truth box, and a first lower margin from a lower boundary of the truth box, wherein the first margin includes the first left margin, the first upper margin, the first right margin, and the first lower margin;

[0185] Determine respectively a second left margin of the specified sampling point from the left boundary of the prediction box, a second upper margin from the upper boundary of the prediction box, a second right margin from the right boundary of the prediction box, and a second lower margin from the lower boundary of the prediction box, wherein the second margin includes the second left margin, the second upper margin, the second right margin, and the second lower margin.

[0186] Step 548: Determine the positioning quality error based on the first margin and the second margin.

[0187] Optionally, determining the positioning quality error according to the first margin and the second margin includes:

[0188] Determine a left boundary positioning quality error according to the first left margin and the second left margin;

[0189] Determine the upper boundary positioning quality error according to the first upper margin and the second upper margin;

[0190] Determine a right boundary positioning quality error according to the first right margin and the second right margin;

[0191] The lower boundary positioning quality error is determined based on the first lower margin and the second lower margin.

[0192] Step 550: Determine the positioning quality weight of the prediction box according to the positioning quality error.

[0193] Optionally, determining a positioning quality weight in a positioning prediction result of a regression branch according to the positioning quality error includes:

[0194] According to the left boundary positioning quality error, determine the left boundary positioning quality weight;

[0195] According to the upper boundary positioning quality error, determine the upper boundary positioning quality weight;

[0196] According to the right boundary positioning quality error, determine the right boundary positioning quality weight;

[0197] According to the lower boundary positioning quality error, determine the lower boundary positioning quality weight;

[0198] The positioning quality weight of the prediction box is determined according to the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight and the lower boundary positioning quality weight.

[0199] Step 552: Calculate the prediction weight of each prediction box according to the classification weight, the center quality weight and the positioning quality weight.

[0200] Step 554: Determine the target box from each prediction box according to the prediction weight of each prediction box.

[0201] Step 556: Acquire the image to be tracked.

[0202] Step 558: Obtain the target tracking result of the image to be tracked.

[0203] The target tracking method provided by the present application obtains an image to be tracked; then the image to be tracked is input into a trained target tracking model to obtain a target tracking result of the image to be tracked, wherein the target tracking model includes a classification branch and a regression branch, the target tracking model is obtained by training a preset neural network based on regression samples and classification samples, the regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch. That is, the target tracking method provided by the present application first trains a target tracking model including a classification branch and a regression branch: through the cross-guided training method of the initial classification loss and the initial regression loss, the classification branch and the regression branch interact better during training, that is, by determining the regression sample for the training of the regression branch through the initial classification loss, the regression branch can pay more attention to the position with high classification weight; by determining the classification sample for the training of the classification branch through the initial regression loss, the classification task can have a higher classification weight for the high-precision prediction box, which effectively ensures the alignment of the classification and regression results during the training process, thereby improving the robustness and accuracy of the target tracking model. Then, the target tracking model is used to track the target in the image to be tracked, which effectively improves the efficiency and accuracy of target tracking using the target tracking model.

[0204] Corresponding to the above method embodiment, the present application also provides a target tracking device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of a target tracking device provided by an embodiment of the present application. Figure 6 As shown, the device comprises:

[0205] An acquisition module 602 is configured to acquire an image to be tracked;

[0206] The tracking module 604 is configured to input the image to be tracked into a trained target tracking model to obtain a target tracking result of the image to be tracked;

[0207] Among them, the target tracking model includes a classification branch and a regression branch. The target tracking model is obtained by training a preset neural network based on regression samples and classification samples. The regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch.

[0208] Optionally, in Figure 6 Based on the target tracking device shown in FIG. , the device also includes a target tracking model training module, such as Figure 7 As shown, Figure 7 A schematic diagram of the structure of a target tracking model training module provided in an embodiment of the present application is shown, and the target tracking model training module includes:

[0209] An acquisition submodule 702 is configured to acquire a training image group, input the training image group into a preset neural network, and obtain an initial classification loss and an initial regression loss of each initial sample in the search image, wherein the preset neural network includes a classification branch and a regression branch, and the training image group includes a template image and a search image;

[0210] A determination submodule 704 is configured to determine, based on the search image, a regression sample set according to an initial classification loss of each initial sample, and determine a classification sample set according to the initial regression loss of each initial sample, wherein the regression sample set includes regression samples carrying regression labels, and the classification sample set includes classification samples carrying classification labels;

[0211] A calculation submodule 706 is configured to calculate a total training loss according to the classification label of each classification sample and the classification prediction result of the classification branch, and the regression label of each regression sample and the positioning prediction result of the regression branch;

[0212] An adjustment submodule 708 is configured to adjust the network parameters of the preset neural network based on the total training loss, and return to execute the step of obtaining the training image group;

[0213] The training submodule 710 is configured to stop training when a preset stopping condition is reached to obtain a target tracking model.

[0214] Optionally, the calculation submodule 706 is further configured to:

[0215] Based on the classification labels of the classification samples and the classification prediction results of the classification branches, an updated classification loss is obtained; based on the regression labels of the regression samples and the positioning prediction results of the regression branches, an updated regression loss is obtained;

[0216] Based on the updated classification loss and the updated regression loss, the total training loss is calculated.

[0217] Optionally, the updated classification loss includes classification loss and center quality loss, and the updated regression loss includes positioning loss and positioning quality loss;

[0218] The calculation submodule 706 is further configured to:

[0219] Calculating classification loss based on the classification labels of the classification samples and the classification weights in the classification prediction results of the classification branches;

[0220] Calculating the center quality loss based on the classification labels of the classification samples and the center quality weights in the classification prediction results of the classification branches;

[0221] Calculating the positioning loss based on the regression label of each regression sample and the positioning weight in the positioning prediction result of the regression branch;

[0222] Based on each regression label, a true value frame of a positive regression sample in each regression sample is determined, and the positioning quality loss is calculated according to the true value frame of the positive regression sample and the prediction frame in the positioning prediction result of the regression branch.

[0223] Optionally, the calculation submodule 706 is further configured to:

[0224] Determine first margins from the specified sampling points in the positive regression sample to the four boundaries of the true value box, and second margins from the specified sampling points to the four boundaries of the prediction box;

[0225] A positioning quality loss is determined according to the first margin and the second margin.

[0226] Optionally, the calculation submodule 706 is further configured to:

[0227] Respectively determine a first left margin from a specified sampling point in the positive regression sample to a left boundary of the true value box, a first upper margin from an upper boundary of the true value box, a first right margin from a right boundary of the true value box, and a first lower margin from a lower boundary of the true value box, wherein the first margin includes the first left margin, the first upper margin, the first right margin, and the first lower margin;

[0228] Determine respectively a second left margin of the specified sampling point from the left boundary of the prediction box, a second upper margin from the upper boundary of the prediction box, a second right margin from the right boundary of the prediction box, and a second lower margin from the lower boundary of the prediction box, wherein the second margin includes the second left margin, the second upper margin, the second right margin, and the second lower margin.

[0229] Optionally, the positioning quality loss includes a left boundary positioning quality loss, an upper boundary positioning quality loss, a right boundary positioning quality loss and a lower boundary positioning quality loss;

[0230] Optionally, the calculation submodule 706 is further configured to:

[0231] determining a left border positioning quality loss according to the first left margin and the second left margin;

[0232] Determining an upper boundary positioning quality loss according to the first upper margin and the second upper margin;

[0233] determining a right border positioning quality loss according to the first right margin and the second right margin;

[0234] A lower boundary positioning quality loss is determined according to the first lower margin and the second lower margin.

[0235] Optionally, the calculation submodule 706 is further configured to:

[0236] According to the positioning quality loss, a positioning quality weight in the positioning prediction result of the regression branch is determined.

[0237] Optionally, the positioning quality loss includes a left boundary positioning quality loss, an upper boundary positioning quality loss, a right boundary positioning quality loss and a lower boundary positioning quality loss;

[0238] Optionally, the calculation submodule 706 is further configured to:

[0239] Determining a left boundary positioning quality weight according to the left boundary positioning quality loss;

[0240] Determining an upper boundary positioning quality weight according to the upper boundary positioning quality loss;

[0241] Determining a right boundary positioning quality weight according to the right boundary positioning quality loss;

[0242] Determining a lower boundary positioning quality weight according to the lower boundary positioning quality loss;

[0243] The positioning quality weight in the positioning prediction result of the regression branch is determined according to the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight and the lower boundary positioning quality weight.

[0244] Optionally, each of the initial samples includes at least one positive initial sample and at least one negative initial sample;

[0245] The determining submodule 704 is further configured to:

[0246] Based on the search image, comparing the initial classification loss of each positive initial sample with a first preset threshold value;

[0247] Determine the positive initial sample whose initial classification loss is less than the first preset threshold as a positive regression sample;

[0248] The positive initial samples whose initial classification loss is greater than or equal to the first preset threshold and the negative initial samples are determined as negative regression samples.

[0249] Optionally, each of the initial samples includes at least one positive initial sample and at least one negative initial sample;

[0250] The determining submodule 704 is further configured to:

[0251] Based on the search image, comparing the initial regression loss of each positive initial sample with a second preset threshold value respectively;

[0252] Determine the positive initial sample whose initial regression loss is less than the second preset threshold as a positive classification sample;

[0253] The positive initial samples whose initial regression loss is greater than or equal to the second preset threshold and the negative initial samples are determined as negative classification samples.

[0254] Optionally, the template image includes a plurality of feature image points, and the search image includes a marked box;

[0255] The determining submodule 704 is further configured to:

[0256] For any one of the plurality of feature points, determining whether a sampling point of an initial sample corresponding to the feature point in the search image falls within the annotation frame;

[0257] If yes, then the initial sample is determined to be a positive initial sample;

[0258] If not, it is determined that the initial sample is a negative initial sample.

[0259] Optionally, the device further comprises a testing module configured to:

[0260] Acquire a test image, input the test image into the target tracking model, and obtain a classification prediction result and a positioning prediction result of the target in the test image, wherein the classification prediction result includes the type, classification weight, and center quality weight of the target, and the positioning prediction result includes a prediction box and a positioning quality weight of the prediction box;

[0261] Calculating the prediction weight of each prediction box according to the classification weight, the center quality weight and the positioning quality weight;

[0262] According to the prediction weights of the prediction boxes, a target box is determined from the prediction boxes.

[0263] The target tracking device provided by the present application obtains the image to be tracked; then the image to be tracked is input into the trained target tracking model to obtain the target tracking result of the image to be tracked, wherein the target tracking model includes a classification branch and a regression branch, the target tracking model is obtained by training a preset neural network based on regression samples and classification samples, the regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch. That is, the target tracking device provided by the present application first trains the target tracking model including the classification branch and the regression branch: through the cross-guidance training method of the initial classification loss and the initial regression loss, the classification branch and the regression branch interact better during training, that is, the regression sample is determined by the initial classification loss for the training of the regression branch, so that the regression branch can pay more attention to the position with high classification weight; the classification sample is determined by the initial regression loss for the training of the classification branch, so that the classification task has a higher classification weight for the high-precision prediction box, which effectively ensures the alignment of the classification and regression results during the training process, thereby improving the robustness and accuracy of the target tracking model. Then, the target tracking model is used to track the target in the image to be tracked, which effectively improves the efficiency and accuracy of target tracking using the target tracking model.

[0264] The above is a schematic scheme of a target tracking device of this embodiment. It should be noted that the technical scheme of the target tracking device and the technical scheme of the target tracking model training method or the target tracking method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the target tracking device can be referred to the description of the technical scheme of the target tracking model training method or the target tracking method mentioned above.

[0265] Figure 8 The structure block diagram of a computing device 800 provided according to an embodiment of the present application is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and the database 850 is used to store data.

[0266] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0267] In one embodiment of the present application, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0268] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 800 may also be a mobile or stationary server.

[0269] Among them, when the processor 820 executes the computer instructions, it implements the steps of the target tracking model training method or the target tracking model method.

[0270] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the target tracking model training method or the target tracking model method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the target tracking model training method or the target tracking model method mentioned above.

[0271] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the target tracking model training method or the target tracking model method as described above.

[0272] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the target tracking model training method or the target tracking model method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the target tracking model training method or the target tracking model.

[0273] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0274] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0275] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0276] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0277] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A target tracking method, characterized in that: include: Acquire the image to be tracked; Inputting the image to be tracked into a trained target tracking model to obtain a target tracking result of the image to be tracked; The target tracking model includes a classification branch and a regression branch. The target tracking model is obtained by training a preset neural network based on regression samples and classification samples. The regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch. Before inputting the image to be tracked into the trained target tracking model, the method further includes: Acquire a training image group, wherein the training image group includes a template image and a search image; Based on the search image, determining a regression sample set according to the initial classification loss of each initial sample; Wherein, each of the initial samples includes at least one positive initial sample and at least one negative initial sample; The step of determining a regression sample set based on the search image and according to the initial classification loss of each initial sample comprises: Based on the search image, comparing the initial classification loss of each positive initial sample with a first preset threshold value; Determine the positive initial sample whose initial classification loss is less than the first preset threshold as a positive regression sample; Determine the positive initial samples whose initial classification loss is greater than or equal to the first preset threshold, and the negative initial samples as negative regression samples; Wherein, each of the initial samples includes at least one positive initial sample and at least one negative initial sample; The step of determining a classification sample set based on the search image and according to the initial regression loss of each initial sample comprises: Based on the search image, comparing the initial regression loss of each positive initial sample with a second preset threshold value respectively; Determine the positive initial sample whose initial regression loss is less than the second preset threshold as a positive classification sample; The positive initial samples whose initial regression loss is greater than or equal to the second preset threshold and the negative initial samples are determined as negative classification samples.

2. The method according to claim 1, characterized in that: Before inputting the image to be tracked into the trained target tracking model, the method further includes: Acquire a training image group, input the training image group into a preset neural network, and obtain an initial classification loss and an initial regression loss of each initial sample in the search image, wherein the preset neural network includes a classification branch and a regression branch, and the training image group includes a template image and a search image; Based on the search image, determining a regression sample set according to the initial classification loss of each initial sample, and determining a classification sample set according to the initial regression loss of each initial sample, wherein the regression sample set includes regression samples carrying regression labels, and the classification sample set includes classification samples carrying classification labels; Calculating the total training loss according to the classification label of each classification sample and the classification prediction result of the classification branch, as well as the regression label of each regression sample and the positioning prediction result of the regression branch; Based on the total training loss, adjusting the network parameters of the preset neural network, and returning to execute the step of obtaining the training image group; When the preset stopping condition is reached, the training is stopped to obtain the target tracking model.

3. The method according to claim 2, characterized in that The total training loss is calculated according to the classification label of each classification sample and the classification prediction result of the classification branch, as well as the regression label of each regression sample and the positioning prediction result of the regression branch, including: Based on the classification labels of the classification samples and the classification prediction results of the classification branches, an updated classification loss is obtained; based on the regression labels of the regression samples and the positioning prediction results of the regression branches, an updated regression loss is obtained; Based on the updated classification loss and the updated regression loss, the total training loss is calculated.

4. The method according to claim 3, characterized in that The updated classification loss includes classification loss and center quality loss, and the updated regression loss includes positioning loss and positioning quality loss; The step of obtaining an updated classification loss based on the classification label of each classification sample and the classification prediction result of the classification branch, and obtaining an updated regression loss based on the regression label of each regression sample and the positioning prediction result of the regression branch, comprises: Calculating classification loss based on the classification labels of the classification samples and the classification weights in the classification prediction results of the classification branches; Based on each classification label, determine the true value of the center mass of each classification sample, and calculate the center mass loss according to the true value of the center mass of each classification sample and the center mass weight in the classification prediction result of the classification branch; Calculating positioning loss based on the regression label of each regression sample and the prediction box in the positioning prediction result of the regression branch; Based on each regression label, a true value frame of a positive regression sample in each regression sample is determined, and the positioning quality loss is calculated according to the true value frame of the positive regression sample and the positioning weight in the positioning prediction result of the regression branch.

5. The method according to claim 2, characterized in that: The template image includes a plurality of feature points, and the search image includes a marked box; Before determining the regression sample set based on the search image and according to the initial classification loss of each initial sample, the method further includes: For any one of the plurality of feature points, determining whether a sampling point of an initial sample corresponding to the feature point in the search image falls within the annotation frame; If yes, then the initial sample is determined to be a positive initial sample; If not, it is determined that the initial sample is a negative initial sample.

6. The method according to claim 1, characterized in that Also includes: Acquire a test image, input the test image into the target tracking model, and obtain a classification prediction result and a positioning prediction result of the target in the test image, wherein the classification prediction result includes a classification weight and a center quality weight, and the positioning prediction result includes a prediction box and a positioning quality weight of the prediction box; Calculating the prediction weight of each prediction box according to the classification weight, the center quality weight and the positioning quality weight; According to the prediction weights of the prediction boxes, a target box is determined from the prediction boxes.

7. The method according to claim 6, characterized in that The step of inputting the test image into the target tracking model to obtain a classification prediction result and a positioning prediction result of the target in the test image includes: Inputting the test image into the classification branch of the target tracking model to obtain the classification weight and center quality weight of the target in the test image; Inputting the test image into the regression branch of the target tracking model to obtain a predicted box of the target in the test image; Calculating a positioning quality error according to a true value frame of the target and the prediction frame in a positioning prediction result of the target; According to the positioning quality error, a positioning quality weight of the prediction box in the positioning prediction result is determined.

8. The method according to claim 7, characterized in that The calculating the positioning quality error according to the true value frame of the target and the prediction frame in the positioning prediction result of the target includes: Determine first margins from the specified sampling point in the target to four boundaries of the true value box, and second margins from the specified sampling point to four boundaries of the prediction box; A positioning quality error is determined according to the first margin and the second margin.

9. The method according to claim 8, characterized in that The determining of first margins from the specified sampling point in the target to four boundaries of the true value box, and second margins from the specified sampling point to four boundaries of the prediction box, includes: Respectively determine a first left margin from a specified sampling point in the target to a left boundary of the truth box, a first upper margin from an upper boundary of the truth box, a first right margin from a right boundary of the truth box, and a first lower margin from a lower boundary of the truth box, wherein the first margin includes the first left margin, the first upper margin, the first right margin, and the first lower margin; Determine respectively a second left margin of the specified sampling point from the left boundary of the prediction box, a second upper margin from the upper boundary of the prediction box, a second right margin from the right boundary of the prediction box, and a second lower margin from the lower boundary of the prediction box, wherein the second margin includes the second left margin, the second upper margin, the second right margin, and the second lower margin.

10. The method according to claim 9, characterized in that The positioning quality error includes a left boundary positioning quality error, an upper boundary positioning quality error, a right boundary positioning quality error and a lower boundary positioning quality error; The determining the positioning quality error according to the first margin and the second margin includes: Determining a left boundary positioning quality error according to the first left margin and the second left margin; Determining an upper boundary positioning quality error according to the first upper margin and the second upper margin; Determining a right boundary positioning quality error according to the first right margin and the second right margin; A lower boundary positioning quality error is determined according to the first lower margin and the second lower margin.

11. The method according to any one of claims 7 to 10, characterized in that: The positioning quality error includes a left boundary positioning quality error, an upper boundary positioning quality error, a right boundary positioning quality error and a lower boundary positioning quality error; The determining, according to the positioning quality error, a positioning quality weight of the prediction box in the positioning prediction result includes: Determining a left boundary positioning quality weight according to the left boundary positioning quality error; Determining an upper boundary positioning quality weight according to the upper boundary positioning quality error; Determining a right boundary positioning quality weight according to the right boundary positioning quality error; Determining a lower boundary positioning quality weight according to the lower boundary positioning quality error; The positioning quality weight of the prediction box in the positioning prediction result is determined according to the left boundary positioning quality weight, the upper boundary positioning quality weight, the right boundary positioning quality weight and the lower boundary positioning quality weight.

12. A target tracking device, characterized in that: include: An acquisition module, configured to acquire an image to be tracked; A tracking module is configured to input the image to be tracked into a trained target tracking model to obtain a target tracking result of the image to be tracked; The target tracking model includes a classification branch and a regression branch. The target tracking model is obtained by training a preset neural network based on regression samples and classification samples. The regression samples are determined according to the initial classification loss of the initial samples obtained through the classification branch, and the classification samples are determined according to the initial regression loss of the initial samples obtained through the regression branch. Before inputting the image to be tracked into the trained target tracking model, the method further includes: Acquire a training image group, wherein the training image group includes a template image and a search image; Based on the search image, determining a regression sample set according to the initial classification loss of each initial sample; Wherein, each of the initial samples includes at least one positive initial sample and at least one negative initial sample; The step of determining a regression sample set based on the search image and according to the initial classification loss of each initial sample comprises: Based on the search image, comparing the initial classification loss of each positive initial sample with a first preset threshold value; Determine the positive initial sample whose initial classification loss is less than the first preset threshold as a positive regression sample; Determine the positive initial samples whose initial classification loss is greater than or equal to the first preset threshold, and the negative initial samples as negative regression samples; Wherein, each of the initial samples includes at least one positive initial sample and at least one negative initial sample; The step of determining a classification sample set based on the search image and according to the initial regression loss of each initial sample comprises: Based on the search image, comparing the initial regression loss of each positive initial sample with a second preset threshold value respectively; Determine the positive initial sample whose initial regression loss is less than the second preset threshold as a positive classification sample; The positive initial samples whose initial regression loss is greater than or equal to the second preset threshold and the negative initial samples are determined as negative classification samples.

13. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer instructions, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method described in any one of claims 1 to 11 are implemented.

Images