Tracking method, system, electronic device and medium based on minimum entropy criterion update

Abstract

The application discloses a tracking method and system based on minimum entropy criterion updating, electronic equipment and medium, and relates to the field of target tracking. The method comprises the following steps: acquiring a video image to be tracked; determining the position of a tracking target in an initial frame; determining a target feature vector of a frame image tracking sequence; determining the position of the tracking target; determining a feature vector of a t-th frame image; determining a prediction box and a corresponding prediction score of a plurality of tracking targets of the t-th frame image; determining a template to be updated; when the t-th frame image is the last frame image of the frame image tracking sequence, acquiring a (t+1)-th frame image of the frame image tracking sequence, and taking the (t+1)-th frame image as the t-th frame image; and obtaining a tracking result when the t-th frame image is the last frame image of the frame image tracking sequence. The application can improve the effect of target tracking when tracking encounters occlusion and illumination change.

Description

Technical Field

[0001] This invention relates to the field of target tracking, and in particular to a tracking method, system, electronic device, and medium based on the minimum entropy criterion for updating. Background Technology

[0002] Target tracking is a crucial topic in computer vision, widely applied in fields such as drones, pedestrian detection, surveillance, and thermal infrared tracking. Currently, deep learning is extensively used in visual target tracking, particularly tracking models based on the Transformer architecture, which overcome the semantic loss inherent in Siamese network tracking during linear matching of relevance operations, achieving superior tracking performance. However, it neglects the importance of model updates, making it prone to model drift when the target's appearance is affected by complex environmental factors.

[0003] To address the impact of complex environments during tracking and enable trackers to adapt to changes in both the target and the environment, model updates are crucial. Specifically, Chen, X., et al. proposed a visual tracking method based on the Transformer architecture, Transformer Tracking (TransT). While TransT addresses the loss of semantic information in the locally linear matching process by incorporating an attention-based feature fusion network, it neglects the vital importance of model updates. When tracking encounters challenges such as occlusion and changes in lighting, these factors can cause a loss of robustness and accuracy; therefore, incorporating model update techniques is essential. Summary of the Invention

[0004] The purpose of this invention is to provide a tracking method, system, electronic device, and medium based on the minimum entropy criterion, which can improve the tracking effect when encountering occlusion or changes in lighting.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A tracking method based on the minimum entropy criterion for updating, the tracking method comprising:

[0007] Obtain a frame image tracking sequence composed of video images to be tracked;

[0008] The position of the tracking target is determined from the preset initial frame of the frame image tracking sequence to obtain the target position of the initial frame;

[0009] Based on the initial frame target position, a convolutional neural network is applied to determine the target feature vector of the frame image tracking sequence;

[0010] The position of the tracked target is determined from the t-th frame image of the frame image tracking sequence, and the target position of the t-th frame image is obtained;

[0011] Based on the target position of the t-th frame image, a convolutional neural network is applied to determine the feature vector of the t-th frame image;

[0012] The target feature vector and the feature vector of the t-th frame image are input into the feature fusion network to obtain the prediction boxes of multiple tracked targets in the t-th frame image and the prediction scores corresponding to each prediction box;

[0013] Based on the preset score threshold and the prediction score corresponding to each prediction box, it is determined whether there is a prediction box in the t-th frame image that meets the conditions as a high confidence template corresponding to the t-th frame image.

[0014] When there is no prediction box that meets the conditions in the t-th frame image as the high confidence template corresponding to the t-th frame image, obtain the (t+1)-th frame image of the frame image tracking sequence, and use the (t+1)-th frame image as the t-th frame image, and return to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image".

[0015] When there is a prediction box that meets the conditions in the t-th frame image, which is used as the high-confidence template corresponding to the t-th frame image, the template to be updated is determined according to the minimum entropy ratio of the high-confidence template corresponding to the t-th frame image and the minimum entropy ratio of the dynamic label template, and the dynamic label template is updated; the dynamic label template is composed of the high-confidence templates determined from the previous t-1 frames of the frame image tracking sequence;

[0016] Determine whether the t-th frame image is the last frame image in the frame image tracking sequence;

[0017] When the t-th frame image is not the last frame image of the frame image tracking sequence, a convolutional neural network is applied according to the template to be updated to determine the feature vector to be updated, and the feature vector to be updated is fused with the target feature vector to obtain the updated target feature vector. The t+1-th frame image of the frame image tracking sequence is obtained, and the t+1-th frame image is used as the t-th frame image. The process returns to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image".

[0018] When the t-th frame image is the last frame image of the frame image tracking sequence, a tracking result is obtained; the tracking result is the position of the tracking target in each frame image of the video image to be tracked.

[0019] Optionally, based on the initial frame target position, a convolutional neural network is applied to determine the target feature vector of the frame image tracking sequence, specifically including:

[0020] Based on the initial frame target position, a convolutional neural network is applied to determine the target features of the frame image tracking sequence;

[0021] The target features are reshaped to obtain the target feature vector of the frame image tracking sequence.

[0022] Optionally, based on a preset score threshold and the prediction score corresponding to each prediction box, it is determined whether there is a prediction box in the t-th frame image that meets the conditions to serve as a high-confidence template for the t-th frame image, specifically including:

[0023] From the predicted scores corresponding to each predicted box, select the predicted boxes with scores greater than a preset threshold to obtain an initial predicted box sequence;

[0024] Determine the first prediction box corresponding to the maximum prediction score and the second prediction box corresponding to the minimum prediction score from the initial prediction box sequence;

[0025] Calculate the center offset of the first and second prediction boxes;

[0026] Based on the center offset and the preset center offset threshold, determine whether there is a prediction box in the t-th frame image that meets the conditions to serve as the high confidence template corresponding to the t-th frame image.

[0027] Optionally, based on the center offset and a preset center offset threshold, it is determined whether there is a prediction box in the t-th frame image that meets the conditions to serve as a high-confidence template for the t-th frame image, specifically including:

[0028] When the center offset is less than the preset center offset threshold, the prediction box in the t-th frame image that meets the condition is used as the high confidence template corresponding to the t-th frame image.

[0029] When the center offset is greater than the preset center offset threshold, there is no prediction box in the t-th frame image that meets the condition to serve as the high confidence template corresponding to the t-th frame image.

[0030] Optionally, the feature fusion network consists of a self-attention-based self-context enhancement module and a cross-attention-based cross-feature enhancement module.

[0031] Optionally, the feature vector to be updated and the target feature vector are linearly fused to obtain the updated target feature vector.

[0032] A tracking system based on the minimum entropy criterion for updating, applied to the aforementioned tracking method based on the minimum entropy criterion for updating, the tracking system comprising:

[0033] The first acquisition module is used to acquire a frame image tracking sequence composed of video images to be tracked;

[0034] The first position determination module is used to determine the position of the tracking target from a preset initial frame of the frame image tracking sequence, and obtain the target position of the initial frame;

[0035] The first feature vector removal module is used to determine the target feature vector of the frame image tracking sequence by applying a convolutional neural network based on the target position of the initial frame.

[0036] The second position determination module is used to determine the position of the tracking target from the t-th frame image of the frame image tracking sequence, and obtain the target position of the t-th frame image;

[0037] The second feature vector removal module is used to determine the feature vector of the t-th frame image by applying a convolutional neural network based on the target position of the t-th frame image.

[0038] The score determination module is used to input the target feature vector and the feature vector of the t-th frame image into the feature fusion network to obtain the prediction boxes of multiple tracked targets in the t-th frame image and the prediction score corresponding to each prediction box;

[0039] The first judgment module is used to determine whether there is a prediction box in the t-th frame image that meets the conditions as a high confidence template corresponding to the t-th frame image, based on a preset score threshold and the prediction score corresponding to each prediction box.

[0040] The second acquisition module is used to acquire the (t+1)th frame image of the frame image tracking sequence when there is no prediction box that meets the conditions in the t-th frame image as the high confidence template corresponding to the t-th frame image, and use the (t+1)th frame image as the t-th frame image, and return to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence, and obtain the target position of the t-th frame image";

[0041] The module for determining the template to be updated is used to determine the template to be updated based on the minimum entropy ratio of the high-confidence template corresponding to the t-th frame image and the minimum entropy ratio of the dynamic marker template when there is a prediction box that meets the conditions as the high-confidence template corresponding to the t-th frame image. The module also updates the dynamic marker template. The dynamic marker template is composed of high-confidence templates determined from the previous t-1 frames of the frame image tracking sequence.

[0042] The second judgment module is used to determine whether the t-th frame image is the last frame image of the frame image tracking sequence;

[0043] The third acquisition module is used to, when the t-th frame image is not the last frame image of the frame image tracking sequence, apply a convolutional neural network according to the template to be updated, determine the feature vector to be updated, fuse the feature vector to be updated with the target feature vector to obtain the updated target feature vector, acquire the (t+1)-th frame image of the frame image tracking sequence, use the (t+1)-th frame image as the t-th frame image, and return to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image";

[0044] The result determination module is used to obtain the tracking result when the t-th frame image is the last frame image of the frame image tracking sequence; the tracking result is the position of the tracking target in each frame image of the video image to be tracked.

[0045] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor running the computer program to cause the electronic device to perform the tracking method based on the minimum entropy criterion update described above.

[0046] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned tracking method based on the minimum entropy criterion for updating.

[0047] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] This invention provides a tracking method, system, electronic device, and medium based on the minimum entropy criterion. By using the minimum entropy criterion to determine the template confidence, reliable historical information can be provided, improving tracking robustness when severe occlusion or environmental influences occur. By fusing the high-confidence template with the feature vector of the initial frame, the lack of nonlinear expressive power in the Transformer tracking process can be solved, enhancing the accuracy and success rate of the tracker. When the target undergoes challenges such as rapid movement, occlusion, scale changes, and in-plane rotation, the tracker can still perform accurate and effective tracking. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of the tracking method based on the minimum entropy criterion of this invention.

[0051] Figure 2 This is a flowchart illustrating the specific application of the Transformer tracking method based on the minimum entropy criterion in Example 1.

[0052] Figure 3 This is the tracking result of the TransT algorithm on the 10th frame of the OTB100 dataset Bolt.

[0053] Figure 4 This is the tracking result of the method of the present invention for the 10th frame of the Bolt dataset in the OTB100 dataset.

[0054] Figure 5 This is the tracking result of the TransT algorithm for frame 149 of the OTB100 dataset Bolt.

[0055] Figure 6 This is the tracking result of the method of the present invention for the 149th frame of the Bolt dataset in the OTB100 dataset.

[0056] Figure 7 This is the tracking result of the TransT algorithm for the 10th frame of the Skating1 dataset OTB100.

[0057] Figure 8 This is the tracking result of the method of the present invention for the 10th frame of Skating1 in the OTB100 dataset.

[0058] Figure 9 This is the tracking result of the TransT algorithm for frame 197 of the OTB100 dataset Skating1.

[0059] Figure 10 This is the tracking result of the method of the present invention for frame 197 of the Skating1 dataset OTB100. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] The purpose of this invention is to provide a tracking method, system, electronic device, and medium based on the minimum entropy criterion, which can improve the tracking effect when encountering occlusion or changes in lighting.

[0062] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] Example 1

[0064] like Figure 1 and Figure 2 As shown, the present invention provides a tracking method based on the minimum entropy criterion for updating, the tracking method comprising:

[0065] Step S1: Obtain a frame image tracking sequence composed of video images to be tracked; in practical applications, read the tracking sequence to obtain the initial frame image.

[0066] Step S2: Determine the position of the tracking target from the preset initial frame of the frame image tracking sequence to obtain the target position of the initial frame; in practical applications, the initial position of the tracked target is determined according to the ground truth.

[0067] Step S3: Based on the initial frame target position, apply a convolutional neural network to determine the target feature vector of the frame image tracking sequence; S3 specifically includes:

[0068] Step S31: Based on the target position of the initial frame, apply a convolutional neural network to determine the target features of the frame image tracking sequence.

[0069] Step S32: Reshape the target features to obtain the target feature vector of the frame image tracking sequence.

[0070] In practical applications, the target to be tracked is determined based on the initial frame, and target features are extracted from the tracking sequence using a convolutional neural network. F z And reshape it into a feature vector F z1 The reshaping process uses the `reshape` function to change the dimension of the target feature, resulting in a feature vector. In this invention, the target feature is a three-dimensional feature vector, and the feature vector is a two-dimensional feature vector. Reshaping the target feature into a feature vector refers to reshaping the extracted three-dimensional feature vector into a two-dimensional feature vector. Furthermore, the target feature refers to the feature extracted using the ResNet network.

[0071] Step S4: Determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image.

[0072] Step S5: Based on the target position of the t-th frame image, apply a convolutional neural network to determine the feature vector of the t-th frame image.

[0073] In practical applications, the t-th frame is read, and target features are extracted based on the search region. F x And reshape it into a feature vector F x1 .

[0074] Step S6: Input the target feature vector and the feature vector of the t-th frame image into the feature fusion network to obtain the prediction boxes of multiple tracked targets in the t-th frame image and the prediction scores corresponding to each prediction box; specifically, the feature fusion network consists of a self-attention-based self-context enhancement module and a cross-attention-based cross-feature enhancement module.

[0075] In practical applications, F z1 and F x1 Input the feature fusion network of the TransT tracking algorithm, perform fusion feature classification and regression, and obtain m predicted scores and corresponding predicted boxes.

[0076] Step S7: Based on the preset score threshold and the prediction score corresponding to each prediction box, determine whether there is a prediction box in the t-th frame image that meets the conditions to serve as a high-confidence template for the t-th frame image; S7 specifically includes:

[0077] Step S71: Select prediction boxes with scores greater than a preset score threshold from the prediction scores corresponding to each prediction box to obtain an initial prediction box sequence.

[0078] Step S72: Determine the first prediction box corresponding to the maximum prediction score and the second prediction box corresponding to the minimum prediction score from the initial prediction box sequence.

[0079] Step S73: Calculate the center offset of the first prediction box and the second prediction box.

[0080] Step S74: Based on the center offset and the preset center offset threshold, determine whether there is a prediction box in the t-th frame image that meets the conditions to serve as the high-confidence template corresponding to the t-th frame image. S74 specifically includes:

[0081] Step S741: When the center offset is less than the preset center offset threshold, the prediction box in the t-th frame image that meets the condition is used as the high confidence template corresponding to the t-th frame image.

[0082] Step S742: When the center offset is greater than the preset center offset threshold, there is no prediction box in the t-th frame image that meets the condition as the high confidence template corresponding to the t-th frame image.

[0083] In practical applications, a threshold σ is set. When score > σ, the scores from the set of scores greater than σ are selected. max and score minFor each corresponding predicted bounding box, calculate the center offset ξ between the two boxes. When the offset ξ is less than the threshold ρ, set the template corresponding to the current frame as the high-confidence template T.

[0084] Calculate the center position offset:

[0085]

[0086] in, The offset of the center position, 'a' represents the score. max The center point of the prediction box is represented by b, where b represents the score. min The location of the center point of the corresponding prediction box.

[0087] Step S8: When there is no prediction box that meets the conditions in the t-th frame image as the high confidence template corresponding to the t-th frame image, obtain the (t+1)-th frame image of the frame image tracking sequence, and use the (t+1)-th frame image as the t-th frame image, and return to the step "Determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image".

[0088] Step S9: When there is a prediction box that meets the conditions in the t-th frame image as the high confidence template corresponding to the t-th frame image, determine the template to be updated according to the minimum entropy ratio of the high confidence template corresponding to the t-th frame image and the minimum entropy ratio of the dynamic label template, and update the dynamic label template; the dynamic label template is composed of the high confidence templates determined by the previous t-1 frames of the frame image tracking sequence.

[0089] In practical applications, based on the m prediction boxes provided by the prediction head... Each of them Represents a prediction box, using Label the class as positive or negative. Samples based on partial labels. Where z represents a possible set of labels containing The label y of the smaller instance x. From this, the minimum entropy ratio of the high-confidence template T is calculated. And set n dynamic tag templates (High-confidence historical template), set Choose the minimum entropy ratio of the labeled template. and Smaller templates are identified as those requiring updating. The minimum entropy ratio of the templates to be updated is calculated:

[0090]

[0091] in, Defined as the optimal bounding box probability. Defined as entropy, with scalar λ representing the weight between the probability and the prior. Let be the balancing parameter. And we define entropy as:

[0092]

[0093] Calculate the minimum entropy ratio of the labeled template:

[0094]

[0095] set up The set of marker templates representing the entire tracking process, defined by the time window. Minimum within The corresponding template composition, where k represents any number of frames within the time window. Let... The minimum entropy ratio of the labeled template. The minimum entropy ratio of the high-confidence template T.

[0096] Step S10: Determine whether the t-th frame image is the last frame image of the frame image tracking sequence.

[0097] Step S11: When the t-th frame image is not the last frame image of the frame image tracking sequence, a convolutional neural network is applied according to the template to be updated to determine the feature vector to be updated, and the feature vector to be updated is fused with the target feature vector to obtain the updated target feature vector. The (t+1)-th frame image of the frame image tracking sequence is obtained, and the (t+1)-th frame image is used as the t-th frame image. The process returns to step "Determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image". Specifically, the feature vector to be updated and the target feature vector are linearly fused to obtain the updated target feature vector.

[0098] In practical applications, the template to be updated is used to extract features into a feature vector with the same size and dimensions as the initial template. ,Will and Linearly fused into cumulative feature vectors , participate in the tracking of the (t+1)th frame, t=t+1, and repeat steps S4 to S11.

[0099] Linear fusion cumulative feature vector:

[0100]

[0101] in, The initial template feature vector, This is a high-confidence template feature vector. This is the cumulative eigenvector.

[0102] Step S12: When the t-th frame image is the last frame image of the frame image tracking sequence, a tracking result is obtained. The tracking result is the position of the tracked target in each frame image of the video image to be tracked. In practical applications, target tracking is completed when the t-th frame image is the last frame image of the frame image tracking sequence.

[0103] To verify the overall tracking performance of this invention, it was validated on 100 videos on the OTB100 dataset, a mainstream single-target tracking platform, and the overall performance was evaluated in terms of accuracy and success rate.

[0104] Table 1 shows the comparison of the present invention with the TransT tracker in terms of accuracy and success rate across all 100 videos of OTB100. As shown in the table, the present invention improves accuracy by 2.9% and success rate by approximately 1.4% compared to the baseline TransT tracker.

[0105] Table 1. Comparison of OTB100 dataset results

[0106]

[0107] Figure 3 and Figure 5 The TransT algorithm is used for Blot tracking results. The rectangular box is the tracking box. The target can be tracked at 10 frames, but drift occurs at 149 frames. Figure 4 and Figure 6 The method of this invention can accurately track the target based on the Blot tracking results; Figure 7 and Figure 9 This is the tracking result of the TransT algorithm for Skating1. The rectangle is the tracking box. The target can be tracked at 10 frames, but drift occurs at 197 frames. Figure 8 and Figure 10 The method of this invention can accurately track the target based on the tracking results of Skating1. Therefore, the tracking method based on the minimum entropy criterion provided by this invention has significant tracking performance.

[0108] The method provided by this invention has the following advantages over previous technologies:

[0109] (1) Using the minimum entropy criterion to determine the template confidence can provide reliable historical information and improve tracking robustness when there is severe occlusion or environmental influence.

[0110] (2) By linearly fusing the high-confidence template with the initial frame feature vector, the problem of lack of nonlinear expression ability in the Transformer tracking process can be solved, thereby enhancing the accuracy and success rate of the tracker.

[0111] (3) When the target undergoes challenges such as rapid movement, occlusion, scale changes, and in-plane rotation, the tracker can still perform accurate and effective tracking.

[0112] Example 2

[0113] To implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a tracking system based on the minimum entropy criterion is provided below. The tracking system includes:

[0114] The first acquisition module is used to acquire a frame image tracking sequence composed of video images to be tracked.

[0115] The first position determination module is used to determine the position of the tracking target from a preset initial frame of the frame image tracking sequence, and obtain the target position of the initial frame.

[0116] The first feature vector removal module is used to determine the target feature vector of the frame image tracking sequence by applying a convolutional neural network based on the target position of the initial frame.

[0117] The second position determination module is used to determine the position of the tracking target from the t-th frame image of the frame image tracking sequence, and obtain the target position of the t-th frame image.

[0118] The second feature vector removal module is used to determine the feature vector of the t-th frame image by applying a convolutional neural network based on the target position of the t-th frame image.

[0119] The score determination module is used to input the target feature vector and the feature vector of the t-th frame image into the feature fusion network to obtain the prediction boxes of multiple tracked targets in the t-th frame image and the prediction scores corresponding to each prediction box.

[0120] The first judgment module is used to determine whether there is a prediction box in the t-th frame image that meets the conditions as a high confidence template corresponding to the t-th frame image, based on a preset score threshold and the prediction score corresponding to each prediction box.

[0121] The second acquisition module is used to acquire the (t+1)th frame image of the frame image tracking sequence when there is no prediction box that meets the conditions in the t-th frame image as the high confidence template corresponding to the t-th frame image, and use the (t+1)th frame image as the t-th frame image, and return to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image".

[0122] The template to be updated determination module is used to determine the template to be updated based on the minimum entropy ratio of the high-confidence template corresponding to the t-th frame image and the minimum entropy ratio of the dynamic marker template when there is a prediction box that meets the conditions as the high-confidence template corresponding to the t-th frame image. The dynamic marker template is then updated. The dynamic marker template is composed of high-confidence templates determined from the previous t-1 frames of the frame image tracking sequence.

[0123] The second judgment module is used to determine whether the t-th frame image is the last frame image of the frame image tracking sequence.

[0124] The third acquisition module is used to, when the t-th frame image is not the last frame image of the frame image tracking sequence, apply a convolutional neural network according to the template to be updated, determine the feature vector to be updated, fuse the feature vector to be updated with the target feature vector to obtain the updated target feature vector, acquire the (t+1)-th frame image of the frame image tracking sequence, use the (t+1)-th frame image as the t-th frame image, and return to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image".

[0125] The result determination module is used to obtain the tracking result when the t-th frame image is the last frame image of the frame image tracking sequence. The tracking result is the position of the tracked target in each frame image of the video image to be tracked.

[0126] Example 3

[0127] This invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the tracking method based on the minimum entropy criterion update according to Embodiment 1.

[0128] Alternatively, the aforementioned electronic device may be a server.

[0129] In addition, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the tracking method based on the minimum entropy criterion update of Embodiment 1.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0131] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A tracking method based on update of a minimum entropy criterion, characterized in that, The tracking method comprises: acquiring a frame image tracking sequence composed of to-be-tracked video images; determining the position of a tracking target from a preset initial frame of the frame image tracking sequence to obtain an initial frame target position; applying a convolutional neural network according to the initial frame target position to determine a target feature vector of the frame image tracking sequence; determining the position of the tracking target from a tth frame image of the frame image tracking sequence to obtain a target position of the tth frame image; applying a convolutional neural network according to the target position of the tth frame image to determine a feature vector of the tth frame image; inputting the target feature vector and the feature vector of the tth frame image into a feature fusion network to obtain a plurality of predicted boxes of the tracking target of the tth frame image and a predicted score corresponding to each predicted box; judging whether there is a predicted box meeting the condition in the tth frame image as a high-confidence template corresponding to the tth frame image according to a preset score threshold and the predicted score corresponding to each predicted box; when there is no predicted box meeting the condition in the tth frame image as the high-confidence template corresponding to the tth frame image, acquiring a (t+1)th frame image of the frame image tracking sequence and taking the (t+1)th frame image as the tth frame image to return to the step of determining the position of the tracking target from the tth frame image of the frame image tracking sequence to obtain the target position of the tth frame image; when there is a predicted box meeting the condition in the tth frame image as the high-confidence template corresponding to the tth frame image, determining a to-be-updated template according to a minimum entropy ratio of the high-confidence template corresponding to the tth frame image and a minimum entropy ratio of a dynamic marked template, and updating the dynamic marked template; the dynamic marked template is composed of high-confidence templates determined from the first t-1 frame images of the frame image tracking sequence; judging whether the tth frame image is the last frame image of the frame image tracking sequence; when the tth frame image is not the last frame image of the frame image tracking sequence, applying a convolutional neural network according to the to-be-updated template to determine a to-be-updated feature vector, fusing the to-be-updated feature vector with the target feature vector to obtain an updated target feature vector, acquiring a (t+1)th frame image of the frame image tracking sequence and taking the (t+1)th frame image as the tth frame image to return to the step of determining the position of the tracking target from the tth frame image of the frame image tracking sequence to obtain the target position of the tth frame image; when the tth frame image is the last frame image of the frame image tracking sequence, obtaining a tracking result; the tracking result is the position of the tracking target in each frame image in the to-be-tracked video images.

2. The tracking method based on minimum entropy criterion update according to claim 1, characterized in that, applying a convolutional neural network according to the initial frame target position to determine a target feature vector of the frame image tracking sequence, specifically comprising: applying a convolutional neural network according to the initial frame target position to determine a target feature of the frame image tracking sequence; reshaping the target feature to obtain a target feature vector of the frame image tracking sequence.

3. The tracking method based on minimum entropy criterion update according to claim 1, characterized in that, Based on a preset score threshold and the prediction score corresponding to each prediction box, it is determined whether there is a prediction box in the t-th frame image that meets the conditions to serve as a high-confidence template for the t-th frame image. Specifically, this includes: From the predicted scores corresponding to each predicted box, select the predicted boxes with scores greater than a preset threshold to obtain an initial predicted box sequence; Determine the first prediction box corresponding to the maximum prediction score and the second prediction box corresponding to the minimum prediction score from the initial prediction box sequence; Calculate the center offset of the first and second prediction boxes; Based on the center offset and the preset center offset threshold, determine whether there is a prediction box in the t-th frame image that meets the conditions to serve as the high confidence template corresponding to the t-th frame image.

4. The tracking method of updating based on the minimum entropy criterion according to claim 3, characterized in that, Based on the center offset and a preset center offset threshold, determine whether there is a prediction box in the t-th frame image that meets the conditions to serve as a high-confidence template for the t-th frame image, specifically including: When the center offset is less than the preset center offset threshold, the prediction box in the t-th frame image that meets the condition is used as the high confidence template corresponding to the t-th frame image. When the center offset is greater than the preset center offset threshold, there is no prediction box in the t-th frame image that meets the condition to serve as the high confidence template corresponding to the t-th frame image.

5. The tracking method based on minimum entropy criterion update according to claim 1, characterized in that, The feature fusion network consists of a self-attention-based self-context enhancement module and a cross-attention-based cross-feature enhancement module.

6. The tracking method based on minimum entropy criterion update according to claim 1, characterized in that, The feature vector to be updated is linearly fused with the target feature vector to obtain the updated target feature vector.

7. A tracking system based on update of a minimum entropy criterion, characterized in that, The tracking system includes: The first acquisition module is used to acquire a frame image tracking sequence composed of video images to be tracked; The first position determination module is used to determine the position of the tracking target from a preset initial frame of the frame image tracking sequence, and obtain the target position of the initial frame; The first feature vector removal module is used to determine the target feature vector of the frame image tracking sequence by applying a convolutional neural network based on the target position of the initial frame. The second position determination module is used to determine the position of the tracking target from the t-th frame image of the frame image tracking sequence, and obtain the target position of the t-th frame image; The second feature vector removal module is used to determine the feature vector of the t-th frame image by applying a convolutional neural network based on the target position of the t-th frame image. The score determination module is used to input the target feature vector and the feature vector of the t-th frame image into the feature fusion network to obtain the prediction boxes of multiple tracked targets in the t-th frame image and the prediction score corresponding to each prediction box; The first judgment module is used to determine whether there is a prediction box in the t-th frame image that meets the conditions as a high confidence template corresponding to the t-th frame image, based on a preset score threshold and the prediction score corresponding to each prediction box. The second acquisition module is used to acquire the (t+1)th frame image of the frame image tracking sequence when there is no prediction box that meets the conditions in the t-th frame image as the high confidence template corresponding to the t-th frame image, and use the (t+1)th frame image as the t-th frame image, and return to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence, and obtain the target position of the t-th frame image"; The module for determining the template to be updated is used to determine the template to be updated based on the minimum entropy ratio of the high-confidence template corresponding to the t-th frame image and the minimum entropy ratio of the dynamic marker template when there is a prediction box that meets the conditions as the high-confidence template corresponding to the t-th frame image. The module also updates the dynamic marker template. The dynamic marker template is composed of high-confidence templates determined from the previous t-1 frames of the frame image tracking sequence. The second judgment module is used to determine whether the t-th frame image is the last frame image of the frame image tracking sequence; The third acquisition module is used to, when the t-th frame image is not the last frame image of the frame image tracking sequence, apply a convolutional neural network according to the template to be updated, determine the feature vector to be updated, fuse the feature vector to be updated with the target feature vector to obtain the updated target feature vector, acquire the (t+1)-th frame image of the frame image tracking sequence, use the (t+1)-th frame image as the t-th frame image, and return to the step "determine the position of the tracking target from the t-th frame image of the frame image tracking sequence to obtain the target position of the t-th frame image"; The result determination module is used to obtain the tracking result when the t-th frame image is the last frame image of the frame image tracking sequence; the tracking result is the position of the tracking target in each frame image of the video image to be tracked.

8. An electronic device, comprising: The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform a tracking method based on the minimum entropy criterion update according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the tracking method based on the minimum entropy criterion update as described in any one of claims 1 to 6.

Images