Target tracking method and device, electronic equipment, storage medium and product

Abstract

The invention provides a target tracking method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: generating a preset number of candidate boxes corresponding to a prompt based on the prompt in a target prompt image, and selecting a tracking target mask from candidate target masks corresponding to all candidate boxes, taking the tracking target mask and the rectangular external frame of the tracking target mask as tracking target prompt data; and based on the tracking target prompt data and the image features of the target prompt image, generating a target mask image corresponding to the target prompt image, and representing the tracking target in the target mask image with a mask. By adopting the technical scheme of the invention, the mask with the highest confidence coefficient can be selected from the candidate target masks corresponding to the candidate box as the tracking target mask for the candidate box corresponding to the prompt in the target prompt image, so that the accuracy of the tracking target mask can be improved even for the prompt with relatively low accuracy, and the user experience is improved. Therefore, the target tracking accuracy based on any type of target prompt is improved.

Description

Technical Field

[0001] This application relates to the field of visual target tracking technology, and in particular to a target tracking method, apparatus, electronic device, storage medium, and product. Background Technology

[0002] Video tracking technology is an important research area in computer vision, with wide applications in various fields, including but not limited to: security monitoring, intelligent transportation, human-computer interaction, military applications, autonomous driving, and medical diagnosis. Video tracking technology detects and tracks moving targets in a sequence of images. By analyzing the captured image sequence, it detects moving targets and calculates their position in each frame.

[0003] Existing target tracking methods rely on user-inputted target prompts to achieve target tracking accuracy. Only precise prompts can guarantee the accuracy of target tracking, such as using precise rectangles to mark the target. However, other prompts with lower precision (such as arrows, sparse points, imprecise boxes, closed polygons, etc.) will affect the accuracy of target tracking. Summary of the Invention

[0004] Based on the above needs, this application proposes a target tracking method, apparatus, electronic device, storage medium, and product that can improve the target tracking accuracy based on any type of target cues.

[0005] To achieve the above objectives, this application proposes the following technical solution:

[0006] According to a first aspect of the embodiments of this application, a target tracking method is provided, comprising:

[0007] Based on the prompts in the target prompt image, a preset number of candidate boxes corresponding to the prompts are generated, and the tracking target mask with the highest confidence is selected from the candidate target masks corresponding to all the candidate boxes. The tracking target mask and the rectangular bounding box of the tracking target mask are used as the tracking target prompt data.

[0008] Based on the tracking target cue data and the image features of the target cue image, a target mask image corresponding to the target cue image is generated, and the tracking target in the target mask image is represented by a mask.

[0009] Optionally, based on the prompt in the target prompt image, a preset number of candidate boxes corresponding to the prompt are generated, and the tracking target mask with the highest confidence is selected from all the candidate target masks corresponding to the candidate boxes, including:

[0010] Based on the prompt in the target prompt image, estimate the target outline mask corresponding to the prompt in the target prompt image, and generate a preset number of candidate boxes corresponding to the prompt according to the target outline mask corresponding to the prompt;

[0011] Based on each candidate bounding box, obtain the candidate target mask and the confidence level of each candidate target mask in the target prompt image, and select the tracking target mask with the highest confidence level from the candidate target masks based on the confidence level of each candidate target mask.

[0012] Optionally, before generating the target mask image corresponding to the target cue image based on the tracking target cue data and the image features of the target cue image, the method further includes:

[0013] Based on the tracking target cue data and the image features of the target cue image, the degree of occlusion of the tracking target is predicted;

[0014] If the degree of occlusion is less than a preset threshold, a target mask image corresponding to the target prompt image is generated based on the image features of the tracking target prompt data and the target prompt image.

[0015] Optionally, target tracking methods also include:

[0016] If the degree of occlusion is not less than a preset threshold, then based on the image features of the target prompt image, a Kalman filter is used to predict and track the target bounding box.

[0017] Based on the image features of the target prediction bounding box and the target cue image, a target mask image corresponding to the target cue image is generated.

[0018] Optionally, target tracking methods also include:

[0019] If the degree of occlusion is less than a preset threshold, the parameters of the Kalman filter are updated based on the tracking target cue data and the image features of the target cue image.

[0020] Optionally, if no tracking target cue is available in the current image frame, the method further includes:

[0021] Based on the initial image features of the current image frame and the image memory features of past image frames, the first image feature of the current image frame is determined; the image memory features of the past image frames are the fusion features of the image features of the past image frames and the tracking target cue data.

[0022] Based on the first image features of the current image frame, predict the tracking target cue data corresponding to the current image frame;

[0023] Based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, a target mask image corresponding to the current image frame is generated.

[0024] Optionally, before generating the target mask image corresponding to the current image frame based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, the method further includes:

[0025] Based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, predict the degree of occlusion of the tracking target in the current image frame;

[0026] If the degree of occlusion is less than a preset threshold, a target mask image corresponding to the current image frame is generated based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame.

[0027] Optionally, after predicting the degree of occlusion of the tracked target in the current image frame based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, the method further includes:

[0028] If the degree of occlusion is not less than a preset threshold, then based on the image features of the current image frame, a Kalman filter is used to predict the target prediction box corresponding to the current image frame, and based on the target prediction box corresponding to the current image frame and the image features of the current image frame, a target mask image corresponding to the current image frame is generated.

[0029] Optionally, based on the prompt in the target prompt image, a preset number of candidate boxes corresponding to the prompt are generated, and a tracking target mask is selected from the candidate target masks corresponding to all the candidate boxes. The tracking target mask and the rectangular bounding box of the tracking target mask are used as tracking target prompt data. Based on the tracking target prompt data and the image features of the target prompt image, a target mask image corresponding to the target prompt image is generated, including:

[0030] The target prompt image is input into a pre-built target tracking model. Based on the prompt in the target prompt image, the target tracking model generates a preset number of candidate boxes corresponding to the prompt, and selects a tracking target mask from the candidate target masks corresponding to all the candidate boxes. The tracking target mask and the rectangular bounding box of the tracking target mask are used as tracking target prompt data. Based on the tracking target prompt data and the image features of the target prompt image, a target mask image corresponding to the target prompt image is generated.

[0031] The target tracking model is the SAM2 model.

[0032] According to a second aspect of the embodiments of this application, a target tracking device is provided, comprising:

[0033] The target mask determination module is used to generate a preset number of candidate boxes corresponding to the prompts in the target prompt image, and select the tracking target mask with the highest confidence from the candidate target masks corresponding to all the candidate boxes, and use the tracking target mask and the rectangular bounding box of the tracking target mask as the tracking target prompt data;

[0034] The mask image generation module is used to generate a target mask image corresponding to the target prompt image based on the image features of the tracking target prompt data and the target prompt image, wherein the tracking target in the target mask image is represented by a mask.

[0035] According to a third aspect of the embodiments of this application, an electronic device is provided, including: a memory and a processor;

[0036] The memory is connected to the processor and is used to store programs;

[0037] The processor is used to implement the target tracking method described above by running the program in the memory.

[0038] According to a fourth aspect of the embodiments of this application, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the above-described target tracking method.

[0039] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including computer program instructions, which, when executed by a processor, cause the processor to implement the above-described target tracking method.

[0040] The target tracking method proposed in this application generates a preset number of candidate bounding boxes based on the prompts in the target prompt image, and selects a tracking target mask from the candidate target masks corresponding to all candidate bounding boxes. The tracking target mask and its rectangular bounding box are used as the tracking target prompt data. Based on the tracking target prompt data and the image features of the target prompt image, a target mask image corresponding to the target prompt image is generated, and the tracking target in the target mask image is represented by the mask. Using the technical solution of this application, the mask with the highest confidence level can be selected from the candidate target masks corresponding to the candidate bounding boxes in the target prompt image as the tracking target mask. Even for prompts with low precision, the accuracy of the tracking target mask can be improved, thereby improving the target tracking accuracy based on any type of target prompt. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 A flowchart illustrating a target tracking method provided in an embodiment of this application;

[0043] Figure 2 A flowchart illustrating another target tracking method provided in an embodiment of this application;

[0044] Figure 3 A flowchart illustrating another target tracking method provided in an embodiment of this application;

[0045] Figure 4 A flowchart illustrating another target tracking method provided in an embodiment of this application;

[0046] Figure 5 A flowchart illustrating another target tracking method provided in an embodiment of this application;

[0047] Figure 6 A flowchart illustrating another target tracking method provided in an embodiment of this application;

[0048] Figure 7 This is a schematic diagram of the target tracking model provided in the embodiments of this application;

[0049] Figure 8 This is a schematic diagram of the structure of a target tracking device provided in an embodiment of this application;

[0050] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] The technical solutions of this application are applicable to visual target tracking scenarios. By employing the technical solutions of this application, target tracking can be achieved for any type of target cues, improving the accuracy of the target mask and thus enhancing the overall accuracy of target tracking.

[0052] Video tracking is primarily based on computer vision technology and can be applied to various fields such as security monitoring, intelligent transportation, human-computer interaction, military applications, autonomous driving, and medical diagnosis. Video tracking technology detects and tracks moving targets in a sequence of images. By analyzing the captured image sequence, it detects moving targets and calculates their position in each frame. The analysis of the captured image sequence involves initializing the target features in the image sequence, establishing a template, and detecting the initialized target through feature matching within the video sequence context.

[0053] For video tracking, users input target cues in a specific frame of the video to indicate the target to be tracked. Existing target tracking methods use these cues as a reference to initialize the target and then match and detect the initialized target in the video sequence. Therefore, the accuracy of the target cues directly affects the accuracy of target detection, i.e., the accuracy of target tracking. Thus, existing target tracking methods require precise user cues to ensure accuracy, such as using precise bounding boxes to mark the target. However, different users have different marking habits, such as using arrows, closed polygons, sparse dots, or imprecise boxes. These types of cues have low precision and cannot be used to accurately initialize the target, thus affecting the accuracy of target tracking.

[0054] Based on this, this application proposes a method that can achieve target tracking for any type of target cues, improve the accuracy of tracking target masks, and thus improve the accuracy of target tracking, thereby solving the problem of low target tracking accuracy for any type of target cues in the prior art.

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] Exemplary methods

[0057] See Figure 1 As shown in the figure, this application proposes a target tracking method. The method includes:

[0058] S101. Based on the prompts in the target prompt image, generate a preset number of candidate boxes corresponding to the prompts, and select the tracking target mask with the highest confidence from the candidate target masks corresponding to all candidate boxes. Use the tracking target mask and the rectangular bounding box of the tracking target mask as the tracking target prompt data.

[0059] When tracking a target in a video sequence, the user pre-inputs a target cue in a frame of the video sequence to indicate the target to be tracked. This image containing the input cue is called the target cue image. In this embodiment, the target cue image is first analyzed to generate a candidate bounding box corresponding to the cue. The target enclosed by this candidate bounding box is the target corresponding to the cue. The target cue image may contain various different cuees. Each cue is analyzed, and a corresponding candidate bounding box is generated for each cue. Cue types include arrows, sparse dots, closed polygons, and imprecise rectangles. Imprecise rectangles are rectangles much larger than the target in the image or only enclose a portion of the target, making it impossible to accurately determine the target in the image. In this embodiment, target analysis of the target cue image can be performed using algorithms such as image contour extraction and / or foreground / background map generation to analyze the target contour and thus identify the rectangle that encloses the contour of the target.

[0060] Furthermore, a preset number of candidate boxes can be generated based on a prompt in the target prompt image, where the preset number is at least one. Different candidate boxes corresponding to a single prompt are candidate boxes of different sizes. Multiple sizes of candidate boxes ensure multiple choices, and selecting the one with the highest accuracy improves the accuracy of target initialization. Therefore, this embodiment requires mask analysis of all generated candidate boxes to determine the candidate target mask corresponding to each candidate box. That is, target analysis is performed on each candidate box to determine the target within each candidate box, and a masking operation is performed on the target. Then, confidence analysis is performed on the candidate target masks corresponding to each candidate box to determine the confidence level of the candidate target mask corresponding to each candidate box. The candidate target mask with the highest confidence level is selected as the tracking target mask, and the rectangular bounding box of the tracking target mask is determined. The tracking target mask and its rectangular bounding box are then used as the tracking target prompt data in the target prompt image. The bounding box of the rectangle is determined directly based on the coordinates of the target mask. Specifically, the coordinates of each pixel in the target mask are determined, and the minimum, maximum, minimum, and maximum x-coordinates are selected from all pixel coordinates. The minimum and maximum x-coordinates are used as the left and right boundaries of the bounding box, and the minimum and maximum y-coordinates are used as the top and bottom boundaries, thus determining the bounding box. In this embodiment, the confidence analysis of the candidate target mask is typically implemented using a mask decoder in the target tracking model.

[0061] Furthermore, based on the prompts in the target prompt image, a preset number of candidate boxes are generated corresponding to the prompts, and the tracking target mask with the highest confidence is selected from the candidate target masks corresponding to all candidate boxes. Specifically, this includes the following steps:

[0062] First, based on the prompts in the target prompt image, estimate the target outline mask corresponding to the prompt in the target prompt image, and generate a preset number of candidate boxes corresponding to the prompt according to the target outline mask corresponding to the prompt.

[0063] In this embodiment, target analysis is performed on the target prompt image according to the prompts in the target image to estimate the outline of the tracking target corresponding to the prompts in the target image, and to obtain the target outline mask of the tracking target. Specifically, for the prompts in the target prompt image, if the prompt is an arrow type, target analysis needs to be performed along the vector direction of the arrow. If the prompt is a closed polygon type (such as a triangle, polygon, circle, etc.) or an imprecise box type, target analysis needs to be performed on the area enclosed by the closed polygon in the target prompt image. If the prompt is a sparse point type, target analysis needs to be performed on a certain area that expands outward according to the position of the point in the target prompt image.

[0064] In this embodiment, target analysis of the target prompt image can be performed using image contour extraction algorithms and / or foreground / background distribution map generation algorithms to estimate the outline of the tracked target in the target prompt image and obtain the outline mask of the tracked target. This embodiment preferably uses a superposition of image contour extraction algorithms and foreground / background distribution map generation algorithms. Image contour extraction algorithms (such as edge detection algorithms Canny, Sobel, etc.) are used to extract the contour information of the tracked target, and foreground / background distribution map generation algorithms are used for foreground / background segmentation. This avoids the impact of occlusion on the extraction accuracy when extracting the outline of the tracked target. The foreground / background distribution map generation algorithm can generate foreground / background probability maps based on color, texture, or motion information through clustering (such as K-means) or graph cut, or it can directly output pixel-level classification of the foreground / background using semantic segmentation models (such as DeepLab, Mask R-CNN). In addition, the image contour extraction algorithm and the foreground and background distribution generation algorithm can be superimposed in a geometric way, using the target shape contour as the initial boundary of the mask and adjusting the confidence of the boundary by combining the foreground and background distribution maps. Alternatively, it can be superimposed in a probabilistic way, which weights and fuses the geometric information of the target shape contour with the probabilistic information of the foreground and background distribution maps to generate a comprehensive mask.

[0065] After estimating the target outline mask corresponding to the prompt in this embodiment, a preset number of candidate boxes need to be generated according to the target outline mask corresponding to the prompt. The preset number is at least one. If the preset number is one, the bounding rectangle of the target outline mask corresponding to the prompt can be directly used as a candidate box. If the preset number is multiple, the bounding rectangle of the target outline mask corresponding to the prompt can be used as one candidate box, and the length and width of the bounding rectangle can be adjusted to obtain other candidate boxes. Generating multiple candidate boxes for each prompt allows for more analytical scenarios when determining the tracking target, improving the accuracy of target tracking. Furthermore, if there are multiple types of prompts in the target prompt image in this embodiment, a preset number of candidate boxes need to be generated for each type of prompt.

[0066] Second, based on each candidate box, obtain the candidate target mask and the confidence level of each candidate box in the target prompt image, and select the tracking target mask with the highest confidence level from the candidate target masks based on the confidence level of each candidate target mask.

[0067] This embodiment generates a preset number of candidate boxes for each prompt through the above steps. For example, if there are two types of prompts in the target prompt image and the preset number is three, then six candidate boxes can be generated. For each candidate box, this embodiment obtains the candidate target mask corresponding to the candidate box in the target prompt image and calculates the confidence level of the candidate target mask. Specifically, this embodiment performs target analysis on the content in the region corresponding to the candidate box in the target prompt image to determine the target mask as the candidate target mask corresponding to that candidate box. Then, it uses the mask decoder in the target tracking model to perform confidence analysis on the candidate target mask and determine the confidence level corresponding to the candidate target mask. Preferably, the target tracking model is the SAM2 model, and the mask decoder in this model is the Mask Decoder.

[0068] In this embodiment, after determining the candidate target mask corresponding to each candidate box in the target prompt image and the confidence level corresponding to each candidate target mask, the candidate target mask with the highest confidence level is selected as the tracking target mask.

[0069] S102. Based on the tracking target prompt data and the image features of the target prompt image, generate a target mask image corresponding to the target prompt image.

[0070] In this embodiment, image feature extraction is required for each image in the video sequence during target tracking, and image feature extraction is also required for the target cue image to obtain its image features. Specifically, an image encoder is set in the target tracking model to extract image features from the target cue image. For example, if the target tracking model is the SAM2 model, then the image encoder is the Image Encoder in SAM2.

[0071] In this embodiment, the tracking target hint data includes a tracking target mask and a rectangular bounding box of the tracking target mask. The tracking target mask is a dense hint, and the rectangular bounding box of the tracking target mask is a sparse hint. This embodiment performs image mask analysis based on the image features of the tracking target hint data and the target hint image to obtain a target mask image corresponding to the target hint image. The tracking target in the target mask image is represented by a mask; that is, the region in the target illustration image corresponding to the tracking target hint data is marked with a mask to obtain the target mask image. If the tracking target hint data includes a dense hint, i.e., a tracking target mask, then the corresponding target mask image is the image after masking the tracking target in the target hint image. If the tracking target hint data includes a sparse hint, i.e., a rectangular bounding box of the tracking target mask, then the corresponding target mask image also includes the rectangular bounding box of the tracking target mask.

[0072] If this embodiment uses a target tracking model, then firstly, the target prompt data is encoded using a prompt encoder to obtain an embedding vector. Then, a mask decoder is used to decode the embedding vector and the image features of the target prompt image to obtain a segmentation mask. Finally, a target mask image corresponding to the target prompt image is generated based on the segmentation mask. Wherein, if the target tracking model is the SAM2 model, then the prompt encoder is a Prompt Encoder and the mask decoder is a Mask Decoder.

[0073] Furthermore, if the target cue image in this embodiment is the first frame of the video sequence, then the image features of the target cue image used to generate the target mask image in this embodiment are merely the initial image features of the target cue image. If the target cue image in this embodiment is not the first frame of the video sequence, then the image features of the target cue image used to generate the target mask image in this embodiment are a fusion feature between the initial image features of the target cue image and the image memory features of the image frames preceding the target cue image. This fusion feature is preferably obtained by performing cross-attention calculation using an attention mechanism. For example, it can be implemented using Memory Attention in the SAM2 model. The image memory features of an image frame include the image features of that image frame and the tracking target cue data of that image frame. If there is no cue in the image frame and no cue in the image frames preceding it, then the tracking target cue data of that image frame is empty. If there is no cue in the image frame, but there is a cue in the image frames preceding it, then the tracking target cue data of that image frame is predicted based on the tracking target cue data of the previous image frames. If there is a cue in the image frame, then the tracking target cue data of that image frame is also obtained using the above steps.

[0074] The target tracking model used in this embodiment is an extension of the pre-trained large model SAM2, which inherits the memory of the SAM2 model, eliminating the need for additional training and fine-tuning, thus improving the efficiency of target tracking.

[0075] As described above, the target tracking method proposed in this application generates a preset number of candidate boxes corresponding to the prompts in the target prompt image, and selects a tracking target mask from the candidate target masks corresponding to all candidate boxes. The tracking target mask and its bounding rectangle are used as tracking target prompt data. Based on the tracking target prompt data and the image features of the target prompt image, a target mask image corresponding to the target prompt image is generated, and the tracking target in the target mask image is represented by the mask. Using the technical solution of this embodiment, the mask with the highest confidence level can be selected from the candidate target masks corresponding to the prompts in the target prompt image as the tracking target mask. Even for prompts with low accuracy, the accuracy of the tracking target mask can be improved, thereby improving the target tracking accuracy based on any type of target prompt.

[0076] As an optional implementation, this application also proposes a target tracking method.

[0077] See Figure 2 As shown, the method includes:

[0078] S201. Based on the prompts in the target prompt image, generate a preset number of candidate boxes corresponding to the prompts, and select the tracking target mask with the highest confidence from the candidate target masks corresponding to all candidate boxes. Use the tracking target mask and the rectangular bounding box of the tracking target mask as the tracking target prompt data.

[0079] S202. Based on the image features of the tracking target cue data and the target cue image, predict the degree of occlusion of the tracking target.

[0080] In this embodiment, after generating the target cue image tracking target cue data, it is necessary to predict the degree of occlusion of the tracked target based on the image features of the tracking target cue data and the target cue image. That is, for the image features of the target cue image, the occlusion status of the tracked target in the features corresponding to the tracking target cue data is analyzed. Specifically, in the target tracking model, a mask decoder is used to predict the degree of occlusion of the tracked target, for example, the Mask Decoder in the SAM2 model. The mask decoder can calculate the logits score token parameter, which is used to predict whether the segmented object is occluded and calculate the occlusion score. The larger the occlusion score, the smaller the degree of occlusion, and the smaller the occlusion score, the larger the degree of occlusion.

[0081] In addition, the mask decoder in the target tracking model can also calculate the IoU Token parameter and the mask Token parameter. The IoU Token parameter is used to predict whether the segmented objects belong to the same target in the temporal context, which plays a role in target re-identification; the mask Token parameter is used to output a mask on the feature map.

[0082] S203. If the degree of occlusion is less than a preset threshold, then a target mask image corresponding to the target prompt image is generated based on the image features of the tracking target prompt data and the target prompt image.

[0083] In this embodiment, when the predicted occlusion level of the tracked target in the target hint image is less than a preset threshold, it indicates that the occlusion level of the tracked target in the target hint image is not high. Therefore, a target mask image corresponding to the target hint image is directly generated based on the tracking target hint data and the image features of the target hint image. The specific steps for generating the target mask image corresponding to the target hint image based on the tracking target hint data and the image features of the target hint image have been described in detail in the above embodiments and will not be repeated in this embodiment.

[0084] The specific execution method of step S201 in this embodiment is the same as that of step S101 in the above embodiment, and will not be described in detail in this embodiment.

[0085] As an optional implementation, this application also proposes a target tracking method.

[0086] See Figure 3 As shown, the method includes:

[0087] S301. Based on the prompts in the target prompt image, generate a preset number of candidate boxes corresponding to the prompts, and select the tracking target mask with the highest confidence from the candidate target masks corresponding to all candidate boxes. Use the tracking target mask and the rectangular bounding box of the tracking target mask as the tracking target prompt data.

[0088] S302. Based on the image features of the tracking target cue data and the target cue image, predict the degree of occlusion of the tracking target.

[0089] S303. If the degree of occlusion is not less than a preset threshold, then based on the image features of the target prompt image, a Kalman filter is used to predict the target prediction box.

[0090] If the predicted occlusion level of the tracked target in the target cue image is not less than a preset threshold, it indicates a high degree of occlusion. Directly using this target cue data to generate a target mask image can easily lead to segmentation errors, affecting the accuracy of target tracking. Therefore, it is necessary to use a Kalman filter to predict the target bounding box based on the image features of the target cue image. The Kalman filter is trained for prediction during each application of the target tracking method. Specifically, each time the predicted occlusion level of the tracked target in the image is less than the preset threshold, it indicates that the determined target cue data is relatively accurate. At this point, the Kalman filter is trained using the target cue data and the corresponding image features. That is, the image features of the image are input into the Kalman filter to make it output a target bounding box. The Kalman filter's parameters are adjusted to minimize the difference between the predicted bounding box and the bounding box in the target cue data. Through continuous prediction and self-updating over long-term application, the accuracy of the Kalman filter is ensured.

[0091] S304. Based on the image features of the target prediction bounding box and the target cue image, generate a target mask image corresponding to the target cue image.

[0092] In this embodiment, when the degree of occlusion is not less than a preset threshold, it is necessary to generate a target mask image corresponding to the target prompt image based on the image features of the tracking target prediction box and the target prompt image predicted by the Kalman filter. This is the same as the specific execution method of generating a target mask image corresponding to the target prompt image based on the rectangular bounding box in the tracking target prompt data and the image features of the target prompt image in the above embodiment, and will not be described in detail in this embodiment.

[0093] In this embodiment, when the occlusion corresponding to the target prompt data is severe and affects the target tracking accuracy, the Kalman filter can be used to predict the target prediction box, achieving interpretable tracking even under occlusion conditions and improving the accuracy of target tracking.

[0094] The specific execution method of steps S301-S302 in this embodiment is the same as that of steps S201-S202 in the above embodiment, and will not be described in detail in this embodiment.

[0095] As an optional implementation, this application also proposes a target tracking method.

[0096] See Figure 4 As shown, the method includes:

[0097] S401. Based on the prompts in the target prompt image, generate a preset number of candidate boxes corresponding to the prompts, and select the tracking target mask with the highest confidence from the candidate target masks corresponding to all candidate boxes. Use the tracking target mask and the rectangular bounding box of the tracking target mask as the tracking target prompt data.

[0098] S402. Based on the image features of the tracking target cue data and the target cue image, predict the degree of occlusion of the tracking target.

[0099] S403. If the degree of occlusion is less than a preset threshold, then a target mask image corresponding to the target prompt image is generated based on the image features of the tracking target prompt data and the target prompt image.

[0100] S404. Based on the tracking target cue data and the image features of the target cue image, update the parameters of the Kalman filter.

[0101] The Kalman filter is trained for prediction during each application of the target tracking method. When the occlusion degree of the tracked target in the predicted target cue image is less than a preset threshold, it indicates that the determined target cue data is relatively accurate. At this time, the target cue data and the image features of the corresponding target cue image are directly used to train the Kalman filter for prediction. That is, the image features of the target cue image are input into the Kalman filter so that the Kalman filter outputs a target prediction box. The Kalman filter is adjusted with the goal of minimizing the difference between the target prediction box and the bounding box of the rectangle in the target cue data. Through long-term application, the Kalman filter continuously predicts and updates itself to ensure its accuracy.

[0102] The specific execution method of steps S401-S403 in this embodiment is the same as that of steps S201-S203 in the above embodiment, and will not be described in detail in this embodiment.

[0103] As an optional implementation, this application also proposes a target tracking method. See [link to relevant documentation]. Figure 5 As shown, when there is no tracking target cue in the current image frame, the method includes:

[0104] S501. Based on the initial image features of the current image frame and the image memory features of past image frames, determine the first image features of the current image frame.

[0105] If there are no clues for the tracked target in the current image frame, this embodiment first needs to extract image features from the current image frame to obtain the initial image features of the current image frame. Then, the initial image features of the current image frame are fused and analyzed with the image memory features of past image frames, so that the image features of the current image frame, combined with the image features of historical image frames and target mask information, determine the first image feature of the current image frame.

[0106] The image memory features of past image frames include image features of past image frames and tracking target cue data of past image frames. That is, the image memory features of past image frames are a fusion feature of image features of past image frames and tracking target cue data. The past image frame is at least one image frame before the current image frame, preferably all image frames before the current image frame. If the past image frame is a target cue image, the tracking target cue data of the past image frame is obtained through the tracking target cue data acquisition method described in the above embodiments, which will not be repeated in this embodiment. If the past image frame is not a target cue image, the tracking target cue data of the past image frame is obtained through the tracking target cue data acquisition method of this embodiment. Furthermore, if the past image frame is a target cue image, and the predicted occlusion degree of the tracking target is less than a preset threshold, the tracking target cue data obtained in step S101 is directly used as the tracking target cue data of the past image frame. If the predicted occlusion degree of the tracking target is not less than the preset threshold, the tracking target prediction box obtained in step S304 is used as the tracking target cue data of the past image frame.

[0107] Specifically, this embodiment performs cross-attention calculation on the initial image features of the current image frame and the image memory features of past image frames to achieve the combination of current image features and historical image features, i.e., contextual combination. This enables the determined first image feature of the current image frame to combine the features mentioned above and the features of the historical tracking target.

[0108] This embodiment can be implemented using a target tracking model. The image encoder in the target tracking model extracts the initial image features of the current image frame, and the attention module in the target tracking model determines the first image feature of the current image frame based on the initial image features of the current image frame and the image memory features of past image frames. For example, in the SAM2 model, the Image Encoder extracts the initial image features of the current image frame, and the Memory Attention module determines the first image feature of the current image frame based on the initial image features of the current image frame and the image memory features of past image frames.

[0109] S502. Based on the first image features of the current image frame, predict the tracking target cue data corresponding to the current image frame.

[0110] After determining the first image feature of the current image frame through the above steps, since the first image feature contains the image features of historical image frames and the tracking target cue data of the tracked target in the historical image frames, the tracking target cue data corresponding to the current image frame can be predicted. That is, the movement trajectory of the tracked target can be predicted, and the tracking target cue data corresponding to the current image frame can be determined according to the movement trajectory and the tracking target cue data.

[0111] This embodiment can be implemented using a target tracking model. The mask decoder in the target tracking model predicts the tracking target cue data corresponding to the current image frame based on the first image feature of the current image frame, such as the Mask Decoder in the SAM2 model.

[0112] S503. Based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, generate the target mask image corresponding to the current image frame.

[0113] In this embodiment, after predicting the tracking target cue data corresponding to the current image frame, a target mask image corresponding to the current image frame is generated based on the tracking target cue data and the image features of the current image frame. The specific execution method of this step is the same as that of the above embodiment, which generates a target mask image corresponding to the target cue image based on the tracking target cue data and the image features of the target cue image, and will not be repeated in this embodiment.

[0114] As an optional implementation, this application also proposes a target tracking method. See [link to relevant documentation]. Figure 6 As shown, when there is no tracking target cue in the current image frame, the method includes:

[0115] S601. Based on the initial image features of the current image frame and the image memory features of past image frames, determine the first image feature of the current image frame.

[0116] S602. Based on the first image feature of the current image frame, predict the tracking target prompt data corresponding to the current image frame.

[0117] S603. Based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, predict the degree of occlusion of the tracking target in the current image frame.

[0118] In this embodiment, the specific execution method for predicting the degree of occlusion of the tracked target in the current image frame based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame is the same as the specific execution method for predicting the degree of occlusion of the tracked target based on the tracking target cue data and the image features of the target cue image in the above embodiment. Therefore, this embodiment will not repeat the details.

[0119] S604. If the degree of occlusion is less than a preset threshold, then based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, generate the target mask image corresponding to the current image frame.

[0120] In this embodiment, the specific execution method for generating the target mask image corresponding to the current image frame based on the tracking target prompt data and the image features of the current image frame is the same as the specific execution method for generating the target mask image corresponding to the target prompt image based on the tracking target prompt data and the image features of the target prompt image in the above embodiment, and will not be repeated in this embodiment.

[0121] S605. If the degree of occlusion is not less than a preset threshold, then based on the image features of the current image frame, use a Kalman filter to predict the target prediction box corresponding to the current image frame; based on the target prediction box corresponding to the current image frame and the image features of the current image frame, generate the target mask image corresponding to the current image frame.

[0122] In this embodiment, the specific execution method of predicting the target prediction box corresponding to the current image frame using a Kalman filter based on the image features of the current image frame, and generating the target mask image corresponding to the current image frame based on the target prediction box and the image features of the current image frame, is the same as the specific execution method of predicting the target prediction box using a Kalman filter based on the image features of the target prompt image, and generating the target mask image corresponding to the target prompt image based on the target prediction box and the image features of the target prompt image in the above embodiment. Therefore, this embodiment will not repeat the details.

[0123] As an optional implementation, another embodiment of this application discloses that a target cue image is input into a pre-built target tracking model. The target tracking model generates a preset number of candidate boxes corresponding to the cue in the target cue image, selects a tracking target mask from the candidate target masks corresponding to all candidate boxes, uses the tracking target mask and its bounding rectangle as tracking target cue data, and generates a target mask image corresponding to the target cue image based on the tracking target cue data and the image features of the target cue image. The target tracking model includes a cue sampler, a cue encoder, and a mask decoder. The cue sampler generates a preset number of candidate boxes corresponding to the cue based on the cue in the target cue image. The mask decoder selects a tracking target mask from the candidate target masks corresponding to all candidate boxes. The cue encoder uses the tracking target mask and its bounding rectangle as tracking target cue data. The mask decoder generates a target mask image corresponding to the target cue image based on the tracking target cue data and the image features of the target cue image.

[0124] Furthermore, the mask decoder can predict the degree of occlusion of the tracked target based on the image features of the tracking target cue data and the target cue image. If the occlusion degree is less than a preset threshold, the mask decoder generates a target mask image corresponding to the target cue image based on the image features of the tracking target cue data and the target cue image, and updates the parameters of the Kalman filter based on the image features of the tracking target cue data and the target cue image. If the occlusion degree is not less than the preset threshold, the Kalman filter predicts the tracking target bounding box based on the image features of the target cue image, and then the mask decoder generates a target mask image corresponding to the target cue image based on the tracking target bounding box and the image features of the target cue image.

[0125] In addition, the target tracking model includes an attention module. This module determines the first image feature of the current image frame based on the initial image features of the current frame and the image memory features of past image frames. Then, a mask decoder predicts the target cue data corresponding to the current image frame based on this first image feature. Finally, based on the target cue data and the image features of the current image frame, a target mask image corresponding to the current image frame is generated.

[0126] The target tracking model mentioned above is preferably the SAM2 model.

[0127] As an optional implementation, see [link to implementation details]. Figure 7As shown, another embodiment of this application discloses the structure of a target tracking model, including: an image encoder, a memory attention module, a mask decoder, a Kalman filter, a memory encoder, a motion memory, a prompt sampler, and a prompt encoder. The image encoder, memory attention module, mask decoder, memory encoder, motion memory, and prompt encoder are all existing modules in the SAM2 model. The prompt sampler and Kalman filter are improvements to the SAM2 model.

[0128] Specifically, image A is the target cue image or the current image frame, and image B is the target mask image corresponding to image A. The image encoder, memory attention module, mask decoder, Kalman filter, prompt sampler, and prompt encoder have been specifically described in the above embodiments. The memory encoder is used to encode the image features of the image frame and the tracking target cue data to obtain image memory features; the motion memory is used to store the image memory features transmitted by the memory encoder and also to store the tracking target prediction boxes transmitted by the Kalman filter.

[0129] Figure 7 The solid arrows represent the processes that need to be executed for each image frame in the video sequence, while the dashed arrows represent the processes that need to be executed only for image frames in the video sequence that contain tracking target cues (i.e., target cues images).

[0130] Exemplary device

[0131] Accordingly, embodiments of this application also provide a target tracking device, see below. Figure 8 As shown, the device includes:

[0132] The target mask determination module 100 is used to generate a preset number of candidate boxes corresponding to the prompts based on the prompts in the target prompt image, and select the tracking target mask with the highest confidence from the candidate target masks corresponding to all candidate boxes, and use the tracking target mask and the rectangular bounding box of the tracking target mask as the tracking target prompt data.

[0133] The mask image generation module 110 is used to generate a target mask image corresponding to the target prompt image based on the image features of the tracking target prompt data and the target prompt image. The tracking target in the target mask image is represented by a mask.

[0134] As can be seen from the above description, it is possible to select the mask with the highest confidence from the candidate target masks corresponding to the candidate boxes in the target prompt image as the tracking target mask. Even for prompts with low precision, it can improve the accuracy of tracking target masks, thereby improving the target tracking accuracy based on any type of target prompt.

[0135] As an optional implementation, another embodiment of this application discloses a target mask determination module 100, specifically used for:

[0136] Based on the prompts in the target prompt image, estimate the target outline mask corresponding to the prompt in the target prompt image, and generate a preset number of candidate boxes corresponding to the prompt according to the target outline mask corresponding to the prompt;

[0137] Based on each candidate bounding box, obtain the candidate target mask and the confidence level of each candidate target mask in the target prompt image. Based on the confidence level of each candidate target mask, select the tracking target mask with the highest confidence level from the candidate target masks.

[0138] As an optional implementation, another embodiment of this application discloses a target tracking device, which further includes an occlusion prediction module.

[0139] The occlusion prediction module is used to predict the degree of occlusion of the tracked target based on the image features of the tracking target cue data and the target cue image;

[0140] The mask image generation module 110 is specifically used to generate a target mask image corresponding to the target prompt image based on the image features of the tracking target prompt data and the target prompt image if the degree of occlusion is less than a preset threshold.

[0141] As an optional implementation, another embodiment of this application discloses a target tracking device, which further includes a box prediction module.

[0142] The bounding box prediction module is used to predict the target bounding box based on the image features of the target prompt image and using a Kalman filter if the degree of occlusion is not less than a preset threshold.

[0143] The mask image generation module 110 is specifically used to generate a target mask image corresponding to the target prompt image based on the image features of the tracking target prediction box and the target prompt image.

[0144] As an optional implementation, another embodiment of this application discloses a target tracking device, which further includes a parameter update module.

[0145] The parameter update module is used to update the parameters of the Kalman filter based on the tracking target cue data and the image features of the target cue image if the degree of occlusion is less than a preset threshold.

[0146] As an optional implementation, another embodiment of this application discloses a target tracking device, which further includes: a feature determination module and a prompt prediction module.

[0147] The feature determination module is used to determine the first image feature of the current image frame based on the initial image features of the current image frame and the image memory features of past image frames; the image memory features of past image frames are the fusion features of the image features of past image frames and the tracking target cue data.

[0148] The prompt prediction module is used to predict the tracking target prompt data corresponding to the current image frame based on the first image features of the current image frame;

[0149] The mask image generation module 110 is also used to generate a target mask image corresponding to the current image frame based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame.

[0150] As an optional implementation, another embodiment of this application discloses an occlusion prediction module, which is further used to predict the degree of occlusion of the tracking target in the current image frame based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame.

[0151] The mask image generation module 110 is also used to generate a target mask image corresponding to the current image frame based on the tracking target prompt data corresponding to the current image frame and the image features of the current image frame if the degree of occlusion is less than a preset threshold.

[0152] As an optional implementation, another embodiment of this application discloses a bounding box prediction module, which is further configured to predict the tracking target prediction bounding box corresponding to the current image frame based on the image features of the current image frame using a Kalman filter if the degree of occlusion is not less than a preset threshold; the mask image generation module 110 is further configured to generate a target mask image corresponding to the current image frame based on the tracking target prediction bounding box corresponding to the current image frame and the image features of the current image frame.

[0153] The target tracking device provided in this embodiment belongs to the same concept as the target tracking method provided in the above embodiments of this application. It can execute the target tracking method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the target tracking method. Technical details not described in detail in this embodiment can be found in the specific processing content of the target tracking method provided in the above embodiments of this application, and will not be repeated here.

[0154] Exemplary electronic devices

[0155] Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 9 As shown, the device includes:

[0156] Memory 200 and processor 210;

[0157] The memory 200 is connected to the processor 210 and is used to store programs;

[0158] The processor 210 is configured to implement the target tracking method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0159] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0160] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0161] A bus can include a pathway for transmitting information between various components of a computer system.

[0162] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0163] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0164] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0165] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0166] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0167] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0168] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of any of the target tracking methods provided in the above embodiments of this application.

[0169] Exemplary computer program products and storage media

[0170] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the target tracking methods according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0171] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0172] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor in the target tracking methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0173] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0174] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0175] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0176] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0177] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0178] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0179] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0180] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0182] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0183] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A target tracking method, characterized in that, include: Based on the prompts in the target prompt image, a preset number of candidate boxes corresponding to the prompts are generated, and the tracking target mask with the highest confidence is selected from the candidate target masks corresponding to all the candidate boxes. The tracking target mask and the rectangular bounding box of the tracking target mask are used as the tracking target prompt data. Based on the tracking target cue data and the image features of the target cue image, a target mask image corresponding to the target cue image is generated, and the tracking target in the target mask image is represented by a mask.

2. The method according to claim 1, characterized in that, Based on the prompts in the target prompt image, a preset number of candidate boxes corresponding to the prompts are generated, and the tracking target mask with the highest confidence is selected from all the candidate target masks corresponding to the candidate boxes, including: Based on the prompt in the target prompt image, estimate the target outline mask corresponding to the prompt in the target prompt image, and generate a preset number of candidate boxes corresponding to the prompt according to the target outline mask corresponding to the prompt; Based on each candidate bounding box, obtain the candidate target mask and the confidence level of each candidate target mask in the target prompt image, and select the tracking target mask with the highest confidence level from the candidate target masks based on the confidence level of each candidate target mask.

3. The method according to claim 1, characterized in that, Before generating the target mask image corresponding to the target prompt image based on the tracking target prompt data and the image features of the target prompt image, the method further includes: Based on the tracking target cue data and the image features of the target cue image, the degree of occlusion of the tracking target is predicted; If the degree of occlusion is less than a preset threshold, a target mask image corresponding to the target prompt image is generated based on the image features of the tracking target prompt data and the target prompt image.

4. The method according to claim 3, characterized in that, Also includes: If the degree of occlusion is not less than a preset threshold, then based on the image features of the target prompt image, a Kalman filter is used to predict and track the target bounding box. Based on the image features of the target prediction bounding box and the target cue image, a target mask image corresponding to the target cue image is generated.

5. The method according to claim 3, characterized in that, Also includes: If the degree of occlusion is less than a preset threshold, the parameters of the Kalman filter are updated based on the tracking target cue data and the image features of the target cue image.

6. The method according to claim 1, characterized in that, If no tracking target cue is found in the current image frame, the method further includes: Based on the initial image features of the current image frame and the image memory features of past image frames, the first image feature of the current image frame is determined; the image memory features of the past image frames are the fusion features of the image features of the past image frames and the tracking target cue data. Based on the first image features of the current image frame, predict the tracking target cue data corresponding to the current image frame; Based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, a target mask image corresponding to the current image frame is generated.

7. The method according to claim 6, characterized in that, Before generating the target mask image corresponding to the current image frame based on the tracking target cue data and the image features of the current image frame, the method further includes: Based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, predict the degree of occlusion of the tracking target in the current image frame; If the degree of occlusion is less than a preset threshold, a target mask image corresponding to the current image frame is generated based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame.

8. The method according to claim 7, characterized in that, Based on the tracking target cue data corresponding to the current image frame and the image features of the current image frame, after predicting the degree of occlusion of the tracking target in the current image frame, the method further includes: If the degree of occlusion is not less than a preset threshold, then based on the image features of the current image frame, a Kalman filter is used to predict the target prediction box corresponding to the current image frame, and based on the target prediction box corresponding to the current image frame and the image features of the current image frame, a target mask image corresponding to the current image frame is generated.

9. The method according to claim 1, characterized in that, Based on the prompts in the target prompt image, a preset number of candidate bounding boxes are generated corresponding to the prompts. A tracking target mask is selected from the candidate target masks corresponding to all the candidate bounding boxes. The tracking target mask and its bounding rectangle are used as tracking target prompt data. Based on the tracking target prompt data and the image features of the target prompt image, a target mask image corresponding to the target prompt image is generated, including: The target prompt image is input into a pre-built target tracking model. Based on the prompt in the target prompt image, the target tracking model generates a preset number of candidate boxes corresponding to the prompt, and selects a tracking target mask from the candidate target masks corresponding to all the candidate boxes. The tracking target mask and the rectangular bounding box of the tracking target mask are used as tracking target prompt data. Based on the tracking target prompt data and the image features of the target prompt image, a target mask image corresponding to the target prompt image is generated. The target tracking model is the SAM2 model.

10. A target tracking device, characterized in that, include: The target mask determination module is used to generate a preset number of candidate boxes corresponding to the prompts in the target prompt image, and select the tracking target mask with the highest confidence from the candidate target masks corresponding to all the candidate boxes, and use the tracking target mask and the rectangular bounding box of the tracking target mask as the tracking target prompt data; The mask image generation module is used to generate a target mask image corresponding to the target prompt image based on the image features of the tracking target prompt data and the target prompt image, wherein the tracking target in the target mask image is represented by a mask.

11. An electronic device, characterized in that, include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the target tracking method as described in any one of claims 1 to 9 by running a program in the memory.

12. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the target tracking method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, cause the processor to implement the target tracking method as described in any one of claims 1 to 9.

Images