Target tracking methods, devices, electronic equipment and storage media

Abstract

This application provides a target tracking method, apparatus, electronic device, and storage medium. The target tracking method includes: acquiring template image features of a template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked; acquiring first image features of a target image to be detected; processing the first image features according to the template image features using a cross-attention mechanism to obtain second image features, wherein the second image features include target features of the target to be detected; and matching the template target features with the second image features to obtain a tracking result for the target to be tracked. By processing the template image features and the target image features using a cross-attention mechanism, the target features to be detected contain feature information of the template target features, improving the generalization ability of the features and achieving high-precision tracking.

Description

Technical Field

[0001] This application relates to the field of intelligent analysis technology, and in particular to a target tracking method, apparatus, electronic device, and storage medium. Background Technology

[0002] In projects involving intelligent masking of videos, the tracking targets mainly include product logos, advertising watermarks, and prohibited items. These targets exhibit various deformations in the video, posing a significant challenge to the accuracy of target tracking. Improving target tracking accuracy requires the tracking algorithm to extract features with stronger representational capabilities; simultaneously, improving the algorithm's tracking accuracy must also reduce false detections.

[0003] In existing technologies, feature extraction networks such as ResNet, Overfeat, VGG (Visual Geometry Group), and AlexNet are typically used to extract features from template images and images to be detected. The extracted template image features are then matched with the features of the image to be detected, and the tracking of the target is achieved based on the matching results. However, the tracking accuracy is often not high when using the above methods. Summary of the Invention

[0004] The purpose of this application is to provide a target tracking method, apparatus, electronic device, and storage medium to improve the accuracy of target tracking. The specific technical solution is as follows:

[0005] In a first aspect of this application, a target tracking method is provided, comprising:

[0006] Obtain template image features of a template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked;

[0007] Obtain the first image features of the image to be detected;

[0008] According to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features, wherein the second image features include the target features of the target to be detected;

[0009] The template target features are matched with the second image features to obtain the tracking result of the target to be tracked.

[0010] In one possible implementation, obtaining the template image features of the template image includes:

[0011] Get multiple template images;

[0012] A preset feature extraction network is used to extract features from each template image to obtain multiple third image features;

[0013] For each of the third image features, the third image feature is processed using channel attention mechanism and spatial attention mechanism to obtain the fourth image feature corresponding to the third image feature;

[0014] The template image features are obtained by fusing multiple fourth image features.

[0015] In one possible implementation, the step of processing each of the third image features using channel attention and spatial attention mechanisms to obtain the corresponding fourth image feature includes:

[0016] For each third image feature, channel attention mechanism is applied to the third image feature to obtain the fifth image feature;

[0017] The third image feature is fused with the fifth image feature to obtain the sixth image feature;

[0018] The sixth image feature is processed using a spatial attention mechanism to obtain the seventh image feature;

[0019] The third image feature, the seventh image feature, and the fifth image feature are fused together to obtain the fourth image feature corresponding to the third image feature.

[0020] In one possible implementation, acquiring the first image feature of the image to be detected includes:

[0021] The image to be detected is extracted using a pre-defined feature extraction network to obtain the eighth image feature.

[0022] The eighth image features are processed using channel attention and spatial attention mechanisms to obtain the first image features.

[0023] In one possible implementation, acquiring multiple template images includes:

[0024] Obtain each template image in the template sequence, wherein the template sequence stores T frames of template images, where T is a preset positive integer;

[0025] The method further includes:

[0026] The surveillance video is sampled according to a preset sampling frequency to obtain the current template image;

[0027] Extract a sequence of images to be detected from the surveillance video, wherein the sequence of images to be detected includes the images to be detected;

[0028] If the current template image is not an image in the image sequence to be detected, delete the template image that was added earliest in the template sequence and add the current template image to the template sequence.

[0029] In one possible implementation, the method further includes:

[0030] When the image to be detected belongs to the template image, calculate the first intersection-union ratio between the location region of the template target feature in the image to be detected and the location region of the target to be tracked in the previous frame image, wherein the previous frame image is the image used for target tracking in the previous frame of the image to be detected in the video to be tracked;

[0031] If the first cross-union ratio is greater than the preset first cross-union ratio threshold, the tracking result of the target to be tracked is obtained according to the location region of the template target feature in the image to be detected;

[0032] If the first cross-union ratio is not greater than the preset first cross-union ratio threshold, the following step is triggered: according to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features.

[0033] In one possible implementation, the method further includes:

[0034] Based on the tracking results of the target to be tracked, calculate the second cross-union ratio and confidence difference between the position regions of the target to be tracked in two adjacent frames of the image to be detected;

[0035] If the second cross-union ratio is greater than a preset second cross-union ratio threshold and / or the confidence difference is greater than a preset confidence deviation threshold, the tracking of the target to be tracked shall be interrupted.

[0036] In one possible implementation, the method further includes:

[0037] The tracking results of the target to be tracked are filtered by a preset filter to obtain a corrected tracking result.

[0038] In one possible implementation, the complexity of the preset feature extraction network is less than the preset complexity.

[0039] In a second aspect of this application, a target tracking device is also provided, comprising:

[0040] A template image feature acquisition module is used to acquire template image features of a template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked;

[0041] A first image feature acquisition module is used to acquire first image features of the image to be detected.

[0042] The processing module is used to process the first image features according to the template image features using a cross-attention mechanism to obtain the second image features, wherein the second image features include the target features of the target to be detected;

[0043] A matching module is used to match the template target features with the second image features to obtain the tracking result of the target to be tracked.

[0044] In one possible implementation, the template image feature acquisition module includes:

[0045] A submodule for acquiring multiple template images;

[0046] The first feature extraction submodule is used to extract features from each template image using a preset feature extraction network to obtain multiple third image features.

[0047] The first submodule for characterization processing is used to process each of the third image features using channel attention and spatial attention mechanisms to obtain the fourth image feature corresponding to the third image feature.

[0048] The fusion submodule is used to fuse multiple fourth image features to obtain the template image features.

[0049] In one possible implementation, the characterization processing first submodule includes:

[0050] A channel attention processing unit is used to process each third image feature using a channel attention mechanism to obtain a fifth image feature.

[0051] The first fusion unit is used to fuse the third image feature with the fifth image feature to obtain the sixth image feature;

[0052] A spatial attention processing unit is used to process the sixth image feature using a spatial attention mechanism to obtain the seventh image feature.

[0053] The second fusion unit is used to fuse the third image feature, the seventh image feature, and the fifth image feature to obtain the fourth image feature corresponding to the third image feature.

[0054] In one possible implementation, the first image feature acquisition module includes:

[0055] The second feature extraction submodule is used to extract features from the image to be detected using a preset feature extraction network to obtain the eighth image features.

[0056] The second submodule for characterization processing is used to process the eighth image features using channel attention and spatial attention mechanisms to obtain the first image features.

[0057] In one possible implementation, the acquisition submodule includes:

[0058] A template image acquisition unit is used to sample the surveillance video at a preset sampling frequency to acquire the current template image;

[0059] The image sequence extraction unit extracts an image sequence to be detected from the surveillance video, wherein the image sequence to be detected includes the image to be detected;

[0060] The template image update unit deletes the earliest added template image in the template sequence and adds the current template image to the template sequence when the current template image is not an image in the image sequence to be detected.

[0061] In one possible implementation, the apparatus further includes a filtering module, which is configured to: when the image to be detected belongs to the template image, calculate a first intersection-union ratio (IUU) between the location region of the template target feature in the image to be detected and the location region of the target to be tracked in the previous frame image, wherein the previous frame image is the previous frame of the image to be detected in the video to be tracked used for target tracking;

[0062] If the first cross-union ratio is greater than the preset first cross-union ratio threshold, the tracking result of the target to be tracked is obtained according to the location region of the template target feature in the image to be detected;

[0063] If the first cross-union ratio is not greater than the preset first cross-union ratio threshold, the following step is triggered: according to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features.

[0064] In one possible implementation, the device further includes an interruption processing module, which is configured to: calculate the cross-union ratio and confidence difference between the positions of the target to be tracked in two adjacent frames of the image to be detected based on the tracking result of the target to be tracked; and interrupt the tracking of the target to be tracked if the cross-union ratio is greater than a preset second cross-union ratio threshold and / or the confidence difference is greater than a preset confidence deviation threshold.

[0065] In one possible implementation, the device further includes a filtering module, which is used to filter the tracking result of the target to be tracked through a preset filter to obtain a corrected tracking result.

[0066] In one possible implementation, the first feature extraction submodule and the second feature extraction submodule are used to extract features with a complexity lower than a preset complexity.

[0067] In another aspect of this application, an electronic device is also provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0068] Memory, used to store computer programs;

[0069] When a processor executes a program stored in memory, it implements any of the steps described above.

[0070] In another aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements any of the target tracking methods described above.

[0071] The target tracking method, apparatus, electronic device, and storage medium provided in this application propose a target tracking method based on a cross-attention mechanism. This method processes template image features and target image features using a cross-attention mechanism, enabling the target features to contain feature information of the template target features. This improves the generalization ability of the features and makes the target features to be tracked in the target features more accurate. This solves the problem of low tracking accuracy caused by the relatively simple feature extraction network in the prior art, and improves the accuracy of target tracking. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0073] Figure 1This is a schematic flowchart of a target tracking method according to an embodiment of this application;

[0074] Figure 2 This is a schematic diagram of a process for obtaining template image features of a template image according to an embodiment of this application.

[0075] Figure 3 This is a flowchart illustrating the processing methods of the channel attention mechanism and spatial attention mechanism in embodiments of this application.

[0076] Figure 4 This is a flowchart illustrating one possible implementation of step S304 in an embodiment of this application.

[0077] Figure 5 This is a schematic diagram of a process for obtaining the first image feature of an image to be detected according to an embodiment of this application;

[0078] Figure 6 This is a schematic diagram of a process for obtaining multiple template images according to an embodiment of this application;

[0079] Figure 7 This is a schematic diagram of a process for filtering tracking results according to an embodiment of this application;

[0080] Figure 8 A flowchart illustrating the interruption handling of tracking results in an embodiment of this application;

[0081] Figure 9 A flowchart illustrating the filtering process for the tracking results in an embodiment of this application;

[0082] Figure 10 This is a schematic diagram of a target tracking device according to an embodiment of this application;

[0083] Figure 11 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0084] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0085] To improve the accuracy of target tracking, this application provides a target tracking method, comprising: acquiring template image features of a template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked; acquiring first image features of an image to be detected; processing the first image features according to the template image features using a cross-attention mechanism to obtain second image features, wherein the second image features include target features of the target to be detected; and matching the template target features with the second image features to obtain a tracking result of the target to be tracked.

[0086] The target tracking method provided in this application proposes a target tracking method based on a cross-attention mechanism. By processing the template image features and the features to be detected using a cross-attention mechanism, the features to be detected contain the feature information of the template target features, which improves the generalization ability of the features and makes the features to be tracked in the features to be detected more accurate. This solves the problem of low tracking accuracy caused by the relatively simple feature extraction network in the prior art, and realizes high-precision tracking of the target to be tracked.

[0087] The following is a detailed explanation:

[0088] See Figure 1 , Figure 1 This is a first schematic diagram of a target tracking method according to an embodiment of this application. The method includes:

[0089] S101, Obtain template image features of template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked.

[0090] The template image is an image containing the target to be tracked. In one example, frame extraction can be used to extract video frames containing the target as template images. The target to be tracked can be customized according to the actual situation; for example, it can be vehicles, pedestrians, product logos, advertising watermarks, or items prohibited from appearing in the video. Taking a product logo as an example, the template image can be a screenshot from the video containing the product logo, and the template target feature is the feature of the product logo.

[0091] S102, Obtain the first image features of the image to be detected.

[0092] The image to be detected is a video frame that needs to be checked to determine whether it contains the target to be tracked. The first image feature is the target to be tracked in the image to be detected, such as vehicles, pedestrians, product logos, advertising watermarks, or items that are prohibited from appearing in the video.

[0093] S103, according to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features, wherein the second image features include the target features of the target to be detected.

[0094] By employing a cross-attention mechanism, similarity is calculated after extracting features from the template image and the image to be detected. The template image features are then passed to the image to be detected by calculating the cross-attention matrix between the template and the image to be detected. Based on the template image features, the first image features of the image to be detected are further processed to obtain the second image features, thereby improving the generalization ability of the features. Generalization ability refers to the ability of a machine learning algorithm to adapt to new samples. This step reduces the probability of false detection and greatly improves the tracking accuracy.

[0095] The template image features include template target information and background information. These features are transferred to the target image by calculating a cross-attention matrix between the template and the image to be detected. In other words, the template target information and background information are transferred to the target image through this cross-attention matrix. During the further processing of the first image features of the target image, the response of the target information features is improved, while the response of the background information features is suppressed, thereby reducing background interference.

[0096] In one example, taking a product logo as the target to be tracked, when extracting features from the product logo in the image to be detected, in order to improve tracking accuracy, a cross-attention mechanism can be used. After extracting the features of the template image and the features of the image to be detected, similarity calculation is performed. Then, by calculating the cross-attention matrix between the template and the image to be detected, the product logo features extracted from the image to be detected are further processed based on the template image to suppress background interference and obtain logo features with improved generalization ability, making the tracking of product logo features in the image to be detected more accurate.

[0097] S104, the template target features are matched with the second image features to obtain the tracking result of the target to be tracked.

[0098] The template target features are used as a matching template and matched with the second image features to determine whether there is a feature portion in the second image features that matches the template target features. If so, the region in the image to be detected corresponding to that feature portion is taken as the region where the target to be tracked is located. Based on the region where the target to be tracked is located in the image to be detected, the target to be tracked is tracked, thereby obtaining the tracking result of the target to be tracked.

[0099] In one example, taking a product logo as the target to be tracked, the product logo features in the template image are used as a matching template and matched with the features of the target image enhanced by a cross-attention mechanism. This determines whether the product logo features exist in the target image. If they do, the region in the target image corresponding to those product logo features is taken as the region where the product logo is located. Based on the region where the product logo is located in the target image, the product logo is tracked, thus obtaining the tracking result.

[0100] In this process, the region in the image to be detected corresponding to the feature part is divided into bounding boxes. The region where the target to be tracked is located is set as the tracking box region, and the region where the template image corresponding to the feature part is located is set as the template box region.

[0101] The target tracking method provided in this application enhances the generalization ability of the target features to be tracked through a cross-attention mechanism, improves the accuracy of the target features to be tracked in the image to be detected, and solves the problem that the tracking accuracy is not high due to the inaccuracy of the target features to be tracked in the image to be detected in the prior art, thereby achieving high-precision tracking of the target to be tracked.

[0102] In one example, it is necessary to block all product logos appearing in a video to be detected:

[0103] The first step is to obtain template video frames, which include the image features of the product logo that needs to be masked and the features of the product logo itself.

[0104] The second step is to obtain the video frames of the video to be detected, which includes the image features of the video frames to be detected.

[0105] The third step requires using a cross-attention mechanism to enhance the image features of the video frame to be detected based on the characteristics of the product logo in the template video;

[0106] The fourth step requires using the logo feature in the template video frame as a template to match the image features of the video frame to be detected, and determining whether the image features of the video frame to be detected include the logo feature. If the image features of the video frame to be detected include the logo feature, then the logo feature in the video frame to be detected is tracked and masked.

[0107] In one possible implementation, such as Figure 2 As shown, obtaining the template image features of the template image includes:

[0108] S201, Obtain multiple template images.

[0109] Since the target to be tracked in the template image can have various shapes and positions, multiple template images can be obtained through template video frames.

[0110] S202, use a preset feature extraction network to extract features from each template image to obtain multiple third image features.

[0111] Feature extraction, a concept in computer vision and image processing, refers to using computers to extract information from images and determine whether each point in an image belongs to an image feature. Multiple third-party image features can be obtained by extracting features from multiple template images using a pre-defined feature extraction network. Commonly used feature extraction networks in related technologies can be employed, such as ResNet, Overfeat, VGG, and AlexNet. For example, taking a product logo as the target to be tracked, a CNN (Convolutional Neural Network) can be used to extract features from the product logo in multiple template images, resulting in multiple product logo features from the template images.

[0112] S203, for each of the third image features, the third image feature is processed using channel attention mechanism and spatial attention mechanism to obtain the fourth image feature corresponding to the third image feature.

[0113] By utilizing channel attention and spatial attention mechanisms, the third image features are enhanced with channel attention and spatial attention to improve the representational ability of the template image features for the template image, thus obtaining the fourth image features.

[0114] For the third image feature, attention weights are inferred sequentially along the channel and spatial dimensions, and then multiplied with the template image to adaptively adjust the third feature, thereby obtaining the enhanced fourth feature, which improves the performance of the feature.

[0115] Each channel of a feature represents a dedicated detector, and the channel attention mechanism is used to focus on which features are meaningful.

[0116] After enhancing channel attention, it is meaningful to focus on features in a specific area through spatial attention mechanisms.

[0117] The channel attention and spatial attention modules can be combined in parallel or in sequence, but combining them in sequence and placing channel attention first can achieve better performance improvement.

[0118] For example, by using channel attention and spatial attention mechanisms, the advertising logo features in multiple template images can be enhanced by channel attention and spatial attention mechanisms, thereby improving the representation ability of the advertising logo features in these template images.

[0119] S204, the multiple fourth image features are fused to obtain the template image features.

[0120] Feature fusion is a computer science term introduced in 2018. It involves extracting different feature vectors from the same pattern and optimizing their combination to generate new fused features from multiple existing feature sets. Feature fusion can obtain the most differentiated information from multiple original feature sets involved in the fusion process, and can eliminate redundant information caused by correlations between different feature sets, enabling subsequent decision-making. Feature fusion can include methods such as feature addition and multiplication.

[0121] By adding multiple different fourth image features, taking a product logo as an example, the size, shape, and transparency of the product logo in multiple images to be detected may be different. By fusing multiple template images, product logo features that include all states are obtained, making the template image features more inclusive and improving tracking accuracy.

[0122] In one possible implementation, such as Figure 3 As shown, for each of the third image features, the third image feature is processed using channel attention and spatial attention mechanisms to obtain the fourth image feature corresponding to the third image feature, including:

[0123] S301, for each third image feature, perform channel attention mechanism processing on the third image feature to obtain the fifth image feature.

[0124] In the process of obtaining the fourth image feature from the third image feature, the third image feature is first processed by the channel attention mechanism to re-encode the third image feature, thereby strengthening the channel attention of the third image and obtaining the fifth image feature.

[0125] S302, the third image feature is fused with the fifth image feature to obtain the sixth image feature.

[0126] The sixth image feature is obtained by multiplying the third image feature by the fifth image feature.

[0127] S303, the sixth image feature is processed by spatial attention mechanism to obtain the seventh image feature.

[0128] The sixth image feature is re-encoded after undergoing spatial attention enhancement processing to obtain the seventh image feature.

[0129] S304, the third image feature, the seventh image feature, and the fifth image feature are fused together to obtain the fourth image feature corresponding to the third image feature.

[0130] The seventh image feature and the fifth image feature are multiplied together and then added to the third image feature. The fusion result is then recoded to obtain the fourth image feature.

[0131] In one possible implementation, the complexity of the preset feature extraction network is less than the preset complexity.

[0132] The complexity of a network can be judged by the number of network layers. Fewer network layers indicate lower network complexity, while more network layers indicate higher network complexity.

[0133] like Figure 4 As shown, MobileNet-v3 can be selected as the preset feature extraction network. MobileNet-v3 combines the depthwise separable convolutions of MobileNet-v1, the Inverted Residuals and LinearBottleneck of MobileNet-v2, and the SE module, and uses NAS (Neural Architecture Search) to search for the network configuration and parameters.

[0134] MobileNet-v3 comes in two versions: MobileNet-v3 Large and MobileNet-v3 Small, each suited to different resource requirements. MobileNet-v3 Small, compared to MobileNet-v2, improves accuracy by approximately 3.2% and reduces time by 15% on ImageNet classification tasks. MobileNet-v3 Large, compared to MobileNet-v2, improves accuracy by approximately 4.6% and reduces time by 5%. Compared to v2, MobileNet-v3 Large is 25% faster when training to the same accuracy, and also shows some improvement in segmentation algorithms.

[0135] Using the Mobilenet-v3 network as the preset feature extraction network can achieve high-speed and high-precision feature extraction.

[0136] In one possible implementation, such as Figure 5 As shown, obtaining the first image feature of the image to be detected includes:

[0137] S501: Use a preset feature extraction network to extract features from the image to be detected, and obtain the eighth image feature;

[0138] S502: The eighth image feature is processed using channel attention mechanism and spatial attention mechanism to obtain the first image feature.

[0139] The method for processing the eighth image features to obtain the first image features is the same as the method for processing the third image features to obtain the fourth image features, and will not be repeated here. By applying attention and spatial attention mechanisms to the eighth image features to obtain the first image features, the feature representation capability is improved, further enhancing tracking accuracy.

[0140] In one possible implementation, acquiring multiple template images includes:

[0141] Obtain each template image in the template sequence, wherein the template sequence stores T frames of template images, where T is a preset positive integer.

[0142] For example, when T is preset to 20, the template sequence stores 20 template images. The third image features in the 20 template images are fused and enhanced with channel attention and spatial attention mechanisms to obtain template image features to be matched with the first image features.

[0143] like Figure 6 As shown, the method further includes:

[0144] The monitoring video is sampled according to the preset sampling frequency to obtain the current template image.

[0145] If the current template image is not an image in the image sequence to be detected, delete the template image that was added earliest in the template sequence and add the current template image to the template sequence.

[0146] When the template image belongs to the image sequence to be detected, it means that the template image does not need to be updated. When the template image does not belong to the historical tracking results and is not an image in the image sequence to be detected, the earliest template image added to the template sequence between the templates is deleted, and the template image is added to supplement the template sequence.

[0147] Historical tracking results refer to the results obtained by tracking the image to be detected based on the previous template sequence when the current template image is used as the template for tracking the image to be detected.

[0148] By supplementing the template sequence with new templates, the template images in the new template sequence are re-fused and enhanced with channel attention and spatial attention mechanisms to obtain new template image features, ensuring the timeliness and accuracy of the template image features.

[0149] In one possible implementation, such as Figure 7 As shown, the method further includes:

[0150] When the image to be detected belongs to the template image, calculate the first intersection-union ratio between the location region of the template target feature in the image to be detected and the location region of the target to be tracked in the previous frame image, wherein the previous frame image is the image used for target tracking in the previous frame of the image to be detected in the video to be tracked;

[0151] If the first cross-union ratio is greater than the preset first cross-union ratio threshold, the tracking result of the target to be tracked is obtained according to the location region of the template target feature in the image to be detected;

[0152] If the first cross-union ratio is not greater than the preset first cross-union ratio threshold, the following step is triggered: according to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features.

[0153] Typically, template images are obtained by randomly selecting frames from the video to be detected; while the image to be detected includes images of all frames in a certain segment of the video to be detected; therefore, the image to be detected may contain template images.

[0154] Since the template target features in the template image are obtained directly through the feature extraction network, while the target features in the image to be detected require tracking based on the template target features, unnecessary tracking of the template target features in the template image will occur when the image to be detected contains the template image.

[0155] To reduce tracking time and avoid unnecessary tracking, a first intersection-union ratio (IUU) can be calculated between the location region of the template target feature in the current frame's image to be detected and the location region of the target to be tracked in the previous frame's image. When the first IUU is greater than a preset first IUU threshold, the tracking result of the target to be tracked can be directly obtained based on the location region of the template target feature in the current image to be detected. When the first IUU is not greater than the preset first IUU threshold, the following step is triggered: according to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features.

[0156] In one possible implementation, such as Figure 8 As shown, the method further includes:

[0157] Based on the tracking results of the target to be tracked, calculate the cross-union ratio and confidence difference between the positions of the target to be tracked in two adjacent frames of the image to be detected;

[0158] If the cross-union ratio is greater than a preset second cross-union ratio threshold and / or the confidence difference is greater than a preset confidence deviation threshold, the tracking of the target to be tracked shall be interrupted.

[0159] If the cross-union ratio (CUB) between the positions of the target to be tracked in two adjacent frames of the image to be detected is greater than the preset second CUB threshold, which means there is a large displacement between the positions of the target to be tracked in two adjacent frames of the image to be detected, it indicates that there may be a false tracking situation, and the tracking of the target to be tracked needs to be interrupted in time.

[0160] If the difference between the confidence scores of the target to be tracked in two adjacent frames of the image to be detected is greater than the preset confidence deviation threshold, that is, if there is a cliff-like difference between the confidence scores of the target to be tracked in two adjacent frames of the image to be detected, it indicates that there may be a false tracking situation, and the tracking of the target to be tracked needs to be interrupted in time.

[0161] In one possible implementation, such as Figure 9 As shown, the method further includes:

[0162] S901: The tracking result of the target to be tracked is filtered by a preset filter;

[0163] S902: Obtain the corrected tracking results.

[0164] For example, using a Kalman filter to filter the tracking results of the target can make the corrected tracking results smoother, prevent jitter in the tracking results, and ensure user comfort.

[0165] This application also provides a target tracking device, such as... Figure 10 As shown, the device includes:

[0166] Template image feature acquisition module 101 is used to acquire template image features of a template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked;

[0167] First image feature acquisition module 102, the first image feature acquisition module is used to acquire first image features of the image to be detected;

[0168] Processing module 103 is used to process the first image features according to the template image features using a cross-attention mechanism to obtain second image features, wherein the second image features include the target features of the target to be detected;

[0169] The matching module 104 is used to match the template target features with the second image features to obtain the tracking result of the target to be tracked.

[0170] In one possible implementation, the template image feature acquisition module includes:

[0171] A submodule for acquiring multiple template images;

[0172] The first feature extraction submodule is used to extract features from each template image using a preset feature extraction network to obtain multiple third image features.

[0173] The first submodule for characterization processing is used to process each of the third image features using a channel attention mechanism and a spatial attention mechanism to obtain a fourth image feature corresponding to the third image feature.

[0174] The fusion submodule is used to fuse multiple fourth image features to obtain the template image features.

[0175] In one possible implementation, the characterization processing first submodule includes:

[0176] A channel attention processing unit is used to process each third image feature using a channel attention mechanism to obtain a fifth image feature.

[0177] The first fusion unit is used to fuse the third image feature with the fifth image feature to obtain the sixth image feature;

[0178] A spatial attention processing unit is used to process the sixth image feature using a spatial attention mechanism to obtain the seventh image feature.

[0179] The second fusion unit is used to fuse the third image feature, the seventh image feature, and the fifth image feature to obtain the fourth image feature corresponding to the third image feature.

[0180] In one possible implementation, the first image feature acquisition module includes:

[0181] The second feature extraction submodule is used to extract features from the image to be detected using a preset feature extraction network to obtain the eighth image features.

[0182] The second submodule for characterization processing is used to process the eighth image features using channel attention and spatial attention mechanisms to obtain the first image features.

[0183] In one possible implementation, the acquisition submodule includes:

[0184] A template image acquisition unit is used to sample the surveillance video at a preset sampling frequency to acquire the current template image;

[0185] The image sequence extraction unit extracts an image sequence to be detected from the surveillance video, wherein the image sequence to be detected includes the image to be detected;

[0186] The template image update unit deletes the earliest added template image in the template sequence and adds the current template image to the template sequence when the current template image is not an image in the image sequence to be detected.

[0187] In one possible implementation, the apparatus further includes a filtering module, which is configured to: when the image to be detected belongs to the template image, calculate a first intersection-union ratio (IUU) between the location region of the template target feature in the image to be detected and the location region of the target to be tracked in the previous frame image, wherein the previous frame image is the previous frame of the image to be detected in the video to be tracked used for target tracking;

[0188] If the first cross-union ratio is greater than the preset first cross-union ratio threshold, the tracking result of the target to be tracked is obtained according to the location region of the template target feature in the image to be detected;

[0189] If the first cross-union ratio is not greater than the preset first cross-union ratio threshold, the following step is triggered: according to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features.

[0190] In one possible implementation, the device further includes an interruption processing module, which is configured to: calculate the cross-union ratio and confidence difference between the positions of the target to be tracked in two adjacent frames of the image to be detected based on the tracking result of the target to be tracked; and interrupt the tracking of the target to be tracked if the cross-union ratio is greater than a preset second cross-union ratio threshold and / or the confidence difference is greater than a preset confidence deviation threshold.

[0191] In one possible implementation, the device further includes a filtering module, which is used to filter the tracking result of the target to be tracked through a preset filter to obtain a corrected tracking result.

[0192] In one possible implementation, the first feature extraction submodule and the second feature extraction submodule are used to extract features with a complexity lower than a preset complexity.

[0193] This application also provides an electronic device, such as... Figure 11 As shown, it includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0194] Memory 113 is used to store computer programs;

[0195] When the processor 111 executes the program stored in the memory 113, it implements the target tracking method in any of the above embodiments.

[0196] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0197] The communication interface is used for communication between the aforementioned terminal and other devices.

[0198] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0199] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0200] In another embodiment provided in this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the target tracking methods described in the above embodiments.

[0201] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0202] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0203] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the embodiments for apparatus, electronic devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0204] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A target tracking method characterized by, include: Obtain template image features of a template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked; Obtain the first image features of the image to be detected; According to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features, wherein the second image features include the target features of the target to be detected; the template image features include: template target information and background information. The template image features are passed to the target image by calculating the cross-attention matrix between the template image and the target image, so that during the processing of the first image features of the target image, the response of the target information features is improved and the response of the background information features is suppressed; The template target features are matched with the second image features to obtain the tracking result of the target to be tracked; When the image to be detected belongs to the template image, calculate the first intersection-union ratio between the location region of the template target feature in the image to be detected and the location region of the target to be tracked in the previous frame image, wherein the previous frame image is the image used for target tracking in the previous frame of the image to be detected in the video to be tracked; If the first cross-union ratio is greater than the preset first cross-union ratio threshold, the tracking result of the target to be tracked is obtained according to the location region of the template target feature in the image to be detected; If the first cross-union ratio is not greater than the preset first cross-union ratio threshold, the following step is triggered: according to the template image features, the first image features are processed using a cross-attention mechanism to obtain the second image features.

2. The method of claim 1, wherein, The acquisition of template image features includes: Get multiple template images; A preset feature extraction network is used to extract features from each template image to obtain multiple third image features; For each of the third image features, the third image feature is processed using channel attention mechanism and spatial attention mechanism to obtain the fourth image feature corresponding to the third image feature; The template image features are obtained by fusing multiple fourth image features.

3. The method of claim 2, wherein, For each of the third image features, the third image feature is processed using channel attention and spatial attention mechanisms to obtain the corresponding fourth image feature, including: For each third image feature, channel attention mechanism is applied to the third image feature to obtain the fifth image feature; The third image feature is fused with the fifth image feature to obtain the sixth image feature; The sixth image feature is processed using a spatial attention mechanism to obtain the seventh image feature; The third image feature, the seventh image feature, and the fifth image feature are fused together to obtain the fourth image feature corresponding to the third image feature.

4. The method according to claim 1, characterized in that, The acquisition of the first image feature of the image to be detected includes: The image to be detected is extracted using a pre-defined feature extraction network to obtain the eighth image feature. The eighth image features are processed using channel attention and spatial attention mechanisms to obtain the first image features.

5. The method according to claim 2, characterized in that, The acquisition of multiple template images includes: Obtain each template image in the template sequence, wherein the template sequence stores T frames of template images, where T is a preset positive integer; The method further includes: The surveillance video is sampled according to a preset sampling frequency to obtain the current template image; Extract a sequence of images to be detected from the surveillance video, wherein the sequence of images to be detected includes the images to be detected; If the current template image is not an image in the image sequence to be detected, delete the template image that was added earliest in the template sequence and add the current template image to the template sequence.

6. The method according to claim 1, characterized in that, The method further includes: Based on the tracking results of the target to be tracked, calculate the second cross-union ratio and confidence difference between the position regions of the target to be tracked in two adjacent frames of the image to be detected; If the second cross-union ratio is greater than a preset second cross-union ratio threshold and / or the confidence difference is greater than a preset confidence deviation threshold, the tracking of the target to be tracked shall be interrupted.

7. The method according to claim 1, characterized in that, The method further includes: The tracking results of the target to be tracked are filtered by a preset filter to obtain a corrected tracking result.

8. The method according to claim 2 or 4, characterized in that, The complexity of the preset feature extraction network is less than the preset complexity.

9. A target tracking device, comprising: A template image feature acquisition module is used to acquire template image features of a template image, wherein the template image includes a target to be tracked, and the template image features include template target features of the target to be tracked; A first image feature acquisition module is used to acquire first image features of the image to be detected. The processing module is configured to process the first image features according to the template image features using a cross-attention mechanism to obtain second image features, wherein the second image features include the target features of the target to be detected; the template image features include template target information and background information, and the template image features are passed to the target image by calculating the cross-attention matrix between the template image and the target image, so as to improve the response of the target information features and suppress the response of the background information features during the processing of the first image features of the target image; A matching module is used to match the template target features with the second image features to obtain the tracking result of the target to be tracked; A filtering processing module is configured to: when the image to be detected belongs to the template image, calculate a first intersection-union ratio (IUGR) between the location region of the template target feature in the image to be detected and the location region of the target to be tracked in the previous frame image, wherein the previous frame image is the image used for target tracking in the previous frame of the image to be detected in the video to be tracked; when the first IUGR is greater than a preset first IUGR threshold, obtain the tracking result of the target to be tracked based on the location region of the template target feature in the image to be detected; when the first IUGR is not greater than the preset first IUGR threshold, trigger the execution step: process the first image features according to the template image features using a cross-attention mechanism to obtain second image features.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-8.

Images