Multi-target tracking method, device, equipment and storage medium

Abstract

The present application discloses a multi-target tracking method, apparatus, device and storage medium, which relates to the field of computer vision technology. The method realizes target association by fusing feature maps and adaptively embedding updated target detection features, thereby improving the accuracy of the multi-target tracker. The method comprises: obtaining a fused feature map of adjacent image frames, wherein the fused feature map is obtained by fusing pyramid feature maps with set weights; performing target detection on the fused feature map to obtain target detection features; updating the target detection features using embedded identity features to obtain embedded updated target detection features, wherein the embedded identity features are extracted from a preset image frame, wherein the preset image frame is an image frame with a preceding position in the adjacent image frames; performing target association on the embedded updated target detection features using an adaptive filtering mode to obtain target correspondences in adjacent image frames.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a multi-target tracking method, apparatus, device, and storage medium. Background Art

[0002] The main task of multi-target tracking is to output the trajectories of all targets in a given video while maintaining the identity of each target. Tracked targets can be pedestrians, vehicles, or other objects. With the advancement of computer vision technology, multi-target tracking has been widely used in numerous fields, such as intelligent video surveillance and human-computer interaction. Furthermore, multi-target tracking is the foundation of computer vision tasks such as pose estimation, behavior recognition, behavior analysis, and video analysis.

[0003] Traditional multi-target tracking methods mainly include Markov decision making, joint probability association, and particle filtering. These methods have large predicted position errors and are not robust to occlusion and interference from similar objects. With the widespread application of deep learning in the field of computer vision, multi-target tracking can be achieved based on deep learning. Specifically, a multi-target tracker identifies objects in each frame of a video sequence, extracts target features, and then associates them based on the feature data. However, due to factors such as irregular motion caused by camera movement and view changes, it is difficult for multi-target trackers to accurately extract and match target features. This is especially true for drone videos shot at multiple scales. Multi-target trackers are unable to extract enough features for reliable detection and tracking, which affects the accuracy of multi-target tracking. Summary of the Invention

[0004] In view of this, the present application provides a multi-target tracking method, apparatus, device and storage medium, the main purpose of which is to solve the problem in the existing technology that for drone videos shot at multiple scales, the multi-target tracker cannot extract enough features for reliable detection and tracking, affecting the accuracy of multi-target tracking.

[0005] According to a first aspect of the present application, a multi-target tracking method is provided, comprising:

[0006] Obtaining a fused feature map of adjacent image frames, where the fused feature map is obtained by fusing the pyramid feature maps with set weights;

[0007] Performing target detection on the fused feature map to obtain target detection features, wherein the target detection features include at least target boundary features, target identity features, and target probability features. The target identity features are feature vectors used to distinguish targets. The target probability features are visual image features used to locate targets. The visual image features represent target probability by color depth.

[0008] Updating the target detection feature using an embedded identity feature to obtain an embedded updated target detection feature, wherein the embedded identity feature is extracted from the preset image frame, and the preset image frame is an image frame that is earlier in sequence among adjacent image frames;

[0009] An adaptive association mode is used to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames.

[0010] Furthermore, obtaining a fusion feature map of adjacent image frames includes:

[0011] Using the pre-trained feature extraction model, the feature maps obtained by sampling adjacent image frames using different paths are superimposed in multiple layers to obtain superimposed image features;

[0012] In the process of multi-layer superposition, the image features with consistent resolution in different layers are added and fused through lateral connections to obtain a pyramid feature map;

[0013] The pyramid feature maps are weightedly fused according to set weights to obtain fused feature maps of adjacent image frames.

[0014] Furthermore, before performing target detection on the fused feature map to obtain target detection features, the method further includes:

[0015] A feature detection model is obtained by training the model using a depth-wise separable convolution consisting of channel-by-channel convolution and point-by-point convolution. One convolution kernel of the channel-by-channel convolution is responsible for one channel. The number of input channels and output channels corresponding to the feature detection model are the same. The fused feature map is weighted summed over the number of channels through a convolution operation, and the corresponding output is a target detection feature with the same number of convolution kernels.

[0016] Accordingly, a pre-trained feature detection model is used to perform target detection on the fused feature map to obtain target detection features.

[0017] Furthermore, before updating the target detection feature by using the embedded identity feature to obtain the embedded updated target detection feature, the method further includes:

[0018] Taking an image frame that is located earlier in sequence among the adjacent image frames as a preset image frame, and extracting the embedded identity features in the preset image frame;

[0019] Accordingly, the updating of the target detection feature by using the embedded identity feature to obtain the embedded updated target detection feature includes:

[0020] Selecting a preset number of key position points in the target probability feature, wherein the key position points are position points in the target probability feature whose probability ranking is before a preset value;

[0021] Extracting a preset number of key point features from the embedded identity feature according to the key point positions in the target probability feature;

[0022] After compressing the key point features, the target detection features are updated using the compressed key point features to obtain embedded updated target detection features.

[0023] Furthermore, after compressing the key point features, the target detection features are updated using the compressed key point features to obtain embedded updated target detection features, including:

[0024] performing an association operation on the adjacent image frames to obtain an attention weight for feature enhancement, wherein the attention weight is used to guide the key point features that the subsequent image frame in the adjacent image frames should focus on based on the image frame in the preceding image frame;

[0025] After compressing the key point features, weighted summing is performed on the compressed key point features according to the feature-enhanced attention weights to obtain the attention features that are in the front position in the adjacent image frames;

[0026] The target detection feature is updated using the attention feature to obtain an embedded updated target detection feature.

[0027] Furthermore, the updating of the target detection feature using the attention feature to obtain the embedded updated target detection feature includes:

[0028] Using the attention feature, feature association is performed on an image frame that comes earlier in sequence and an image frame that comes later in sequence in the adjacent image frames to obtain association features of the adjacent image frames;

[0029] The target detection feature is updated according to the associated feature to obtain an embedded updated target detection feature.

[0030] Furthermore, the use of the adaptive association mode to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames includes:

[0031] Calculating target matching coefficients between adjacent image frames using Kalman filtering;

[0032] If the target matching coefficient is greater than a preset threshold, performing target association on the embedded updated target detection features using a first association mode to obtain target correspondences in adjacent image frames;

[0033] If the target matching coefficient is less than or equal to a preset threshold, a second association mode is used to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames.

[0034] According to a second aspect of the present application, a multi-target tracking device is provided, comprising:

[0035] an acquiring unit, configured to acquire a fused feature map of adjacent image frames, wherein the fused feature map is obtained by fusing the pyramid feature maps with set weights;

[0036] a detection unit, configured to perform target detection on the fused feature map to obtain target detection features, wherein the target detection features include at least target boundary features, target identity features, and target probability features, wherein the target identity features are feature vectors used to distinguish targets, and the target probability features are visual image features used to locate targets, wherein the visual image features represent target probability by color depth;

[0037] an updating unit, configured to update the target detection feature using an embedded identity feature to obtain an embedded updated target detection feature, wherein the embedded identity feature is extracted from the preset image frame, and the preset image frame is an image frame that is earlier in sequence among adjacent image frames;

[0038] The association unit is used to perform target association on the embedded updated target detection features using an adaptive association mode to obtain target correspondences in adjacent image frames.

[0039] Furthermore, the acquisition unit is specifically configured to:

[0040] Using the pre-trained feature extraction model, the feature maps obtained by sampling adjacent image frames using different paths are superimposed in multiple layers to obtain superimposed image features;

[0041] In the process of multi-layer superposition, the image features with consistent resolution in different layers are added and fused through lateral connections to obtain a pyramid feature map;

[0042] The pyramid feature maps are weightedly fused according to set weights to obtain fused feature maps of adjacent image frames.

[0043] Furthermore, the device further comprises:

[0044] A training unit is configured to perform model training using depth-separable convolution consisting of channel-by-channel convolution and point-by-point convolution before performing target detection on the fused feature map to obtain target detection features, to obtain a feature detection model, wherein one convolution kernel of the channel-by-channel convolution is responsible for one channel, the number of input channels and the number of output channels corresponding to the feature detection model are the same, and the fused feature map is weightedly summed on the number of channels through a convolution operation, and the corresponding target detection features are outputted with the same number of convolution kernels;

[0045] Correspondingly, the detection unit is specifically configured to perform target detection on the fused feature map using a pre-trained feature detection model to obtain target detection features.

[0046] Furthermore, the device further comprises:

[0047] an extraction unit configured to, before updating the target detection feature using the embedded identity feature to obtain the embedded updated target detection feature, use a preceding image frame in the adjacent image frames as a preset image frame and extract the embedded identity feature in the preset image frame;

[0048] Accordingly, the updating unit is specifically configured to:

[0049] Selecting a preset number of key position points in the target probability feature, wherein the key position points are position points in the target probability feature whose probability ranking is before a preset value;

[0050] Extracting a preset number of key point features from the embedded identity feature according to the key point positions in the target probability feature;

[0051] After compressing the key point features, the target detection features are updated using the compressed key point features to obtain embedded updated target detection features.

[0052] Furthermore, the updating unit is further configured to:

[0053] performing an association operation on the adjacent image frames to obtain an attention weight for feature enhancement, wherein the attention weight is used to guide the key point features that the subsequent image frame in the adjacent image frames should focus on based on the image frame in the preceding image frame;

[0054] After compressing the key point features, weighted summing is performed on the compressed key point features according to the feature-enhanced attention weights to obtain the attention features that are in the front position in the adjacent image frames;

[0055] The target detection feature is updated using the attention feature to obtain an embedded updated target detection feature.

[0056] Furthermore, the updating unit is further configured to:

[0057] Using the attention feature, feature association is performed on an image frame that comes earlier in sequence and an image frame that comes later in sequence in the adjacent image frames to obtain association features of the adjacent image frames;

[0058] The target detection feature is updated according to the associated feature to obtain an embedded updated target detection feature.

[0059] Furthermore, the association unit is specifically used to:

[0060] Calculating target matching coefficients between adjacent image frames using Kalman filtering;

[0061] If the target matching coefficient is greater than a preset threshold, performing target association on the embedded updated target detection features using a first association mode to obtain target correspondences in adjacent image frames;

[0062] If the target matching coefficient is less than or equal to a preset threshold, a second association mode is used to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames.

[0063] According to a third aspect of the present application, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect when executing the computer program.

[0064] According to a fourth aspect of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0065] The present application provides a multi-target tracking method, apparatus, device and storage medium. Compared with the current method of multi-target tracking based on deep learning in the prior art, the present application obtains a fused feature map of adjacent image frames, where the fused feature map is obtained by fusing the pyramid feature map with set weights; performs target detection on the fused feature map to obtain target detection features, where the target detection identification includes at least target boundary features, target identity features and target probability features, where the target identity features are feature vectors used to distinguish targets, and the target probability features are visual image features used to locate targets, where the visual image features represent target probabilities by color depth; uses embedded identity features to update target detection features to obtain embedded updated target detection features, where the embedded identity features are extracted from a preset image frame, where the preset image frame is the image frame with the preceding position in the adjacent image frames; uses an adaptive filtering mode to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames. The entire process shortens the information path based on pyramid features and constructs a fused feature map through weighted fusion, which contains semantic features at different scales and position information. This fused feature map can provide sufficient features to the multi-target tracker, improving its accuracy. Because target detection features change with the position of the acquisition device, the target detection features can be adaptively updated by learning the embedded identity features of adjacent image frames. Adaptive filtering patterns are then used to accurately associate targets, improving multi-target tracking capabilities.

[0066] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0068] Figure 1 1 is a flow chart of a multi-target tracking method in one embodiment of the present application;

[0069] Figure 2 yes Figure 1 A schematic flow chart of a specific implementation of step 101;

[0070] Figure 3 is a flowchart of a multi-target tracking method in another embodiment of the present application;

[0071] Figure 41 is a flow chart of a multi-target tracking method in another embodiment of the present application;

[0072] Figure 5 yes Figure 1 A schematic flow chart of a specific implementation of step 104;

[0073] Figure 6 This is a structural block diagram of a multi-target tracking method in one embodiment of the present application;

[0074] Figure 7 is a flowchart of a multi-target tracking method in another embodiment of the present application;

[0075] Figure 8 is a pyramid feature map in one embodiment of the present application;

[0076] Figure 9 This is a schematic diagram of feature fusion in one embodiment of the present application;

[0077] Figure 10 Schematic diagram of embedding and updating target detection features in one embodiment of the present application;

[0078] Figure 11 Schematic diagram of gradient-balanced focus loss in one embodiment of the present application;

[0079] Figure 12 This is a schematic structural diagram of a multi-target tracking device according to an embodiment of the present application;

[0080] Figure 13 The figure is a schematic diagram of the device structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The present invention will now be discussed with reference to several exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the present invention, rather than to imply any limitation on the scope of the present invention.

[0082] As used herein, the term "including" and variations thereof are to be interpreted as open-ended terms meaning "including, but not limited to." The term "based on" is to be interpreted as "based, at least in part, on." The terms "one embodiment" and "an embodiment" are to be interpreted as meaning "at least one embodiment." The term "another embodiment" is to be interpreted as meaning "at least one other embodiment."

[0083] In related technologies, due to factors such as irregular motion caused by camera movement and view changes, it is difficult for multi-target trackers to accurately extract and match target features. Especially for drone videos shot at multiple scales, multi-target trackers cannot extract enough features for reliable detection and tracking, affecting the accuracy of multi-target tracking.

[0084] In order to solve this problem, this embodiment provides a multi-target tracking method, such as Figure 1 As shown, the following steps are included:

[0085] 101. Obtain a fusion feature map of adjacent image frames.

[0086] Adjacent image frames refer to two temporally consecutive and sequentially adjacent image frames in a video sequence or image stream. For example, in a video captured by a drone, the image frames are arranged in chronological order, so the nth and n+1th frames are considered adjacent image frames.

[0087] In this embodiment, the fused feature map is obtained by fusing the pyramid feature maps with set weights. The set weights here refer to the weights assigned to the feature maps at different layers in the pyramid feature map. Specifically, different weights can be assigned to feature maps at different layers based on the characteristics and importance of the objects in adjacent image frames. For example, for scenes with many small objects, the weights of the bottom-layer feature maps can be appropriately increased; for scenes that require attention to the overall structure and contextual information, the weights of the upper-layer feature maps can be increased.

[0088] Specifically, an image pyramid can be constructed for adjacent image frames. The image pyramid is a multi-scale image representation method that obtains image layers of different resolutions by performing multiple downsampling and filtering operations on the original image. Then, a neural network is used to extract features from each layer of the pyramid to extract representative features from images of different scales. For each layer of the feature map, the feature maps corresponding to the two frames of images can be weighted and summed according to the weights to obtain the fused feature map. For example, suppose two adjacent frames of images and No. The layer pyramid feature maps are and , the corresponding weight is and , then the fused The layer feature map is , ,By fusing the feature maps of all layers, we finally obtain the fused feature maps of adjacent image frames.

[0089] 102. Perform target detection on the fused feature map to obtain target detection features.

[0090] In this embodiment, the target detection identification includes at least target boundary features, target identity features and target probability features. The target identity features are feature vectors used to distinguish targets. Here, the target identity features can be ID features used to characterize the target identity. The target probability features are visual image features used to locate the target. The visual image features characterize the target probability through color depth. Here, the visual image features can be heat maps used to characterize the location probability.

[0091] Specifically, when performing object detection on the fused feature map, a pre-trained object detection model can be trained using a large-scale, multi-target image dataset that includes the target's bounding box annotations, category annotations, and unique ID annotations. During training, the model learns to extract target features from the feature fusion map and performs supervised learning based on this annotation information. The model's parameters are adjusted to minimize the loss function, enabling the model to accurately predict the target's bounding box size, category, and ID features. The object detection model then performs object detection on the feature map, obtaining target boundary features, target identity features, and target probability features.

[0092] For object boundary features, object detection models typically use regression algorithms to predict the position and size of the bounding box. For example, by processing the features of the candidate region, fully connected layers and linear regression functions are used to predict the offset and scale change of the bounding box relative to the candidate region, thereby obtaining the final bounding box coordinates. For object probability features, the object detection model can further process the features within the bounding box to generate visual image features. For heat maps, the average or weighted average of the features within the bounding box can be calculated and used as the heat value of the region. The heat values ​​corresponding to all bounding boxes are plotted on a heat map of the same size as the image. For object identity features, the identity features of the object can be learned by designing an appropriate network structure during object detection model training. For example, a dedicated branch can be added to the model to extract the identity feature vector of the object. This branch can be composed of multiple convolutional layers and fully connected layers, taking the feature fusion map as input and outputting a fixed-length identity feature vector. During training, by comparing the identity feature vectors of different objects, the model can learn the differences between different objects and thus uniquely identify them.

[0093] 103. Update the target detection feature using the embedded identity feature to obtain an embedded updated target detection feature.

[0094] In this embodiment, the embedded identity feature is extracted from a preset image frame. Here, the embedded identity feature is a feature vector that can indicate the characteristic attributes or identity of the target in the image. It can be obtained by mapping the target information in the image frame into a low-dimensional vector space. The preset image frame is the image frame that comes first in the adjacent image frames.

[0095] It can be understood that the embedded identity features have the ability to uniquely identify the target in the image frame. Even if the target appears in different image frames, its embedded identity features are relatively stable, so as to facilitate tracking and identification of the target. For example, in pedestrian tracking scenarios, each person has unique appearance, posture and other features. By extracting the embedded identity features, these features can be converted into a vector, so that the feature vectors of the same pedestrian in different frames have a high similarity, while the feature vectors of different pedestrians have a large difference.

[0096] Specifically, the embedded identity features can be used to calculate the feature similarity between the preceding and succeeding frames in adjacent image frames. Based on the similarity results, the adjacent frames are then feature-correlated. The associated image frame features are then used to update the target detection features, yielding the embedded updated target detection features. The target feature update process can use either a weighted average update or a confidence update.

[0097] For the weighted average update method, if the target features in adjacent image frames are successfully associated, the weighted average method can be used to update the target detection features. Assume that the embedded identity features of the image frame that is in the previous position in the adjacent image frames are , the detection feature of the current frame target is , the updated target detection feature is , then the update formula is ,in, is the weight coefficient, and its value range is . The value of can be adjusted adaptively based on factors such as the target's motion state and the degree of appearance change. For example, when the target moves slowly and its appearance changes little, A larger value can be used when the target moves faster or its appearance changes more. A smaller value can be used.

[0098] For confidence updates, the target detection features can be updated based on the target detection confidence information. If the target detection confidence in the current image frame is high, indicating that the detection result is relatively reliable, the weight of the embedded identity features corresponding to the image frame with the preceding sequence can be appropriately reduced. Conversely, if the confidence is low, the weight of the embedded identity features corresponding to the image frame with the preceding sequence can be increased to improve the accuracy of target detection.

[0099] 104. Perform target association on the embedded updated target detection features using an adaptive association mode to obtain target correspondences in adjacent image frames.

[0100] In this embodiment, the target motion can be linear or nonlinear, influenced by the motion pattern of the acquisition device. For example, nonlinear motion can be generated by coupling between the acquisition device and the target itself. Under different motion patterns, the target in the video captured by the acquisition device will exhibit different motion states. Traditional Kalman filters have difficulty processing such irregular motion patterns. This embodiment uses an adaptive filtering mode to process the image frame features of the target in different motion states, enabling accurate target association.

[0101] Specifically, the target matching value between two adjacent image frames can be calculated, and the filtering mode suitable for target association of the image frames can be determined according to the target matching value, and the corresponding filtering mode can be used to further perform target association on the image frames.

[0102] For example, the target's motion state includes three types. The first motion state is applicable to the A filtering mode, the second motion state is applicable to the B filtering mode, and the third motion state is applicable to the C filtering mode. In the first motion state, the target matching value is less than m, in the second motion state, the target value is greater than m and less than n, and in the third motion state, the target value is greater than n, where m is less than n. Accordingly, if the target matching value is less than m, the A filtering mode is used for target association. If the target matching value is greater than m and less than n, the B filtering mode is used for target association. If the target matching value is greater than n, the A filtering mode is used for target association.

[0103] The multi-target tracking method provided by the embodiment of the present application is compared with the multi-target tracking method based on deep learning in the current prior art. The present application obtains a fusion feature map of adjacent image frames. The fusion feature map is obtained by fusion of pyramid feature maps with set weights; the fusion feature map is subjected to target detection to obtain target detection features. The target detection identification includes at least target boundary features, target identity features, and target probability features. The target identity features are feature vectors used to distinguish targets. The target probability features are visual image features used to locate targets. The visual image features represent target probability by color depth; the target detection features are updated using embedded identity features to obtain embedded updated target detection features. The embedded identity features are extracted from a preset image frame. The preset image frame is the image frame with the previous position in the adjacent image frames; the embedded updated target detection features are associated with targets using an adaptive filtering mode to obtain target correspondence in adjacent image frames. The entire process shortens the information path based on the pyramid features, and constructs a fusion feature map containing semantic features and position information of different scales through weighted fusion. The fusion feature map can provide sufficient features to the multi-target tracker to improve the accuracy of the multi-target tracker. Since the target detection features will change with the position of the acquisition device, by learning the embedded identity features of adjacent image frames, the target detection features can be adaptively updated, and the adaptive filtering mode can be used to accurately complete the target association, thereby improving the ability of multi-target tracking.

[0104] In practical application scenarios, the pyramid feature map is a hierarchical feature representation formed by extracting and fusing features of image frames at different scales. As the number of network layers increases, the receptive field gradually increases and can capture feature information at different scales. In the above embodiment, specifically, Figure 2 As shown, step 101 includes the following steps:

[0105] 201. Using the pre-trained feature extraction model, feature maps obtained by sampling adjacent image frames using different paths are superimposed in multiple layers to obtain superimposed image features.

[0106] 202. In the process of multi-layer superposition, image features with consistent resolution in different layers are added and fused through lateral connections to obtain a pyramid feature map.

[0107] 203. Perform weighted fusion on the pyramid feature maps according to set weights to obtain fused feature maps of adjacent image frames.

[0108] In this embodiment, a pre-trained feature extraction model is used to sample features at different resolutions from adjacent image frames. The sampling path can be top-down, bottom-up, and top-down again. During the multi-layer stacking process, lateral connections and convolutional fusion are used to perform weighted fusion of image features to produce a pyramid feature map, which is composed of image features that are not scaled.

[0109] Specifically, the top-down sampling of feature maps typically begins with high-level features in the network, which have stronger semantic information but lower resolution. Upsampling (such as deconvolution) gradually increases the resolution of high-level feature maps to match that of low-level feature maps. This process allows the semantic information of high-level features to propagate to higher-resolution layers, facilitating the detection of small objects. Low-level features may contain richer location information for small objects, but weaker semantic information, which can be compensated by the semantic information of high-level features.

[0110] Specifically, in the process of bottom-up feature map sampling, starting from the bottom layer of the network, the underlying feature maps have higher resolution and can provide accurate object location information, but relatively less semantic information. Through operations such as pooling or convolution, the resolution of the feature maps is gradually reduced, while the semantic abstraction of the features is increased. This allows the network to capture a wider range of semantic information, which is beneficial for detecting large objects.

[0111] Specifically, in the process of sampling the feature map from top to bottom again, performing the top-down operation again can further refine the features, propagate higher-level semantic information to lower layers, strengthen the fusion and refinement of target features at different scales, and make the final feature map richer and more accurate in both semantics and position information.

[0112] It is understandable that in both the top-down and bottom-up processes, feature maps of the same resolution can be fused through lateral connections. Horizontal connections can directly combine the positional information of low-level features with the semantic information of high-level features, avoiding information loss during transmission and effectively preserving both the details and overall information of the target.

[0113] Accordingly, after feature map fusion, convolution is typically used to process the fused feature maps. Convolution allows for feature extraction and fusion without excessive computational overhead, enabling features from different sources to interact effectively and extract more representative features. Through appropriate convolution kernel design and parameter adjustment, convolution fusion can effectively integrate semantic and positional information while mitigating the effects of noise and redundant information.

[0114] Specifically, when performing weighted fusion on pyramid feature maps, the weights are typically set based on actual needs. Initial weights can be determined through experimentation, experience, or based on some prior knowledge. For each level of feature map, the corresponding weight is multiplied to obtain a weighted feature map. The weighted feature maps of each level are then added together to obtain a fused feature map. When adding, it is necessary to ensure that the feature maps at different levels have the same size and number of channels. If they are not, some adjustments may need to be performed first, such as cropping, padding, or convolution transformations, to ensure that they can be added correctly.

[0115] The above-mentioned fused image features adaptively learn the spatial weights of the fusion of feature maps of each scale based on the pyramid feature map, which can make full use of features of different scales and improve the multi-scale target detection capability.

[0116] In the above embodiment, further, Figure 3 As shown, before step 102, the above Figure 1 The multi-target tracking method in further includes the following steps:

[0117] 301. Deep separable convolution consisting of channel-by-channel convolution and point-by-point convolution is used for model training to obtain a feature detection model.

[0118] Correspondingly, in step 102, a pre-trained feature detection model is used to perform target detection on the fused feature map to obtain target detection features.

[0119] In this embodiment, each convolution kernel in the channel-by-channel convolution is responsible for one channel. The number of input channels and output channels corresponding to the feature detection model are the same. The fused feature map is weighted and summed over the number of channels through the convolution operation, and the corresponding output is the same number of target detection features as the number of convolution kernels. The benefit of this is that the computational effort caused by the simultaneous operation of a large number of convolution kernels on multiple channels in the convolution operation is greatly reduced.

[0120] The resulting fused feature map can be processed using the convolution operation in the feature detection model. This convolution operation first performs channel-by-channel convolution on each channel of the fused feature map, extracting local feature information from each channel. Next, point-by-point convolution is performed. Because point-by-point convolution uses a 1×1 convolution kernel, it linearly combines the convolved feature maps in the channel dimension, integrating information from different channels.

[0121] After performing channel-by-channel and point-by-point convolution operations, the final target detection features are obtained by performing a weighted summation operation on the fused feature map across the number of channels. Weighted summation here means assigning corresponding weights and performing a summation calculation based on the degree of effect of the convolution kernel on the features of different channels during the operation. The number of target detection features outputted is consistent with the number of convolution kernels. These features contain comprehensive information from each channel in the fused feature map and are weighted, making them more effective for target detection tasks.

[0122] Correspondingly, the fused feature map is input into the pre-trained feature detection model. Since the feature detection model has learned how to identify and extract target-related information from image features during the previous training process, different branches or output layers of the feature detection model will predict the target detection features respectively and output target boundary features, target identity features and target probability features.

[0123] Specific feature detection models use regression to predict the object's boundary features, namely its position and size in the image. These are typically a set of coordinates that define the object's bounding box. In other words, feature detection models learn the mapping between the object's shape, position, and other features and the bounding box coordinates.

[0124] The target identity features described above can be heatmaps representing probabilities, indicating the likelihood of the target being present at each location in the image. Accordingly, the feature detection model outputs a heatmap with one or more channels, each of which may correspond to a different target category or attribute. High values ​​in the heatmap indicate a high probability of the target being present, while low values ​​indicate a low probability of the target being present. Heatmaps provide a visual understanding of the distribution of targets in an image.

[0125] To distinguish different object instances, identity features can help accurately track and identify each object in multi-object scenarios. Feature detection models predict the identity of each object, typically by learning unique features of the object, such as its appearance, texture, etc.

[0126] In multi-target tracking scenarios, different targets may have similar appearances or features, making them difficult to distinguish accurately. Embedding identity features can capture the unique identity information of the target, provide a unique identifier for each target, and significantly enhance the ability to distinguish different targets. Furthermore, Figure 4 As shown, before step 103, the above Figure 1 The multi-target tracking method in further includes the following steps:

[0127] 401. Use an image frame that comes earlier in sequence among the adjacent image frames as a preset image frame, and extract embedded identity features in the preset image frame.

[0128] Accordingly, step 103 includes the following steps:

[0129] 402. Select a preset number of key location points from the target probability features.

[0130] 403. Extract a preset number of key point features from the embedded identity features according to the key point positions in the target probability features.

[0131] 404. After compressing the key point features, the object detection features are updated using the compressed key point features to obtain embedded updated object detection features.

[0132] In this embodiment, a convolutional neural network or a pre-trained deep learning model with identity feature extraction capabilities can be used to input the preset image frame into the deep learning model. The last layer or a specific intermediate layer of the deep learning model will output a feature vector of a fixed length, which is the extracted embedded identity feature.

[0133] Key location points are locations in the target probability feature whose probability ranking precedes a preset value, corresponding to key point features extracted from the embedded identity feature with the preset value. For example, k location points are selected by probability feature ranking, and k key point features are extracted from the embedded identity feature. Specifically, the embedded identity feature can be input into a pre-trained model, and the key point features of the embedded identity feature can be extracted using the model. Alternatively, a template library containing different key point features can be constructed, and the embedded identity feature can be matched with templates in the template library. The key point features of the embedded identity feature are determined based on the key point positions of the most matching template.

[0134] Specifically, in the process of updating the target detection features, an association operation is performed on adjacent image frames to obtain feature-enhanced attention weights. The attention weights are used to guide the key point features that the image frames with a preceding order in the adjacent image frames should pay attention to based on the image frames with a preceding order in the adjacent image frames. After compressing the key point features, the compressed key point features are weighted and summed according to the feature-enhanced attention weights to obtain the attention features with a preceding order in the adjacent image frames. The target detection features are updated using the attention features to obtain embedded updated target detection features.

[0135] Specifically, the attention feature can be used to associate the features of the image frame that comes first in the adjacent image frames with the image frame that comes later in the adjacent image frames to obtain the associated features of the adjacent image frames; the target detection feature is updated according to the associated features to obtain the embedded updated target detection feature.

[0136] In this embodiment, the target detection feature update process actively extracts features from the preceding image frame and the following image frame in adjacent image frames, and associates them to obtain feature-enhanced attention weights. The specific association process comprehensively considers factors such as feature similarity, positional relationship, and motion trend. For example, if a target in the preceding image frame moves, changes its posture, or is partially occluded in the following image frame, a complex matching and inference mechanism can be used to determine whether the target in the following image frame is the same as that in the preceding image frame.

[0137] Furthermore, each element of the compressed key point feature is weighted according to the attention weight. Specifically, each weight value in the attention weight matrix is ​​multiplied by the corresponding key point feature element, and then all the multiplied results are summed. Through the weighted summation method, key point features with higher importance will occupy a larger proportion in the attention feature, highlighting information that is important for target detection. Since the target detection features have different levels of features, in the shallow features, the attention features can help enhance the details and edge information of the target, and in the deep features, the attention features focus more on integrating semantic information and improving the ability to judge the target category and overall structure. By updating the target detection features at multiple levels, the target detection features can have the ability to express targets at different scales.

[0138] In practical application scenarios, considering the dynamic change characteristics of the target in adjacent image frames, in order to track the dynamic change characteristics of the target in real time, the adaptive filtering mode can be used to effectively extract and associate the target features. Specifically, Figure 5 As shown, step 104 includes the following steps:

[0139] 501. Calculate target matching coefficients between adjacent image frames using Kalman filtering.

[0140] 502. If the target matching coefficient is greater than a preset threshold, perform target association on the embedded updated target detection features using a first association mode to obtain target correspondences in adjacent image frames.

[0141] 503. If the target matching coefficient is less than or equal to a preset threshold, perform target association on the embedded updated target detection features using a second association mode to obtain target correspondences in adjacent image frames.

[0142] In this embodiment, the Kalman filter is an algorithm that uses a linear system state equation and input / output observation data to optimally estimate the system state. Specifically, when calculating the target matching coefficient between adjacent image frames, the image frame with the later position in the adjacent image frames is used as the previous image frame, and the image frame with the later position in the adjacent image frames is used as the current image frame. The target state and covariance of the current image frame are first predicted based on the target state and state transition matrix of the previous image frame. Then, after obtaining the target measurement value of the current image frame, the measurement residual and residual covariance are calculated to obtain the Kalman gain, which is used to update the target state and covariance. Finally, by calculating the Euclidean distance or Mahalanobis distance between the target states of adjacent image frames, the distance is converted into a target matching coefficient. The smaller the distance, the larger the target matching coefficient. The target matching coefficient can be used to measure the degree of target matching.

[0143] It should be noted that in drone video sequences, object motion is no longer linear, but rather nonlinear, resulting from the coupling of the drone's motion with the object itself. In normal motion mode, the drone flies smoothly and normally, and objects in the video can be seen as moving approximately linearly. In abnormal motion mode, the drone may suddenly rotate or accelerate, resulting in nonlinear motion of objects in the video. To address this irregular motion, target association can be performed by adaptively switching between different association modes based on the drone's motion pattern.

[0144] Specifically, if the target matching coefficient is greater than a preset threshold, it indicates that the drone's motion is normal. A first association mode can be used to associate the embedded and updated target detection features to obtain target correspondences in adjacent image frames. The first association mode can use the intersection-over-union ratio for target association. This mode measures the degree of overlap and similarity between two targets by calculating the ratio of the intersection and union areas of two bounding boxes, achieving target association matching between different frames or different detection results. If the target matching coefficient is less than or equal to the preset threshold, it indicates that the drone's motion is abnormal. A second association mode can be used. This second association mode uses local relationship filtering for association. This association mode captures the target's essential characteristics and its relative position in the scene by modeling and analyzing local features and relationships. This ensures that even if the target undergoes some posture changes or small displacements between different image frames, its local features and surrounding local relationships remain stable and coherent, allowing association between targets across different frames based on this information.

[0145] In actual application scenarios, Figure 6A structural block diagram of a multi-target tracking method is provided, comprising a feature extraction section, a detection section, and a tracking section. The main purpose is to track target correspondences from adjacent image frames in a given video sequence, without obtaining the identity information of each target. The targets can be pedestrians, vehicles, or other objects in the image frames. The feature extraction section adaptively learns the spatial weights for fusion of feature maps at each scale based on the pyramid feature map, performs feature fusion on adjacent image frames, and obtains a fused feature map. This fused feature map can fully utilize features at different scales to enhance the detection capability of multi-scale targets. The detection section uses a deep classifiable convolutional network framework to perform target detection on the fused feature map, obtaining target detection features. The tracking section includes an embedded feature update module and an adaptive association module. The embedded feature update module can use the embedded identity features learned from adjacent image frames to update the target detection features, so that the updated target detection features are fused with the target identity features, thereby enhancing target tracking capability. For the adaptive association module, different association modes can be adaptively selected according to the UAV motion mode. For UAVs with normal motion modes, the IoU association mode can be used, that is, the intersection over union ratio is used to perform target association on the embedded updated target detection features. For UAVs with abnormal motion modes, the local relation filtering association mode is used, that is, the local relation filtering is used to perform target association on the embedded updated target detection features to complete the target association of adjacent image frames.

[0146] Accordingly, combined Figure 6 The structural block diagram of the multi-target tracking method in [1] provides a flow chart of the multi-target tracking method. For details, see Figure 7 As shown in the figure, adjacent image frames are first input into the convolutional neural network of the target detection algorithm. Pyramid feature maps of different scales are constructed through additive fusion in the convolutional neural network. Feature fusion is performed with certain weights to obtain a fused feature map containing multi-scale features. The fused feature map is then input into the detection part for target detection, obtaining the target bounding box, heat map, and identity features. The target bounding box, heat map, and identity features are further embedded and updated to obtain the updated target bounding box, heat map, and identity features. Finally, the updated target bounding box, heat map, and identity features are associated with the target using an adaptive association mode, outputting the target size, target identity, and category information.

[0147] Correspondingly, in the process of constructing the pyramid feature map, such as Figure 8 As shown in the figure, the convolutional neural network uses upsampling to enlarge the small feature map at the top layer to the same size as the feature map of the previous layer. Then, through a bottom-up process, semantic information is transferred from low dimensions to high dimensions. This method utilizes both the strong semantic features of the top layer and the high-resolution information of the bottom layer. Lateral connections are used to bring the downsampled features of the previous layer to the same resolution as the current layer, resulting in three pyramid feature maps of different scales.

[0148] Correspondingly, in the process of feature fusion of pyramid feature maps, such as Figure 9 As shown in the figure, after sampling to obtain three pyramid feature maps of different scales, the three pyramid feature maps of different scales can be input into three adaptive feature fusion blocks ASFF-1, ASFF-2 and ASFF-3 respectively, and the pyramid feature maps are fused with certain weights to obtain a fused feature map containing multi-scale features.

[0149] Correspondingly, in the process of embedding and updating the target detection features, such as Figure 10 As shown in the figure, the process includes three stages. First, for the preset image frame that is in the front position among the adjacent image frames, the embedded identity features of the preset image frame are extracted. , here you can select the corresponding TopK points in the heat map and extract the embedded identity features K key point features of . Will be embedded in the identity feature The K key point features are compressed from 128 dimensions to 16 dimensions to obtain the compressed key point features , used for subsequent feature updates. Secondly, through the correlation operation of adjacent image frames, the attention weights of feature enhancement are obtained , the attention weight can guide the network to focus on the current image frame, where the current image frame is the preset image frame in the adjacent image frames. and After multiplication, the attention features of the preset image frame in the previous position in the adjacent image frames are obtained through ReLU and linear transformation , and finally the attention feature Embedded identity features incorporated into the current image frame , and by embedding identity features Convolve with the target detection features to complete the embedding update of the target detection features.

[0150] It should be noted that, considering the class imbalance of drone videos during target tracking, in order to alleviate the class imbalance during model training, gradient balanced focal loss can be used to supervise the heat map learning ability of the feature detection model to pay more attention to the learning of small target categories. Figure 11 As shown, here the gradient balanced focus loss is based on the cross entropy loss Based on this, we improve on it and design two adaptive weights to reload the loss of heat map learning. Used to measure the difference between the predicted heat map and the true heat map. These two adaptive weights include the category balance weight and small-scale object attention weights .in, For balancing categories, Used to focus on small objects, the two adaptive loss weights are self-adjusted according to the imbalanced categories and sizes of the objects. It can be defined as the following formula:

[0151]

[0152] The attention weight of the above small target Pay attention to small targets, give them greater weight, measure the size of the target by the area of ​​the bounding box, and pay attention to the weight of specific small targets. It can be defined as the following formula:

[0153]

[0154] Among them, w and h represent the width and height of the target respectively. =5.

[0155] Balance weights of the above categories Different weights are assigned to positive and negative samples according to the corresponding gradients, and specific categories have unbalanced weights. It can be defined as the following formula:

[0156]

[0157] in, and Represent the weights of positive samples and negative samples respectively, which will be updated adaptively as the network trains. It is a heat map, which is used to represent the probability that a pixel is the center point of the target.

[0158] Further, as Figure 1-5 The specific implementation of the method, the embodiment of the present application provides a multi-target tracking device, such as Figure 12 As shown, the device includes: an acquisition unit 61, a detection unit 62, an update unit 63, and an association unit 64.

[0159] An acquisition unit 61 is configured to acquire a fused feature map of adjacent image frames, where the fused feature map is obtained by fusing the pyramid feature maps with set weights.

[0160] A detection unit 62 is configured to perform target detection on the fused feature map to obtain target detection features, wherein the target detection features include at least target boundary features, target identity features, and target probability features. The target identity features are feature vectors used to distinguish targets. The target probability features are visual image features used to locate targets. The visual image features represent target probability by color depth.

[0161] an updating unit 63 configured to update the target detection feature using an embedded identity feature to obtain an embedded updated target detection feature, wherein the embedded identity feature is extracted from the preset image frame, and the preset image frame is an image frame that is earlier in sequence among adjacent image frames;

[0162] The association unit 64 is configured to perform target association on the embedded updated target detection features using an adaptive association mode to obtain target correspondences in adjacent image frames.

[0163] The multi-target tracking device provided by the embodiment of the present invention, compared with the multi-target tracking method based on deep learning in the current prior art, obtains a fusion feature map of adjacent image frames. The fusion feature map is obtained by fusion of pyramid feature maps with set weights; performs target detection on the fusion feature map to obtain target detection features. The target detection identification includes at least target boundary features, target identity features, and target probability features. The target identity features are feature vectors used to distinguish targets. The target probability features are visual image features used to locate targets. The visual image features represent target probability by color depth; the target detection features are updated using embedded identity features to obtain embedded updated target detection features. The embedded identity features are extracted from a preset image frame. The preset image frame is the image frame with the previous position in the adjacent image frames; the embedded updated target detection features are associated with targets using an adaptive filtering mode to obtain target correspondence in adjacent image frames. The entire process shortens the information path based on the pyramid features, and constructs a fusion feature map containing semantic features and position information of different scales through weighted fusion. The fusion feature map can provide sufficient features to the multi-target tracker to improve the accuracy of the multi-target tracker. Since the target detection features will change with the position of the acquisition device, by learning the embedded identity features of adjacent image frames, the target detection features can be adaptively updated, and the adaptive filtering mode can be used to accurately complete the target association, thereby improving the ability of multi-target tracking.

[0164] In a specific application scenario, the acquisition unit is specifically used to:

[0165] Using the pre-trained feature extraction model, the feature maps obtained by sampling adjacent image frames using different paths are superimposed in multiple layers to obtain superimposed image features;

[0166] In the process of multi-layer superposition, the image features with consistent resolution in different layers are added and fused through lateral connections to obtain a pyramid feature map;

[0167] The pyramid feature maps are weightedly fused according to set weights to obtain fused feature maps of adjacent image frames.

[0168] In a specific application scenario, the device further includes:

[0169] A training unit is configured to perform model training using depth-separable convolution consisting of channel-by-channel convolution and point-by-point convolution before performing target detection on the fused feature map to obtain target detection features, to obtain a feature detection model, wherein one convolution kernel of the channel-by-channel convolution is responsible for one channel, the number of input channels and the number of output channels corresponding to the feature detection model are the same, and the fused feature map is weightedly summed on the number of channels through a convolution operation, and the corresponding target detection features are outputted with the same number of convolution kernels;

[0170] Correspondingly, the detection unit is specifically configured to perform target detection on the fused feature map using a pre-trained feature detection model to obtain target detection features.

[0171] In a specific application scenario, the device further includes:

[0172] an extraction unit configured to, before updating the target detection feature using the embedded identity feature to obtain the embedded updated target detection feature, use a preceding image frame in the adjacent image frames as a preset image frame and extract the embedded identity feature in the preset image frame;

[0173] Accordingly, the updating unit is specifically configured to:

[0174] Selecting a preset number of key position points in the target probability feature, wherein the key position points are position points in the target probability feature whose probability ranking is before a preset value;

[0175] Extracting a preset number of key point features from the embedded identity feature according to the key point positions in the target probability feature;

[0176] After compressing the key point features, the target detection features are updated using the compressed key point features to obtain embedded updated target detection features.

[0177] In a specific application scenario, the updating unit is further configured to:

[0178] performing an association operation on the adjacent image frames to obtain an attention weight for feature enhancement, wherein the attention weight is used to guide the key point features that the subsequent image frame in the adjacent image frames should focus on based on the image frame in the preceding image frame;

[0179] After compressing the key point features, weighted summing is performed on the compressed key point features according to the feature-enhanced attention weights to obtain the attention features that are in the front position in the adjacent image frames;

[0180] The target detection feature is updated using the attention feature to obtain an embedded updated target detection feature.

[0181] In a specific application scenario, the updating unit is further configured to:

[0182] Using the attention feature, feature association is performed on an image frame that comes earlier in sequence and an image frame that comes later in sequence in the adjacent image frames to obtain association features of the adjacent image frames;

[0183] The target detection feature is updated according to the associated feature to obtain an embedded updated target detection feature.

[0184] In a specific application scenario, the association unit is specifically used to:

[0185] Calculating target matching coefficients between adjacent image frames using Kalman filtering;

[0186] If the target matching coefficient is greater than a preset threshold, performing target association on the embedded updated target detection features using a first association mode to obtain target correspondences in adjacent image frames;

[0187] If the target matching coefficient is less than or equal to a preset threshold, a second association mode is used to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames.

[0188] It should be noted that for other corresponding descriptions of the functional units involved in the multi-target tracking device provided in this embodiment, please refer to Figure 1-Figure 5 The corresponding description in will not be repeated here.

[0189] Based on the above Figure 1-Figure 5 The method shown in FIG. 1 is a method for performing the above-mentioned operation. Accordingly, the embodiment of the present application further provides a storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned operation is performed. Figure 1-Figure 5 The multi-target tracking method shown.

[0190] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0191] Based on the above Figure 1-Figure 5 The method shown, and Figure 12In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a physical device for multi-target tracking, which can be a computer, a smart phone, a tablet computer, a smart watch, a server, or a network device, etc. The physical device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1-Figure 5 The multi-target tracking method shown.

[0192] Optionally, the physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include a display, an input unit such as a keyboard, etc. Optional user interfaces may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.

[0193] In an exemplary embodiment, see Figure 13 The physical device includes a communication bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device. The various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor is configured to execute the program stored in the memory and perform the multi-target tracking method described in the above embodiment.

[0194] Those skilled in the art will understand that the physical device structure for multi-target tracking provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0195] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device used for multi-target tracking, supporting the operation of the information processing program and other software and / or programs. The network communication module is used to enable communication between components within the storage medium and with other hardware and software within the physical information processing device.

[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the technical solution of the present application, compared with the current existing methods, the present application shortens the information path based on the pyramid feature, and constructs a fusion feature map containing semantic features and position information of different scales through weighted fusion. The fusion feature map can provide enough features to the multi-target tracker to improve the accuracy of the multi-target tracker. Since the target detection features will change with the position change of the acquisition device, by learning the embedded identity features of adjacent image frames, the target detection features can be adaptively updated, and the adaptive filtering mode can be used to accurately complete the target association, thereby improving the ability of multi-target tracking.

[0197] Those skilled in the art will understand that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application. Those skilled in the art will understand that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0198] The serial numbers of the above application are for descriptive purposes only and do not represent the advantages or disadvantages of the implementation scenarios. The above disclosure only discloses several specific implementation scenarios of the present application, but the present application is not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present application.

Claims

1. A multi-target tracking method, characterized in that: include: Obtaining a fused feature map of adjacent image frames, where the fused feature map is obtained by fusing the pyramid feature maps with set weights; Performing target detection on the fused feature map to obtain target detection features, wherein the target detection features include at least target boundary features, target identity features, and target probability features. The target identity features are feature vectors used to distinguish targets. The target probability features are visual image features used to locate targets. The visual image features represent target probability by color depth. The target detection feature is updated by using the embedded identity feature to obtain an embedded updated target detection feature, wherein the embedded identity feature is extracted from a preset image frame, and the preset image frame is an image frame that is in the previous position in the adjacent image frames. The specific process of updating the target detection feature is as follows: an association operation is performed on the adjacent image frames to obtain a feature-enhanced attention weight, each weight value in the attention weight is multiplied by the corresponding key point feature element, and all multiplied results are summed so that key point features with higher importance occupy a larger proportion in the attention feature; for different levels of features in the target detection feature, the shallow level features are updated by enhancing the details and edge information of the target through the attention feature, and the deep level features are updated by focusing on the fusion of semantic information through the attention feature, so that the target detection feature has the ability to express targets at different scales; Performing target association on the embedded and updated target detection features using an adaptive association mode to obtain target correspondences in adjacent image frames, specifically using a Kalman filter to calculate target matching coefficients between the adjacent image frames; if the target matching coefficient is greater than a preset threshold, performing target association on the embedded and updated target detection features using a first association mode to obtain target correspondences in the adjacent image frames; If the target matching coefficient is less than or equal to a preset threshold, a second association mode is used to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames.

2. The method according to claim 1, characterized in that The obtaining of a fusion feature map of adjacent image frames includes: Using the pre-trained feature extraction model, the feature maps obtained by sampling adjacent image frames using different paths are superimposed in multiple layers to obtain superimposed image features; In the process of multi-layer superposition, the image features with consistent resolution in different layers are added and fused through lateral connections to obtain a pyramid feature map; The pyramid feature maps are weightedly fused according to set weights to obtain fused feature maps of adjacent image frames.

3. The method according to claim 1, characterized in that Before performing target detection on the fused feature map to obtain target detection features, the method further includes: A feature detection model is obtained by training the model using a depth-wise separable convolution consisting of channel-by-channel convolution and point-by-point convolution. One convolution kernel of the channel-by-channel convolution is responsible for one channel. The number of input channels and output channels corresponding to the feature detection model are the same. The fused feature map is weighted summed over the number of channels through a convolution operation, and the corresponding output is a target detection feature with the same number of convolution kernels. Accordingly, a pre-trained feature detection model is used to perform target detection on the fused feature map to obtain target detection features.

4. The method according to claim 1, wherein Before updating the target detection feature by using the embedded identity feature to obtain the embedded updated target detection feature, the method further includes: Taking an image frame that is located earlier in sequence among the adjacent image frames as a preset image frame, and extracting the embedded identity features in the preset image frame; Accordingly, the updating of the target detection feature by using the embedded identity feature to obtain the embedded updated target detection feature includes: Selecting a preset number of key position points in the target probability feature, wherein the key position points are position points in the target probability feature whose probability ranking is before a preset value; Extracting a preset number of key point features from the embedded identity feature according to the key point positions in the target probability feature; After compressing the key point features, the target detection features are updated using the compressed key point features to obtain embedded updated target detection features.

5. The method according to claim 4, characterized in that After compressing the key point features, updating the target detection features using the compressed key point features to obtain embedded updated target detection features includes: performing an association operation on the adjacent image frames to obtain an attention weight for feature enhancement, wherein the attention weight is used to guide the key point features that the subsequent image frame in the adjacent image frames should pay attention to based on the image frame in the preceding image frame; After compressing the key point features, weighted summing is performed on the compressed key point features according to the feature-enhanced attention weights to obtain the attention features that are in the front position in the adjacent image frames; The target detection feature is updated using the attention feature to obtain an embedded updated target detection feature.

6. The method according to claim 5, characterized in that The updating of the target detection feature by using the attention feature to obtain an embedded updated target detection feature includes: Using the attention feature, feature association is performed on an image frame that comes earlier in sequence and an image frame that comes later in sequence in the adjacent image frames to obtain association features of the adjacent image frames; The target detection feature is updated according to the associated feature to obtain an embedded updated target detection feature.

7. A multi-target tracking device, characterized in that: include: an acquiring unit, configured to acquire a fused feature map of adjacent image frames, wherein the fused feature map is obtained by fusing the pyramid feature maps with set weights; a detection unit, configured to perform target detection on the fused feature map to obtain target detection features, wherein the target detection features include at least target boundary features, target identity features, and target probability features, wherein the target identity features are feature vectors used to distinguish targets, and the target probability features are visual image features used to locate targets, wherein the visual image features represent target probability by color depth; An updating unit is configured to update the target detection feature using an embedded identity feature to obtain an embedded updated target detection feature, wherein the embedded identity feature is extracted from a preset image frame, and the preset image frame is an image frame that is in the front position in the adjacent image frames. The specific process of updating the target detection feature is as follows: performing an association operation on the adjacent image frames to obtain a feature-enhanced attention weight, multiplying each weight value in the attention weight by the corresponding key point feature element, and summing all the multiplied results so that key point features with higher importance occupy a larger proportion in the attention feature; for different levels of features in the target detection feature, the shallow level features are updated by enhancing the details and edge information of the target through the attention feature, and the deep level features are updated by focusing on the fusion of semantic information through the attention feature, so that the target detection feature has the ability to express targets at different scales; an association unit, configured to perform target association on the embedded and updated target detection features using an adaptive association mode to obtain target correspondences in adjacent image frames, specifically using a Kalman filter to calculate target matching coefficients between the adjacent image frames; and if the target matching coefficient is greater than a preset threshold, perform target association on the embedded and updated target detection features using a first association mode to obtain target correspondences in the adjacent image frames; If the target matching coefficient is less than or equal to a preset threshold, a second association mode is used to perform target association on the embedded updated target detection features to obtain target correspondences in adjacent image frames.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the multi-target tracking method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-target tracking method according to any one of claims 1 to 6 are implemented.

Images