Multi-target tracking method and apparatus

Abstract

The present disclosure relates to the technical field of computers, and provides a multi-target tracking method and device. The method comprises: performing target detection on a first scene picture collected at a first time and a second scene picture collected at a second time respectively to obtain a first detection box set, a first feature map, a second detection box set and a second feature map; performing aggregation processing on a first target detection box feature map of the first target detection box in the first feature map to obtain a first aggregated feature map; determining a target region corresponding to the first target detection box in the second scene picture according to the position of the first target detection box in the first scene picture, and performing aggregation processing on a second target detection box feature map to obtain a second aggregated feature map; inputting the first aggregated feature map and the second aggregated feature map into a preset attention tracking model to generate a prediction box; and matching the prediction box and the detection boxes in the second detection box set to obtain a track of the target corresponding to the first target detection box.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a multi-target tracking method and apparatus. Background Technology

[0002] Multi-target tracking is the study of detecting target instances in video or continuous images and then temporarily associating these target instances to form a trajectory.

[0003] As a multi-target tracking scheme, the original SORT (Simple Online And Realtime Tracking) uses Kalman filtering based on simple geometric features for motion modeling. This modeling process assumes uniform motion, which makes it difficult to achieve high tracking accuracy in challenging tracking scenarios such as non-uniform motion and occlusion. Summary of the Invention

[0004] In view of this, the present disclosure provides a multi-target tracking method, apparatus, electronic device, and computer-readable storage medium to solve the technical problem of low multi-target tracking accuracy in the prior art.

[0005] A first aspect of this disclosure provides a multi-target tracking method, comprising: performing target detection on a first scene image acquired at a first time and a second scene image acquired at a second time, respectively, to obtain a first detection box set at the first time, a first feature map of the first scene image, a second detection box set at the second time, and a second feature map of the second scene image; aggregating the first target detection box feature maps of the first target detection box in the first detection box set in the first feature map to obtain a first aggregated feature map; determining the target region corresponding to the first target detection box in the second scene image based on the position of the first target detection box in the first scene image, and aggregating the second target detection box feature maps of the target region in the second feature map to obtain a second aggregated feature map; inputting the first aggregated feature map and the second aggregated feature map into a preset attention tracking model to generate a prediction box; and matching the prediction box and the detection boxes in the second detection box set to obtain the trajectory of the target corresponding to the first target detection box.

[0006] A second aspect of this disclosure provides a multi-target tracking device, comprising: a target detection unit, configured to perform target detection on a first scene image acquired at a first time and a second scene image acquired at a second time, respectively, to obtain a first detection box set at the first time, a first feature map of the first scene image, a second detection box set at the second time, and a second feature map of the second scene image; a first aggregation unit, configured to aggregate the first target detection box feature maps of the first target detection boxes in the first detection box set in the first feature map, to obtain a first aggregated feature map; a second aggregation unit, configured to determine the target region corresponding to the first target detection box in the second scene image based on the position of the first target detection box in the first scene image, and aggregate the second target detection box feature maps of the target region in the second feature map, to obtain a second aggregated feature map; an attention tracking unit, configured to input the first aggregated feature map and the second aggregated feature map into a preset attention tracking model to generate a prediction box; and a matching unit, configured to match the prediction box and the detection boxes in the second detection box set to obtain the trajectory of the target corresponding to the first target detection box.

[0007] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0008] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The beneficial effects of this disclosed embodiment compared to the prior art are: by performing target detection on scene images at different times, the features of the detection box at the previous time and its corresponding target region in the scene image at the next time are obtained. Figure 1 The input is fed into a pre-set attention tracking model, and the attention tracking result is matched with the detection box at the next time step to obtain the target tracking trajectory. Compared with existing technologies, this can improve the accuracy of multi-target tracking. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a multi-target tracking method provided in an embodiment of this disclosure;

[0012] Figure 2 This is a flowchart illustrating another multi-target tracking method provided in an embodiment of this disclosure;

[0013] Figure 3 This is a schematic diagram of the structure of an attention tracking model provided in an embodiment of this disclosure;

[0014] Figure 4 This is a schematic diagram of the structure of a self-attention convolutional network provided in an embodiment of this disclosure;

[0015] Figure 5 This is a schematic diagram of the structure of a multi-target tracking device provided in an embodiment of this disclosure;

[0016] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0018] The multi-target tracking method and apparatus according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0019] Figure 1 This is a flowchart illustrating a multi-target tracking method provided in an embodiment of this disclosure. The method provided in this embodiment can be executed by any electronic device with computer processing capabilities, such as a terminal or server. Figure 1 As shown, this multi-target tracking method includes:

[0020] Step S101: Target detection is performed on the first scene image acquired at the first time and the second scene image acquired at the second time, respectively, to obtain the first detection box set at the first time, the first feature map of the first scene image, the second detection box set at the second time, and the second feature map of the second scene image.

[0021] Specifically, a first scene image captured at the first time t can be input into the target detector to obtain the first detection box set D at the first time t. t The second scene image, captured at the second time t+δ, is input into the target detector to obtain the set D of the second detection boxes at the second time t+δ. t+δThe object detector can be a human detector that targets humans. The object detector can employ a YOLOx model, which provides high-performance object detection. The backbone network of the YOLOx model or other object detectors can extract features from the first scene image and the second scene image respectively, obtaining the corresponding first feature map F. t Second feature map F t+δ .

[0022] In this embodiment of the disclosure, the first time t is the earlier time, and the second time t+δ is the later time. The first scene image and the second scene image can be two images taken successively in the same environment, or they can be two images from the same video taken in that environment.

[0023] Step S102: Aggregate the first target detection box in the first detection box set and the first target detection box feature map in the first feature map to obtain the first aggregated feature map.

[0024] Specifically, a detection box is selected from the first set of detection boxes, namely the first target detection box. The position D of the first target detection box in the first scene image is then determined. t The first object detection box is obtained in the first feature map F t The first object detection bounding box feature map is then fed into the size-independent Roi Align layer to obtain the first aggregated feature map. Similarly, by aggregating the feature maps of other target detection boxes in the first feature map with the feature maps of other target detection boxes in the first feature map, we can obtain other aggregated feature maps corresponding to the feature maps of other target detection boxes in the first feature map. That is, we can obtain the aggregated feature maps corresponding to each detection box in the first feature map. Where i is the index of the target in the first detection box set, i = 1, 2, 3...n, and n is the number of target detection boxes in the first detection box set.

[0025] In this embodiment, the Roi Align (Region of Interest Align) operation is a region feature aggregation method. Through the Roi Align operation, input regions with different feature sizes can be output with the same size. The Roi Align layer can be understood as an aggregation layer.

[0026] Step S103: Based on the position of the first target detection box in the first scene image, determine the target region corresponding to the first target detection box in the second scene image, and perform aggregation processing on the second target detection box feature map of the target region in the second feature map to obtain the second aggregated feature map.

[0027] Specifically, similar to step S102, in step S103, the feature maps of the second target detection boxes corresponding to the target regions in the second scene image corresponding to the selected first target detection box can be aggregated to obtain a second aggregated feature map. Similarly, by inputting the feature maps of other target detection boxes in the second scene image corresponding to the target regions in the first set of detection boxes into the size-independent Roi Align layer for aggregation, we can obtain other aggregated feature maps corresponding to the feature maps of other target detection boxes in the second feature map. In other words, we can obtain the aggregated feature maps corresponding to the target regions in the second scene image corresponding to each detection box in the first set of detection boxes. Where i is the index of the target in the first detection box set.

[0028] Step S104: Input the first aggregated feature map and the second aggregated feature map into the preset attention tracking model to generate a prediction box.

[0029] Furthermore, the aggregated feature map and aggregated feature maps The corresponding inputs are fed into the attention tracking model to obtain a set of predicted bounding boxes. Where i is the index of the target in the first detection box set, i = 1, 2, 3...n, and n is the number of target detection boxes in the first detection box set.

[0030] Step S105: Match the predicted bounding box with the detection boxes in the second set of detection boxes to obtain the trajectory of the target corresponding to the first target detection box.

[0031] Specifically, the set of prediction boxes Detection boxes in the second detection box set at time t+δ By performing matching, an association can be formed, and the target trajectory corresponding to the first target detection box can be obtained. Where i is the index of the target in the first detection box set, i = 1, 2, 3...n, and n is the number of target detection boxes in the first detection box set. Step S105 can be achieved by inputting the predicted box and the detection boxes in the second detection box set into the matching layer.

[0032] According to the multi-target tracking scheme provided in the embodiments of this disclosure, more granular instance association can be achieved by utilizing the currently detected target and previous detection information, thereby improving the target tracking accuracy.

[0033] like Figure 2As shown, the processing of the first scene image 211 and the processing of the second scene image 221 are like two twin processes. Combined with the use of the attention-based attention tracking model 231, this embodiment provides an attention-based twin multi-target tracking network to improve the accuracy of multi-target tracking. Specifically, inputting the first scene image 211 into the target detector 212 yields a first feature map 213. The first target detection box 214's feature map in the first feature map is then input into the aggregation layer 215 to obtain a first aggregated feature map. Inputting the second scene image 221 into the target detector 222 yields a second feature map 223. The second target detection box feature map of the target region 224 corresponding to the first target detection box 214 in the second scene image is then input into the aggregation layer 225 to obtain a second aggregated feature map. Inputting the first and second aggregated feature maps into the attention tracking model 231 yields a predicted bounding box. Inputting the predicted bounding box into the matching layer 232 yields the target trajectory.

[0034] In step S103, the corresponding position in the second scene image can be determined based on the position of the first target detection box in the first scene image, and the coordinate area where the corresponding position is located can be expanded outward by 2 times the size to obtain the target area.

[0035] In this embodiment, expanding the coordinate region outward by a factor of 2 is merely an exemplary description of expanding the coordinate region outward by a certain size; in practical applications, it is not limited to a factor of 2. Expanding the coordinate region outward can improve the accuracy of obtaining the predicted bounding box set and the accuracy of target tracking.

[0036] In step S104, the first aggregated feature map and the second aggregated feature map can be input into the first self-attention module of the attention tracking model to obtain the corresponding denoised first and second denoised feature maps. The first and second denoised feature maps are then input into the information interaction module of the attention tracking model to obtain the first association feature representing the association relationship between the first target detection box and the target region. A prediction box is then generated based on the first association feature.

[0037] Furthermore, when generating the prediction box based on the first associated features, the first associated features can be input into the second self-attention module of the attention tracking model to obtain the second associated features; the second associated features can be input into the fully connected layer of the attention tracking model to obtain the prediction box.

[0038] like Figure 3 As shown, the aggregated feature map and aggregated feature maps The first and second denoised feature maps are respectively input into the first self-attention module 311 and the first self-attention module 321 to obtain the first denoised feature map and the second denoised feature map. The first and second denoised feature maps are then input into the information interaction module 331 to obtain the first associated feature. The first associated feature is then input into the second self-attention module 332 to obtain the second associated feature. The second associated feature is then input into the fully connected layer 333 to obtain the set of predicted bounding boxes.

[0039] In this embodiment of the disclosure, both the first self-attention module and the second self-attention module include a self-attention convolutional network. The process of the self-attention convolutional network processing the first input feature includes: compressing the three channels of the pixel using a 1×1 convolution to obtain intermediate features, which include a first channel feature, a second channel feature, and a third channel feature; obtaining the similarity between pixels based on the first channel feature and the second channel feature; and obtaining the output feature after processing with the attention mechanism using the self-attention convolutional network based on the similarity and the third channel feature.

[0040] like Figure 4 As shown, channel compression of the feature map is performed using a 1×1 convolution 401 on the first input feature X, resulting in the first channel feature Y, the second channel feature Z, and the third channel feature X'. The similarity W between pixels can be calculated using the following formula (1) based on the first channel feature Y and the second channel feature Z:

[0041] W = softmax(YZ) T (1)

[0042] Among them, softmax() is a normalization exponential function. Using the softmax() function to process data can be understood as inputting the data into the normalization layer 402 for processing.

[0043] Based on the following formula (2), similarity W, and third channel feature X', the result after attention mechanism processing, i.e., output feature X″, can be obtained:

[0044] X″=(I+W)X′ (2)

[0045] Specifically, the first self-attention module can extract more compact features from the detection box and the coarsely filtered target region to further reduce noise in the background. The second self-attention module can further refine the association results.

[0046] In this embodiment of the disclosure, when the first denoised feature map and the second denoised feature map are input into the information interaction module of the attention tracking model, the first denoised feature map can be used as a convolution kernel to perform similarity prediction on the target region based on the second denoised feature map to obtain the first associated feature.

[0047] Specifically, the information interaction module enables the exchange of messages between the detection box and the filtered target region, and associates the correct target object in the target region with the detection box to achieve fine-grained instance association. Its specific implementation method can be expressed by the following formula (3):

[0048]

[0049] Where, r i The similarity is indicated by i, where i is the index of the target in the first set of detection boxes, i = 1, 2, 3...n, and n is the number of target detection boxes in the first set of detection boxes.

[0050] In step S105, a match between the predicted bounding box and the current detection box in the second set of detection boxes can be determined when the intersection-union ratio (IUU) is greater than or equal to a set first value. Specifically, the first value can be set to 0.5, but is not limited to this.

[0051] In this embodiment of the invention, the existing YOLOv model can be directly used. The attention tracking model, however, needs to be trained after selecting an initial model framework. During the training process of the attention tracking model, the self-attention convolutional network, information interaction module, and fully connected layer of the attention tracking model are trained using a training dataset to obtain the attention tracking model when the training results converge.

[0052] This disclosure presents an attention-based twin multi-target tracking network for motion modeling and develops explicit template matching to estimate the motion of instances between two frames, thereby improving the accuracy of multi-target tracking.

[0053] According to the multi-target tracking method of this disclosure, target detection is performed on scene images at different times, and the features of the detection box at the previous time and its corresponding target region in the scene image at the next time are combined. Figure 1 The input is fed into a pre-set attention tracking model, and the attention tracking result is matched with the detection box at the next time step to obtain the target tracking trajectory. Compared with existing technologies, this can improve the accuracy of multi-target tracking.

[0054] The following are embodiments of the apparatus disclosed herein, which can be used to execute the embodiments of the method disclosed herein. The multi-target tracking apparatus described below corresponds to the multi-target tracking method described above. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0055] Figure 5 This is a schematic diagram of a multi-target tracking device provided in an embodiment of this disclosure. Figure 5As shown, the multi-target tracking device includes:

[0056] The target detection unit 501 is used to perform target detection on the first scene image acquired at the first time and the second scene image acquired at the second time, respectively, to obtain the first detection box set at the first time, the first feature map of the first scene image, the second detection box set at the second time, and the second feature map of the second scene image.

[0057] Specifically, a first scene image captured at the first time t can be input into the target detector to obtain the first detection box set D at the first time t. t The second scene image, captured at the second time t+δ, is input into the target detector to obtain the set D of the second detection boxes at the second time t+δ. t+δ The object detector can be a human detector that detects human bodies. The object detector can extract features from the first scene image and the second scene image respectively, obtaining the corresponding first feature map F. t Second feature map F t+δ .

[0058] The first aggregation unit 502 is used to aggregate the first target detection box in the first detection box set and the first target detection box feature map in the first feature map to obtain the first aggregated feature map.

[0059] Specifically, a detection box is selected from the first set of detection boxes, namely the first target detection box. The position D of the first target detection box in the first scene image is then determined. t The first object detection box is obtained in the first feature map F t The first object detection bounding box feature map is then fed into the size-independent Roi Align layer to obtain the first aggregated feature map. Similarly, by aggregating the feature maps of other target detection boxes in the first feature map with the feature maps of other target detection boxes in the first feature map, we can obtain other aggregated feature maps corresponding to the feature maps of other target detection boxes in the first feature map. That is, we can obtain the aggregated feature maps corresponding to each detection box in the first feature map. Where i is the index of the target in the first detection box set, i = 1, 2, 3...n, and n is the number of target detection boxes in the first detection box set.

[0060] The second aggregation unit 503 is used to determine the target region corresponding to the first target detection box in the second scene image based on the position of the first target detection box in the first scene image, and to perform aggregation processing on the second target detection box feature map of the target region in the second feature map to obtain the second aggregated feature map.

[0061] Specifically, the feature maps of the second target detection boxes corresponding to the target regions in the second scene image corresponding to the selected first target detection box can be aggregated to obtain a second aggregated feature map. Similarly, by inputting the feature maps of other target detection boxes in the second scene image corresponding to the target regions in the first set of detection boxes into the size-independent Roi Align layer for aggregation, we can obtain other aggregated feature maps corresponding to the feature maps of other target detection boxes in the second feature map. In other words, we can obtain the aggregated feature maps corresponding to the target regions in the second scene image corresponding to each detection box in the first set of detection boxes. Where i is the index of the target in the first detection box set.

[0062] The attention tracking unit 504 is used to input the first aggregated feature map and the second aggregated feature map into a preset attention tracking model to generate a prediction box.

[0063] Furthermore, the aggregated feature map and aggregated feature maps The corresponding inputs are fed into the attention tracking model to obtain a set of predicted bounding boxes. Where i is the index of the target in the first detection box set, i = 1, 2, 3...n, and n is the number of target detection boxes in the first detection box set.

[0064] The matching unit 505 is used to match the predicted box and the detection boxes in the second set of detection boxes to obtain the trajectory of the target corresponding to the first target detection box.

[0065] Specifically, the set of prediction boxes Detection boxes in the second detection box set at time t+δ By performing matching, an association can be formed, and the target trajectory corresponding to the first target detection box can be obtained. Where i is the index of the target in the first detection box set, i = 1, 2, 3...n, and n is the number of target detection boxes in the first detection box set.

[0066] According to the multi-target tracking scheme provided in the embodiments of this disclosure, more granular instance association can be achieved by utilizing the currently detected target and previous detection information, thereby improving the target tracking accuracy.

[0067] The second aggregation unit 503 can determine the corresponding position in the second scene image based on the position of the first target detection box in the first scene image, and expand the coordinate area where the corresponding position is located outward by 2 times the size to obtain the target area.

[0068] In this embodiment, expanding the coordinate region outward by a factor of 2 is merely an exemplary description of expanding the coordinate region outward by a certain size; in practical applications, it is not limited to a factor of 2. Expanding the coordinate region outward can improve the accuracy of obtaining the predicted bounding box set and the accuracy of target tracking.

[0069] The attention tracking unit 504 can input the first aggregated feature map and the second aggregated feature map into the first self-attention module of the attention tracking model to obtain the corresponding denoised first and second denoised feature maps. The first and second denoised feature maps are then input into the information interaction module of the attention tracking model to obtain the first association feature representing the association relationship between the first target detection box and the target region. A prediction box is then generated based on the first association feature.

[0070] Furthermore, when generating a prediction box based on the first associated features, the attention tracking unit 504 can input the first associated features into the second self-attention module of the attention tracking model to obtain the second associated features; and input the second associated features into the fully connected layer of the attention tracking model to obtain the prediction box.

[0071] In this embodiment of the disclosure, both the first self-attention module and the second self-attention module include a self-attention convolutional network. The process of the self-attention convolutional network processing the first input feature includes: compressing the three channels of the pixel using a 1×1 convolution to obtain intermediate features, which include a first channel feature, a second channel feature, and a third channel feature; obtaining the similarity between pixels based on the first channel feature and the second channel feature; and obtaining the output feature after processing with the attention mechanism using the self-attention convolutional network based on the similarity and the third channel feature.

[0072] Specifically, the first self-attention module can extract more compact features from the detection box and the coarsely filtered target region to further reduce noise in the background. The second self-attention module can further refine the association results.

[0073] In this embodiment of the disclosure, when the first denoised feature map and the second denoised feature map are input into the information interaction module of the attention tracking model, the first denoised feature map can be used as a convolution kernel to perform similarity prediction on the target region based on the second denoised feature map to obtain the first associated feature.

[0074] Specifically, the information interaction module enables the exchange of messages between the detection box and the filtered target region, and associates the correct target object in the target region with the detection box to achieve fine-grained instance association. Its specific implementation method can be expressed by the following formula (3):

[0075]

[0076] Where, r i The similarity is indicated by i, where i is the index of the target in the first set of detection boxes, i = 1, 2, 3...n, and n is the number of target detection boxes in the first set of detection boxes.

[0077] The matching unit 505 can determine that the predicted box matches the current detection box when the intersection-union ratio (IUU) between the predicted box and the current detection box in the second set of detection boxes is greater than or equal to a set first value. Specifically, the first value can be set to 0.5, but is not limited to this.

[0078] This disclosure presents an attention-based twin multi-target tracking network for motion modeling and develops explicit template matching to estimate the motion of instances between two frames, thereby improving the accuracy of multi-target tracking.

[0079] Since the functional modules of the multi-target tracking device in the example embodiments of this disclosure correspond to the steps of the example embodiments of the multi-target tracking method described above, for details not disclosed in the device embodiments of this disclosure, please refer to the embodiments of the multi-target tracking method described above.

[0080] The multi-target tracking device according to embodiments of this disclosure performs target detection on scene images at different times, and combines the features of the detection box at the previous time and the corresponding target region in the scene image at the next time. Figure 1 The input is fed into a pre-set attention tracking model, and the attention tracking result is matched with the detection box at the next time step to obtain the target tracking trajectory. Compared with existing technologies, this can improve the accuracy of multi-target tracking.

[0081] Figure 6 This is a schematic diagram of the electronic device 6 provided in an embodiment of this disclosure. Figure 6 As shown, the electronic device 6 of this embodiment includes a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, it implements the steps in the various method embodiments described above. Alternatively, when the processor 601 executes the computer program 603, it implements the functions of each module in the various device embodiments described above.

[0082] Electronic device 6 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 6 may include, but is not limited to, processor 601 and memory 602. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or different components.

[0083] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0084] The memory 602 can be an internal storage unit of the electronic device 6, such as a hard disk or RAM of the electronic device 6. The memory 602 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 6. The memory 602 can also include both internal and external storage units of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device.

[0085] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0086] If the integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0087] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A multi-target tracking method characterized by, The method includes: Target detection is performed on the first scene image captured at the first time and the second scene image captured at the second time, respectively, to obtain the first detection box set at the first time, the first feature map of the first scene image, the second detection box set at the second time, and the second feature map of the second scene image; The feature maps of the first target detection boxes in the first detection box set and the first target detection boxes in the first feature map are aggregated to obtain a first aggregated feature map. Based on the position of the first target detection box in the first scene image, the target region corresponding to the first target detection box is determined in the second scene image, and the second target detection box feature map of the target region in the second feature map is aggregated to obtain the second aggregated feature map; The first aggregated feature map and the second aggregated feature map are input into a preset attention tracking model to generate a prediction box; The predicted bounding box is matched with the detection boxes in the second set of detection boxes to obtain the trajectory of the target corresponding to the first target detection box; The first aggregated feature map and the second aggregated feature map are input into a preset attention tracking model to generate a prediction box, including: The first aggregated feature map and the second aggregated feature map are respectively input into the first self-attention module of the attention tracking model to obtain the corresponding denoised first and second denoised feature maps. The first denoised feature map and the second denoised feature map are input into the information interaction module of the attention tracking model. The first denoised feature map is used as a convolution kernel to perform similarity prediction on the target region based on the second denoised feature map, so as to obtain a first association feature that represents the association relationship between the first target detection box and the target region. The first associated feature is input into the second self-attention module of the attention tracking model to obtain the second associated feature. The second associated feature is then input into the fully connected layer of the attention tracking model to generate the prediction box.

2. The method of claim 1, wherein, Both the first self-attention module and the second self-attention module include a self-attention convolutional network. The process by which the self-attention convolutional network processes the first input feature includes: The first input feature is compressed using a 1×1 convolution on the three channels of the pixel to obtain intermediate features, which include the first channel feature, the second channel feature, and the third channel feature. The similarity between pixels is obtained based on the first channel features and the second channel features; The output features after attention mechanism processing using the self-attention convolutional network are obtained based on the similarity and the third channel features.

3. The method of claim 1, wherein, Matching the predicted bounding box with the detection boxes in the second set of detection boxes includes: When the intersection-union ratio (IUU) between the predicted box and the current detection box in the second set of detection boxes is greater than or equal to a set first value, it is determined that the predicted box matches the current detection box.

4. The method according to any one of claims 1 to 3, characterized in that, Determining the target region corresponding to the first target detection box in the second scene image based on the position of the first target detection box in the first scene image includes: The corresponding position in the second scene image is determined based on the position of the first target detection box in the first scene image; The target area is obtained by expanding the coordinate region of the corresponding position outward by a factor of 2.

5. A multi-target tracking device, characterized by, The device includes: The target detection unit is used to perform target detection on the first scene image acquired at the first time and the second scene image acquired at the second time, respectively, to obtain the first detection box set at the first time, the first feature map of the first scene image, the second detection box set at the second time, and the second feature map of the second scene image. The first aggregation unit is used to aggregate the first target detection box feature map in the first feature map of the first target detection box in the first detection box set to obtain the first aggregated feature map. The second aggregation unit is used to determine the target region corresponding to the first target detection box in the second scene image based on the position of the first target detection box in the first scene image, and to perform aggregation processing on the second target detection box feature map of the target region in the second feature map to obtain the second aggregated feature map. An attention tracking unit is used to input the first aggregated feature map and the second aggregated feature map into a preset attention tracking model to generate a prediction box; A matching unit is used to match the predicted box with the detection boxes in the second set of detection boxes to obtain the trajectory of the target corresponding to the first target detection box; The attention tracking unit is specifically used for: inputting the first aggregated feature map and the second aggregated feature map into the first self-attention module of the attention tracking model to obtain the corresponding denoised first and second denoised feature maps; inputting the first and second denoised feature maps into the information interaction module of the attention tracking model, using the first denoised feature map as a convolution kernel to perform similarity prediction on the target region based on the second denoised feature map to obtain a first association feature representing the association relationship between the first target detection box and the target region; inputting the first association feature into the second self-attention module of the attention tracking model to obtain a second association feature, and inputting the second association feature into the fully connected layer of the attention tracking model to generate the prediction box.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 6. When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images