A multi-target tracking method and system

Abstract

The application provides a multi-target tracking method and system, the method comprising: taking a current frame and N-1 frames of image before the current frame in video data of a target to be tracked as a group of input images; obtaining N convolution kernels containing time sequence information; obtaining a deep feature representation of a time sequence corresponding to the input images; obtaining a position heat map of a target center point; obtaining the position and size of the deep feature representation of the time sequence; constructing a target classification head and a target tracking feature head; training a deep learning skeleton network; obtaining a real position of the target; constructing a bipartite graph correlation model; and obtaining a multi-target tracking correlation result based on the bipartite graph correlation model. The application can solve the technical problem of low tracking precision of the existing tracking method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision target tracking, and in particular to a multi-target tracking method and system. BACKGROUND

[0002] As a research field with a long history and important application value, multi-target tracking plays a role in the computer vision field. Previous topics such as target segmentation and target detection distinguish the foreground and background of a traffic video sequence to provide basic input information for target tracking. Subsequent more macroscopic topics such as scene understanding and behavior analysis need to be based on tracking information to carry out research.

[0003] A multi-target tracking method usually consists of multiple sequentially executed steps, which mainly include input video sequence, detection and feature extraction, and feature association matching. According to the time sequence of the input information required to generate a tracking sequence, it is divided into online tracking and offline tracking. Among them, online tracking can only use historical information and the current frame to obtain association results, while offline tracking can use the entire video sequence for trajectory association. According to the way of generating a trajectory of multi-target tracking, it can be divided into a tracking method based on probabilistic reasoning and a tracking method based on deterministic reasoning. According to the representation of the trajectory of multi-target tracking, it can be divided into point-based tracking, contour-based tracking, pose-based tracking, and mask-based tracking from coarse to fine. The framework of multi-target tracking is mainly divided into two categories: detection-based tracking and joint detection tracking. Among them, the detection-based tracking method is greatly restricted by the detection effect. Poor detection results not only bring great interference to the feature extraction of the tracked target, but also introduce a lot of noise to the association process. In addition, the detection and tracking processes under this framework both involve feature extraction, which actually involves repeated operations on features. Therefore, compared with the detection-based tracking framework, the joint detection multi-target tracking framework can comprehensively utilize multiple feature information, realize the linkage of tracking and detection features, and thus realize an end-to-end tracking method and improve the tracking accuracy.

[0004] After the rise of deep learning, detection methods based on automatic learning have become the trend of target detection. Deep neural networks simulate the neural transmission of the human brain and can extract rich features at multiple levels, with strong non-linear fitting ability. According to existing visualization results, deep learning has an absolute advantage in extracting abstract features. Subsequently, with the improvement of computer computing power, the application of deep learning in the field of computer vision has greatly improved the performance of related tasks, reaching a performance superior to most traditional detection algorithms. However, for the multi-target tracking task, the time sequence information and the spatial position of the target play an important role in feature extraction and discrimination. The existing feature extraction method based on single-frame information and the general deep learning feature extraction mode cannot fully extract the differences between different targets and the commonality of the same target, and the tracking accuracy needs to be improved. SUMMARY

[0005] The present application provides a multi-target tracking method and system, which can solve the technical problem of low tracking accuracy of the existing tracking method.

[0006] According to an aspect of the present application, a multi-target tracking method is provided, which comprises:

[0007] taking a current frame and N-1 frames of image before the current frame in the video data of the target to be tracked as a group of input images;

[0008] performing time series convolution operation on the input images by using a time series adaptive convolution module to obtain N convolution kernels containing time series information;

[0009] inputting the N convolution kernels containing time series information into a deep learning skeleton network to obtain a deep feature representation of the time series sequence corresponding to the input images;

[0010] performing multi-perception on the deep feature representation of the time series sequence to obtain a position heat map of the target center point;

[0011] obtaining the length and width of the detection bounding box of the target center point based on the position heat map of the target center point, and obtaining the position and size of the deep feature representation of the time series sequence based on the length and width of the detection bounding box of the target center point;

[0012] performing multi-perception on the deep feature representation of the time series sequence and the size scaling relationship between the input images and the deep feature representation of the time series sequence to construct a target classification head and a target tracking feature head;

[0013] training the deep learning skeleton network, and in the training process, obtaining the position offset of the target center point based on the size scaling relationship between the input images and the deep feature representation of the time series sequence, obtaining the category of the target based on the target classification head, and obtaining the tracking characteristic representation of the target based on the target tracking feature head;

[0014] obtaining the real position of the target based on the position of the deep feature representation of the time series sequence and the position offset of the target center point;

[0015] constructing a bipartite graph association model based on the position and size of the deep feature representation of the time series sequence, the position offset of the target center point, the category of the target, the tracking characteristic representation of the target and the historical trajectory;

[0016] obtaining a multi-target tracking association result based on the bipartite graph association model.

[0017] Preferably, obtaining a multi-target tracking association result based on the bipartite graph association model comprises:

[0018] the bipartite graph association model predicts the position of the historical trajectory in the current frame by Kalman.

[0019] obtaining a motion similarity between a real position of the target and a position of the historical trajectory in the current frame;

[0020] obtaining an appearance similarity between a tracking feature representation of the target in the current frame and a tracking feature representation of the historical trajectory;

[0021] obtaining a feature similarity matrix based on the motion similarity and the appearance similarity;

[0022] the feature similarity matrix is converted into a cost matrix, and a multi-target tracking association result is obtained by minimizing the cost matrix.

[0023] Preferably, training the deep learning skeleton network comprises:

[0024] obtaining a heat map loss of the deep learning skeleton network, a loss of a bias and a box size of a detection bounding box, and a target tracking feature obtaining loss;

[0025] weighting and summing the heat map loss of the deep learning skeleton network, the loss of the bias and the box size of the detection bounding box, and the target tracking feature obtaining loss to obtain a total loss of the deep learning skeleton network;

[0026] training the deep learning skeleton network based on the total loss of the deep learning skeleton network.

[0027] According to another aspect of the present application, there is provided a multi-target tracking system, the system comprising:

[0028] The detection module is configured to take a current frame and N-1 frames of image before the current frame in video data of a target to be tracked as a set of input images, to perform a time sequence convolution operation on the input images by using a time sequence adaptive convolution module to obtain N convolution kernels containing time sequence information, to input the N convolution kernels containing time sequence information into a deep learning skeleton network to obtain a deep feature representation of a time sequence corresponding to the input images, to perform multiple perceptions on the deep feature representation of the time sequence to obtain a position heat map of a target center point, to obtain a length and a width of a detection bounding box of the target center point based on the position heat map of the target center point, and to obtain a position and a size of the deep feature representation of the time sequence based on the length and the width of the detection bounding box of the target center point, to perform multiple perceptions on the deep feature representation of the time sequence and a size scaling relationship between the input images and the deep feature representation of the time sequence to construct a target classification head and a target tracking feature head, to train the deep learning skeleton network, and in the training process, to obtain a position offset of the target center point based on the size scaling relationship between the input images and the deep feature representation of the time sequence, to obtain a category of the target based on the target classification head, and to obtain a tracking characteristic representation of the target based on the target tracking feature head, and to obtain a real position of the target based on the position of the deep feature representation of the time sequence and the position offset of the target center point.

[0029] The tracking module is configured to construct a bipartite graph association model based on the position and the size of the deep feature representation of the time sequence, the position offset of the target center point, the category of the target, the tracking characteristic representation of the target and a historical trajectory, and to obtain a multi-target tracking association result based on the bipartite graph association model.

[0030] According to still another aspect of the present application, a computer device is provided, which comprises a memory, a processor, and a multi-target tracking program stored in the memory and executable on the processor, and the processor implements any of the above methods when executing the multi-target tracking program.

[0031] By using the time sequence adaptive convolution module, N convolution kernels containing time sequence information are obtained, so that the convolution kernels adaptively give different attentions to images of different frames, and spatial convolution can perform time sequence reasoning. Meanwhile, the module gives more attention to the foreground position change state, obtains more attention weight for the possible foreground position, improves the sensitivity of the model to the position change of the target in the time sequence, and the features of the previous N-1 frames of image are used as attention inspiration to help the model obtain more accurate feature representation of the last frame of image. The present application provides a multi-target tracking method based on space-time attention mechanism in a complex scene, improves the accuracy of the multi-target tracking algorithm in the complex scene, and reduces the problems of tracking failure and label switching caused by frequent interaction between targets. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. It is to be understood that the drawings are solely for purposes of illustration to be taken in connection with the description and are not to be constructed as a definition of the present application. The drawings are schematic representations for purposes of the explanation of the embodiments of the present application, and changes in detail and arrangement of parts can be made therein without departing from the scope of the present application.

[0033] Fig. 1 A block diagram of a multi-target tracking method according to an embodiment of the present application is shown;

[0034] Fig. 2 A flowchart of a multi-target tracking method according to an embodiment of the present application is shown;

[0035] Fig. 3 A flowchart of a timing adaptive convolution according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other on the premise of no conflict. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0037] It should be noted that the terms used herein are only intended to describe specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.

[0038] The relative arrangement of parts and steps, numerical expressions, and numerical values set forth in the examples herein are not meant to limit the scope of the present application unless otherwise specifically stated. Also, it is to be understood that the dimensions of the various parts shown in the drawings are not necessarily to scale as the dimensions can have been exaggerated for the sake of clarity in illustrating the principles of the application. Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail but should be considered as part of the description of the present application where appropriate. In all examples shown and discussed herein, any specific values should be interpreted as merely illustrative and not as a limitation on the scope of the exemplary embodiments. Thus, other examples of the exemplary embodiments can have different values. It is to be noted that like numbers and letters refer to like elements throughout the several views of the drawings and that discussion of each element in a drawing does not mean that all of the elements have to be discussed in every drawing. It is to be understood that the figures and descriptions of the present application are intended to be illustrative, not restrictive. Numerous modifications and adaptations will occur to those skilled in the art, and it is to be understood that all such modifications and adaptations are within the scope of the embodiments of the present application.

[0039] As shown in Figs. 1-3 , the present application provides a multi-target tracking method, the method comprising:

[0040] S10, taking a current frame and N-1 frames of image before the current frame in the video data of the target to be tracked as a set of input images;

[0041] S20, performing time convolution operation on the input images by using a time adaptive convolution module to obtain N convolution kernels containing time information;

[0042] S30, inputting the N convolution kernels containing time information into a deep learning skeleton network to obtain a deep feature representation of a time sequence corresponding to the input images;

[0043] S40, performing multi-perception on the deep feature representation of the time sequence to obtain a position heat map of the target center point;

[0044] S50, obtaining the length and width of the detection bounding box of the target center point based on the position heat map of the target center point, and obtaining the position and size of the deep feature representation of the time sequence based on the length and width of the detection bounding box of the target center point; wherein the detection bounding box can adopt a rectangular bounding box;

[0045] S60, performing multi-perception on the deep feature representation of the time sequence and the size scaling relationship between the input images and the deep feature representation of the time sequence to construct a target classification head and a target tracking feature head;

[0046] S70, training the deep learning skeleton network, in the training process, obtaining the position offset of the target center point based on the size scaling relationship between the input images and the deep feature representation of the time sequence, obtaining the category of the target based on the target classification head, and obtaining the tracking characteristic representation of the target based on the target tracking feature head;

[0047] S80, the real position of the target is obtained based on the position of the time sequence-based deep feature representation and the position offset of the target center point;

[0048] S90, a two-part graph association model is constructed based on the position and size of the time sequence-based deep feature representation, the position offset of the target center point, the category of the target, the tracking characteristic representation of the target and the historical trajectory;

[0049] S100, a multi-target tracking association result is obtained based on the two-part graph association model.

[0050] The N convolution kernels containing time sequence information are obtained through the time sequence adaptive convolution module, so that the convolution kernel adaptively gives different attention to different frames of images, and the spatial convolution can perform time sequence reasoning; at the same time, the module gives more attention to the foreground position change state, obtains more attention weight for the possible foreground position, improves the sensitivity of the model to the position change of the target in time sequence, and the image features of the first N-1 frames are used as attention inspiration to help the model obtain more accurate feature representation of the last frame image. The present application provides a multi-target tracking method based on space-time attention mechanism in a complex scene, improves the accuracy of the multi-target tracking algorithm in the complex scene, and reduces the problems of tracking failure and label switching caused by frequent interaction between targets.

[0051] According to an embodiment of the present application, in S70 of the present application, training the deep learning skeleton network comprises:

[0052] S71, obtaining a heat map loss of the deep learning skeleton network, a loss of the bounding box offset and the bounding box size of the detection, and a target tracking feature acquisition loss;

[0053] S72, the heat map loss of the deep learning skeleton network, the loss of the bounding box offset and the bounding box size of the detection, and the target tracking feature acquisition loss are weighted and summed to obtain the total loss of the deep learning skeleton network;

[0054] S73, training the deep learning skeleton network based on the total loss of the deep learning skeleton network.

[0055] According to an embodiment of the present application, in S100 of the present application, obtaining a multi-target tracking association result based on the two-part graph association model comprises:

[0056] S101, the two-part graph association model predicts the position of the historical trajectory in the current frame through Kalman;

[0057] S102, the motion similarity between the real position of the target and the position of the historical trajectory in the current frame is obtained;

[0058] S103, the appearance similarity between the tracking characteristic representation of the target in the current frame and the tracking characteristic representation of the historical trajectory is obtained;

[0059] S104, obtaining a feature similarity matrix based on the motion similarity and the appearance similarity;

[0060] S105, converting the feature similarity matrix into a cost matrix, and obtaining a multi-target tracking association result by minimizing the cost matrix.

[0061] The application also provides a multi-target tracking system, which comprises:

[0062] The detection module is configured to take a current frame and N-1 frames of image before the current frame in the video data of the target to be tracked as a group of input images, perform a time convolution operation on the input images by using a time adaptive convolution module to obtain N convolution kernels containing time information, input the N convolution kernels containing time information into a deep learning skeleton network to obtain a deep feature representation of a time sequence corresponding to the input images, perform multiple perceptions on the deep feature representation of the time sequence to obtain a position heat map of the target center point, obtain the length and width of a detection bounding box of the target center point based on the position heat map of the target center point, and obtain the position and size of the deep feature representation of the time sequence based on the length and width of the detection bounding box of the target center point, perform multiple perceptions on the deep feature representation of the time sequence and the size scaling relationship between the input images and the deep feature representation of the time sequence, construct a target classification head and a target tracking feature head, train the deep learning skeleton network, and in the training process, obtain a position offset of the target center point based on the size scaling relationship between the input images and the deep feature representation of the time sequence, obtain a category of the target based on the target classification head, obtain a tracking characteristic representation of the target based on the target tracking feature head, and obtain a real position of the target based on the position of the deep feature representation of the time sequence and the position offset of the target center point.

[0063] The tracking module is configured to construct a bipartite graph association model based on the position and size of the deep feature representation of the time sequence, the position offset of the target center point, the category of the target, the tracking characteristic representation of the target and a historical trajectory, and obtain a multi-target tracking association result based on the bipartite graph association model.

[0064] In order to have a further understanding of the application, the following Figs. 1-3 The multi-target tracking method of the application is described in detail, which specifically comprises the following steps:

[0065] Firstly, a deep learning skeleton network structure is set, and a typical deep learning image feature extraction network DLA-34 is adopted, which mainly functions to output multi-level feature maps with multi-scale receptive fields. As shown in FIG. 1, the feature map receptive fields in the DLA-34 are C1: 1 / 4, C2: 1 / 8, C3: 1 / 16 and C4: 1 / 32. Fig. 2 ​

[0066] Secondly, based on the DLA-34 deep learning skeleton network structure, a time-adaptive convolution module is added to each state of the DLA-34, which adaptively adjusts the convolution kernel along the time dimension, so that the model can adaptively assign different convolution kernels to different frames of the image sequence, enable spatial convolution to perform time reasoning, and enable the model to have both time and spatial attention without additional computational cost.

[0067] Thirdly, the current frame and the three frames of images before the current frame in the video data of the target to be tracked are input as a group of input images to the time-adaptive convolution module.

[0068] Fourthly, based on the DLA-34 and the time-adaptive convolution feature extraction module, a more anti-interference deep feature representation is obtained. The features of the first three frames of images are used as attention to help the model obtain a more accurate feature representation of the last frame of image, i.e., the deep feature representation of the time sequence.

[0069] Fifthly, the deep feature representation of the time sequence is subjected to multiple perception to obtain a position heat map of the target center point.

[0070] Sixthly, the length and width of the rectangular bounding box of the target center point are obtained based on the position heat map of the target center point, and the position and size of the deep feature representation of the time sequence are obtained based on the length and width of the rectangular bounding box of the target center point.

[0071] Seventhly, the deep feature representation of the time sequence and the size scaling relationship between the input image and the deep feature representation of the time sequence are subjected to multiple perception to construct a target classification head and a target tracking feature head.

[0072] Eighthly, the heat map loss of the deep learning skeleton network, the loss of the bias and the box size of the detection bounding box, and the loss of the target tracking feature acquisition are obtained.

[0073] The loss function of the position heat map of the target center point is as follows:

[0074]

[0075] In the formula, L heatmap represents the loss of the position heat map of the target center point, is the predicted heat map, M xy is the heat map true value, N is the total number of targets in a single image, and a and β are the first and second hyperparameters, and x and y are the horizontal and vertical pixels of the target.

[0076] The loss function of the bias and the box size of the detection bounding box is as follows:

[0077]

[0078] wherein, L box represents the loss of bounding box detection bias and box size, N is the total number of targets in a single image, i is the target label to be calculated, s represents the size of the predicted target detection box, represents the true value size of the target detection box, o represents the deviation of the predicted center point, represents the deviation of the real center point, and ||·|| represents the sum of the absolute errors between two vectors;

[0079] The target tracking feature acquisition loss function is as follows:

[0080]

[0081] wherein, L identity represents the target tracking feature acquisition loss, N is the total number of targets in a single image, K is the number of categories, p(·) is the predicted target classification feature vector, L i (·) is the true classification one-hot encoding vector of the i-th target.

[0082] In the ninth step, the heat map loss of the deep learning skeleton network, the loss of the bounding box detection bias and the box size, and the target tracking feature acquisition loss are weighted and summed to obtain the total loss of the deep learning skeleton network. The deep learning skeleton network is trained based on the total loss of the deep learning skeleton network. In the training process, the position offset of the target center point is obtained based on the size scaling relationship between the input image and the deep feature representation of the time sequence, the category of the target is obtained based on the target classification head, and the target classification task is learned by training the model using the softmax loss; the tracking characteristic representation of the target is obtained based on the target tracking feature head.

[0083] In the tenth step, the real position of the target is obtained based on the position of the deep feature representation of the time sequence and the position offset of the target center point.

[0084] In the eleventh step, a bipartite graph association model is constructed based on the position and size of the deep feature representation of the time sequence, the position offset of the target center point, the category of the target, the tracking characteristic representation of the target, and the historical trajectory; and a multi-target tracking association result is obtained based on the bipartite graph association model.

[0085] The application also provides a computer device comprising a memory, a processor, and a multi-target tracking program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods when executing the multi-target tracking program.

[0086] In summary, the present application provides a multi-target tracking method and system, through a time sequence adaptive convolution module to obtain N convolution kernels containing time sequence information, so that the convolution kernel adaptively gives different attention to images of different frames, enabling spatial convolution to perform time sequence reasoning; at the same time, the module gives more attention to the foreground position change state, obtains more attention weight for the possible foreground position, improves the sensitivity of the model to the position change of the target in time sequence, and the image features of the first N-1 frames are used as attention inspiration to help the model obtain more accurate feature representation of the last frame image. The present application provides a multi-target tracking method based on space-time attention mechanism in a complex scene, improves the accuracy of the multi-target tracking algorithm in a complex scene, and reduces the problems of tracking failure and label switching caused by frequent interaction between targets.

[0087] The parts of the present application not described in detail are known to those skilled in the art.

[0088] In the description of the present application, it should be understood that the orientation words such as "front, back, up, down, left, right", "horizontal, vertical, perpendicular, horizontal" and "top, bottom" and the like indicate the orientation or positional relationship shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate and imply that the devices or elements referred to must have a specific orientation or be constructed and operated in a specific orientation, therefore cannot be understood as a limitation on the scope of protection of the present application; the orientation words "inner, outer" refer to the inner and outer of the contour of each component itself.

[0089] For the convenience of description, spatial relative terms such as "over", "above", "upper surface", "upper" and the like can be used herein to describe the spatial positional relationship of one device or feature with other devices or features as shown in the drawings. It should be understood that the spatial relative terms are intended to include different orientations in use or operation in addition to the orientation of the devices described in the drawings. For example, if the devices in the drawings are inverted, the device described as "above" or "over" other devices or structures will be positioned "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below" orientations. The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein are interpreted accordingly.

[0090] In addition, it should be noted that the use of the words "first", "second" and the like to define parts is only for the convenience of distinguishing the corresponding parts, and the above words have no special meaning unless otherwise stated, therefore cannot be understood as a limitation on the scope of protection of the present application.

[0091] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. The present application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-target tracking method characterized by, The method comprises: taking a current frame and N-1 frames of image before the current frame in the video data of a target to be tracked as a set of input images; performing a time convolution operation on the input images by using a time adaptive convolution module to obtain N convolution kernels containing time information; inputting the N convolution kernels containing time information into a deep learning skeleton network to obtain a deep feature representation of a time sequence corresponding to the input images; performing multiple perceptions on the deep feature representation of the time sequence to obtain a position heat map of a target center point; obtaining the length and width of a detection bounding box of the target center point based on the position heat map of the target center point, and obtaining the position and size of the deep feature representation of the time sequence based on the length and width of the detection bounding box of the target center point; performing multiple perceptions on the deep feature representation of the time sequence and the size scaling relationship between the input images and the deep feature representation of the time sequence to construct a target classification head and a target tracking feature head; training the deep learning skeleton network, in the training process, obtaining a position offset of the target center point based on the size scaling relationship between the input images and the deep feature representation of the time sequence, obtaining a category of the target based on the target classification head, and obtaining a tracking characteristic representation of the target based on the target tracking feature head; obtaining a real position of the target based on the position of the deep feature representation of the time sequence and the position offset of the target center point; constructing a bipartite graph correlation model based on the position and size of the deep feature representation of the time sequence, the position offset of the target center point, the category of the target, the tracking characteristic representation of the target, and a historical trajectory; obtaining a multi-target tracking correlation result based on the bipartite graph correlation model.

2. The method of claim 1, wherein, Obtaining a multi-target tracking correlation result based on the bipartite graph correlation model comprises: the bipartite graph correlation model predicts a position of the historical trajectory in the current frame through Kalman; obtaining a motion similarity between the real position of the target and the position of the historical trajectory in the current frame; obtaining an appearance similarity between the tracking characteristic representation of the target in the current frame and the tracking characteristic representation of the historical trajectory; obtaining a feature similarity matrix based on the motion similarity and the appearance similarity; the feature similarity matrix is converted into a cost matrix, and the multi-target tracking correlation result is obtained by minimizing the cost matrix.

3. The method according to claim 1 or 2, characterized in that, Training the deep learning skeleton network comprises: obtaining a heat map loss of the deep learning skeleton network, a bias and frame size loss of a detection bounding box, and a target tracking feature acquisition loss; performing a weighted summation on the heat map loss of the deep learning skeleton network, the bias and frame size loss of the detection bounding box, and the target tracking feature acquisition loss to obtain a total loss of the deep learning skeleton network; training the deep learning skeleton network based on the total loss of the deep learning skeleton network.

4. A multi-target tracking system, characterized by The system comprises: The detection module is configured to take a current frame and N-1 frames of image before the current frame in the video data of the target to be tracked as a set of input images; to perform a time convolution operation on the input images by using a time adaptive convolution module to obtain N convolution kernels containing time information; to input the N convolution kernels containing time information into a deep learning skeleton network to obtain a deep feature representation of a time sequence corresponding to the input images; to perform multiple perceptions on the deep feature representation of the time sequence to obtain a position heat map of a target center point; to obtain a length and a width of a detection bounding box of the target center point based on the position heat map of the target center point, and to obtain a position and a size of the deep feature representation of the time sequence based on the length and the width of the detection bounding box of the target center point; to perform multiple perceptions on the deep feature representation of the time sequence and a size scaling relationship between the input images and the deep feature representation of the time sequence to construct a target classification head and a target tracking feature head; to train the deep learning skeleton network, and in the training process, to obtain a position offset of the target center point based on the size scaling relationship between the input images and the deep feature representation of the time sequence, to obtain a category of the target based on the target classification head, and to obtain a tracking characteristic representation of the target based on the target tracking feature head; and to obtain a real position of the target based on the position of the deep feature representation of the time sequence and the position offset of the target center point. The tracking module is configured to construct a bipartite graph association model based on the position and the size of the deep feature representation of the time sequence, the position offset of the target center point, the category of the target, the tracking characteristic representation of the target, and a historical trajectory; and to obtain a multi-target tracking association result based on the bipartite graph association model.

5. A computer device, comprising: The multi-target tracking program is stored in the memory and executable on the processor, and the processor implements the method in any one of claims 1 to 3 when executing the multi-target tracking program.

Images