Target tracking method, system and related equipment for optimized calculation

Abstract

The present invention is applicable to the application field of artificial intelligence technology and provides a target tracking method, system, and related equipment for optimized calculation. The method includes: obtaining video sequence data and obtaining a pedestrian detection target with a pedestrian feature vector and a pedestrian coordinate frame in the current frame from the video sequence data; using a tracker to predict the position of a past pedestrian target identified in the previous frame to obtain a pedestrian prediction coordinate frame with a predicted feature vector and a tracking ID; calculating the intersection-and-union (IoU) between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determining whether the IoU is greater than a preset IoU threshold. If the IoU is greater than the preset IoU threshold, further calculating the cosine distance between the pedestrian feature vector and the predicted feature vector, and using the Hungarian algorithm to calculate the tracking matching result based on the IoU and cosine distance. The present invention adopts a tracking prediction method based on vectors and IoU to reduce the computational complexity of multi-target tracking and improve real-time performance.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology applications, and in particular relates to a target tracking method, system, and related equipment based on optimized calculations of a HiSilicon chip NNIE hardware platform. Background Art

[0002] Multi-target tracking is a research hotspot in computer vision. Combined with object detection, it has become an application for identifying and tracking the same target object in consecutive images. Multi-target tracking is widely used in security video surveillance, autonomous driving, and other fields, playing a particularly important role in security.

[0003] The main purpose of multi-target tracking is to estimate the movement trajectory and activity status of objects in subsequent video sequences. In addition to processing and identifying a series of image sequences and target detection information, it is also necessary to identify the same moving targets in different image frames and assign a separate label to each target. Existing multi-target tracking methods mainly use a tracking ID as a label. When the same target is tracked in subsequent video sequences, the existing tracking ID is assigned to the newly identified target.

[0004] In an existing multi-target tracking method, after the tracked object is identified through a neural network model, a tracker based on a Kalman filter is used to predict the position of the tracked object in the image frame. The Mahalanobis distance and cosine distance between the predicted position and the original position of the tracked object are then calculated. Finally, the Hungarian algorithm is used to perform correlation matching between the predicted position and the actual position to achieve target tracking in different video frames. At the same time, the existing multi-target tracking method also performs multiple matches related to the intersection-over-union results based on the correlation matching results to reduce false detections. In the above method, the actual distance and similarity of the tracked object in different video frames are calculated using the Mahalanobis distance and cosine distance, respectively, which can improve the accuracy of target tracking and thus reduce the jump of tracking IDs. However, for edge devices using the above method for multi-target tracking, their hardware computing performance is low. The calculation of the Mahalanobis distance requires frequent matrix calculations, which greatly increases the computational workload of the processor, affecting the overall performance of the edge device. At the same time, multiple matches also lead to unnecessary calculations, resulting in a waste of computing resources. Summary of the Invention

[0005] The embodiments of the present invention provide a target tracking method, system and related equipment for optimized calculation, which are based on the HiSilicon chip NNIE hardware platform and aim to solve the problem that traditional multi-target tracking methods use a large amount of matrix calculations and repeated calculations, resulting in the squeezing of edge device computing resources.

[0006] In a first aspect, an embodiment of the present invention provides a multi-target tracking method for optimized calculation, the method comprising:

[0007] Acquire video sequence data, and obtain a pedestrian detection target of a current frame and a pedestrian coordinate frame corresponding to the pedestrian detection target by identifying the pedestrian detection target from the video sequence data, wherein the pedestrian coordinate frame has a pedestrian feature vector;

[0008] Using a tracker to predict the position of a past pedestrian target identified in a frame before the current frame, to obtain a pedestrian prediction coordinate frame, wherein the pedestrian prediction coordinate frame has a prediction feature vector and a tracking ID;

[0009] Calculate the intersection-and-union (IoU) ratio between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determine whether the IoU ratio is greater than a preset IoU ratio threshold, wherein:

[0010] If the intersection-over-union ratio is less than or equal to the preset intersection-over-union ratio threshold, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue;

[0011] If the IoU is greater than the preset IoU threshold, the cosine distance between the pedestrian feature vector and the predicted feature vector is further calculated, and the tracking matching result is calculated using the Hungarian algorithm based on the IoU and the cosine distance.

[0012] Furthermore, the pedestrian feature vector and the predicted feature vector are calculated from the pedestrian detection target and the past pedestrian target respectively using a trained deep learning model.

[0013] Furthermore, the tracker includes a plurality of trackers, each of which tracks one of the past pedestrian targets in a frame before the current frame.

[0014] Furthermore, for the pedestrian prediction coordinate frame, the distance between the pedestrian prediction coordinate frame and the last successful matching video frame is marked by a matching counter.

[0015] Furthermore, the step of calculating the cosine distance between the pedestrian feature vector and the predicted feature vector, and obtaining the tracking matching result by using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance, includes the following sub-steps:

[0016] Calculating a cosine distance between the pedestrian feature vector and the predicted feature vector;

[0017] The tracking and matching result is calculated using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance, wherein:

[0018] If the Hungarian algorithm calculates a tracking matching result, outputting the tracking matching result;

[0019] If the Hungarian algorithm calculation does not obtain a tracking matching result, it is further determined whether the matching counter of the pedestrian prediction coordinate frame is greater than the maximum survival threshold, where:

[0020] If the matching counter is less than or equal to the maximum survival threshold, the pedestrian prediction coordinate frame is placed in the target tracking queue;

[0021] If the matching counter is greater than the maximum survival threshold, the pedestrian prediction coordinate frame is deleted, and the target tracking of the pedestrian prediction coordinate frame by the tracker is stopped.

[0022] Furthermore, the multi-target tracking method of the optimization calculation further includes the following steps:

[0023] Assigning the tracking ID of the pedestrian prediction coordinate frame as the tracking and matching result to the pedestrian coordinate frame;

[0024] The tracker is used to assign a tracking ID to the unmatched pedestrian coordinate frame in the target tracking queue, and target tracking is performed on the unmatched pedestrian coordinate frame and the pedestrian prediction coordinate frame.

[0025] Furthermore, the Hungarian algorithm is calculated based on a weighted average of the values ​​of the intersection-over-union ratio and the cosine distance.

[0026] In a second aspect, an embodiment of the present invention further provides a multi-target tracking system for optimized calculation, comprising:

[0027] a data acquisition module, configured to acquire video sequence data, and obtain a pedestrian detection target of a current frame and a pedestrian coordinate frame corresponding to the pedestrian detection target by identifying the pedestrian detection target from the video sequence data, wherein the pedestrian coordinate frame has a pedestrian feature vector;

[0028] a position prediction module, configured to use a tracker to predict the position of a pedestrian target identified in a previous frame of the current frame, and obtain a pedestrian prediction coordinate frame, wherein the pedestrian prediction coordinate frame has a prediction feature vector and a tracking ID;

[0029] The tracking and matching module is used to calculate the intersection-and-union ratio between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determine whether the intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold, wherein:

[0030] If the intersection-over-union ratio is less than or equal to the preset intersection-over-union ratio threshold, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue;

[0031] If the IoU is greater than the preset IoU threshold, the cosine distance between the pedestrian feature vector and the predicted feature vector is further calculated, and the tracking matching result is calculated using the Hungarian algorithm based on the IoU and the cosine distance.

[0032] In a third aspect, an embodiment of the present invention further provides a computer device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the multi-target tracking method of optimized calculation as described in any one of the above embodiments are implemented.

[0033] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the multi-target tracking method of optimized calculation as described in any one of the above embodiments are implemented.

[0034] The beneficial effects achieved by the present invention are that the tracking prediction based on vectors and intersection-over-union ratios replaces methods such as Kalman filtering and Mahalanobis distance that occupy a large amount of computing resources, thereby reducing the amount of computation for multi-target tracking while achieving similar results and enabling deployment on more hardware platforms. At the same time, due to the reduced computing power requirements, the real-time performance of multi-target tracking is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flowchart of the steps of the multi-target tracking method for optimization calculation provided by an embodiment of the present invention;

[0036] Figure 2 This is a sub-flow diagram of step S103b in the multi-target tracking method of optimization calculation provided by an embodiment of the present invention;

[0037] Figure 3 3 is a schematic structural diagram of a multi-target tracking system 300 for optimization calculation provided by an embodiment of the present invention;

[0038] Figure 4 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] Please refer to Figure 1 , Figure 1This is a flowchart of the steps of the multi-target tracking method for optimization calculation provided by an embodiment of the present invention. In this embodiment of the present invention, an edge device is used as an implementation carrier of the method. The multi-target tracking method for optimization calculation is based on the HiSilicon chip NNIE hardware platform. The NNIE platform is a relatively lightweight computer hardware platform commonly used in edge devices. The method of the embodiment of the present invention includes the following steps:

[0041] S101 , obtaining video sequence data, and obtaining a pedestrian detection target of a current frame and a pedestrian coordinate frame corresponding to the pedestrian detection target by identifying the pedestrian coordinate frame from the video sequence data, wherein the pedestrian coordinate frame has a pedestrian feature vector.

[0042] The video sequence data is video data captured in a continuous time period. The purpose of multi-target tracking is to track pedestrians in the video sequence data and mark the position of the same pedestrian in the frame image as the number of video frames increases, thereby achieving continuous marking of pedestrians in the video sequence data. The current frame refers to the video frame currently tracked when performing multi-target tracking. Each pedestrian therein is used as an object of multi-target tracking. In the embodiment of the present invention, it is the pedestrian detection target. The pedestrian detection target corresponds to its coordinate box information in the frame image of the current frame. In the embodiment of the present invention, the pedestrian feature vector is calculated from the pedestrian detection target using a trained deep learning model. The deep learning model can be any neural network model that realizes target detection in an image. At the same time, the deep learning model can also frame the detected target with a rectangular selection box. In the embodiment of the present invention, the pedestrian coordinate box refers to the position coordinate information of the selection box framed by the deep learning model in the entire image frame.

[0043] S102 : Use a tracker to predict the position of a pedestrian target identified in the previous frame of the current frame to obtain a pedestrian prediction coordinate frame, where the pedestrian prediction coordinate frame has a prediction feature vector and a tracking ID.

[0044] Specifically, the predicted feature vector is also calculated from the past pedestrian targets using a trained deep learning model. The trackers include multiple trackers, each of which tracks one of the past pedestrian targets in the previous frame of the current frame. For multi-target tracking, multiple pedestrian targets will be tracked simultaneously in the same video frame. In an embodiment of the present invention, the tracker is used to define an ongoing tracking task and ultimately output a tracking matching result for the target being tracked.

[0045] S103: Calculate an intersection-over-union (IoU) ratio between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determine whether the IoU ratio is greater than a preset IoU ratio threshold.

[0046] The intersection-in-union ratio refers to the proportion of the overlapping parts between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, which are both in the video frame. In an embodiment of the present invention, the preset intersection-in-union ratio threshold is a specific value used to determine whether the pedestrian coordinate frame and the pedestrian prediction coordinate frame are derived from the same pedestrian target to a certain extent.

[0047] According to the result of whether the IoU is greater than the preset IoU threshold, step S103 is further divided into:

[0048] S103a: If the IoU is less than or equal to the preset IoU threshold, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue.

[0049] In S103a, if the intersection-in-union ratio is less than or equal to the preset intersection-in-union ratio threshold, it means that the pedestrian prediction coordinate frame is far away from the pedestrian coordinate frame, and it is impossible to further determine whether the two are related to the same pedestrian target through the relationship between the numerical values. At this time, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue, and the target tracking queue is used to store pedestrian targets that have not been matched or are identified for the first time.

[0050] S103b: If the IoU is greater than the preset IoU threshold, further calculating the cosine distance between the pedestrian feature vector and the predicted feature vector, and obtaining a tracking matching result using the Hungarian algorithm based on the IoU and the cosine distance.

[0051] Furthermore, for the pedestrian prediction coordinate frame, a matching counter is used to mark the distance between the pedestrian prediction coordinate frame and the last successfully matched video frame. In this regard, in multi-target tracking, there may be a situation where there is no matching tracking result for the tracked target in multiple video frames. In this embodiment of the present invention, the matching calculator is used to mark the number of times the pedestrian prediction coordinate frame appears in the above situation. Please refer to Figure 2 , Figure 2 1 is a sub-flow chart of step S103b in the multi-target tracking method of optimization calculation provided by an embodiment of the present invention. Step S103b specifically includes the following sub-steps:

[0052] S201: Calculate the cosine distance between the pedestrian feature vector and the predicted feature vector.

[0053] S202 : Calculate the tracking and matching result using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance.

[0054] Specifically, the Hungarian algorithm is a combinatorial optimization algorithm used to find data pairs that meet the minimum gap in a data set. In an embodiment of the present invention, the Hungarian algorithm is used to track and match the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and match the pedestrian prediction coordinate frame that is closest to the pedestrian coordinate frame in the current frame. In an embodiment of the present invention, the Hungarian algorithm is calculated based on the weighted average of the intersection-over-union ratio and the cosine distance. According to whether the Hungarian algorithm calculates a tracking and matching result, step S202 is further divided into:

[0055] S202a: If the Hungarian algorithm calculates a tracking and matching result, the tracking and matching result is output.

[0056] The tracking and matching result is a pair of the pedestrian coordinate frame and the pedestrian prediction coordinate frame that match each other.

[0057] S202b: If the Hungarian algorithm calculation does not obtain a tracking matching result, further determine whether the matching counter of the pedestrian prediction coordinate frame is greater than a maximum survival threshold.

[0058] Specifically, the maximum survival threshold is a preset value used to determine whether the pedestrian prediction coordinate frame has not been matched for a long time, thereby losing the value of target tracking. For the pedestrian prediction coordinate frame that has not been matched for a long time, the possible reasons include that the corresponding pedestrian target has moved to an area outside the field of view, or is blocked by other environmental objects and cannot be accurately identified. In this case, it is necessary to interrupt tracking of the pedestrian prediction coordinate frame to relieve the process pressure of the tracking system. In step S202b, based on the result of whether the matching counter is greater than the maximum survival threshold, the following steps are further included:

[0059] S203a: If the matching counter is less than or equal to the maximum survival threshold, the pedestrian prediction coordinate frame is placed in the target tracking queue.

[0060] If the matching counter is less than or equal to the maximum survival threshold, it indicates that the pedestrian prediction coordinate frame is still within the range for target tracking, and there is a chance of finding a corresponding matching result in the next tracking match. At this time, the pedestrian prediction coordinate frame is placed in the target tracking queue.

[0061] S203b: If the matching counter is greater than the maximum survival threshold, the pedestrian prediction coordinate frame is deleted, and the target tracking of the pedestrian prediction coordinate frame by the tracker is stopped.

[0062] Preferably, the multi-target tracking method of optimized calculation provided by the embodiment of the present invention further includes the following steps:

[0063] S104: Assign the tracking ID of the pedestrian prediction coordinate frame as the tracking and matching result to the pedestrian coordinate frame.

[0064] The pedestrian coordinate frame and the pedestrian predicted coordinate frame are considered to match each other, because in the embodiment of the present invention, the tracking ID is not assigned to the pedestrian coordinate frame in priority. After the matching is completed, the tracking ID of the corresponding pedestrian predicted coordinate frame is assigned to the pedestrian coordinate frame.

[0065] S105 , using the tracker to assign a tracking ID to the unmatched pedestrian coordinate frame in the target tracking queue, and performing target tracking on the unmatched pedestrian coordinate frame and the pedestrian prediction coordinate frame.

[0066] For the target tracking queue, it includes the pedestrian prediction coordinate frame that has not been matched in the past video frame, and the pedestrian coordinate frame that has not been matched in the current frame, wherein the pedestrian prediction coordinate frame has been assigned the tracking ID, and the pedestrian coordinate frame is matched for the first time. Therefore, the tracking ID is first assigned to the pedestrian coordinate frame, and the pedestrian coordinate frame and the pedestrian prediction coordinate frame in the target tracking queue are further tracked.

[0067] The beneficial effects achieved by the present invention are that the tracking prediction based on vectors and intersection-over-union ratios replaces methods such as Kalman filtering and Mahalanobis distance that occupy a large amount of computing resources, thereby reducing the amount of computation for multi-target tracking while achieving similar results and enabling deployment on more hardware platforms. At the same time, due to the reduced computing power requirements, the real-time performance of multi-target tracking is also improved.

[0068] The embodiment of the present invention also provides a multi-target tracking system with optimized calculation, please refer to Figure 3 , Figure 3 3 is a schematic diagram of the structure of a multi-target tracking system 300 for optimizing calculations provided by an embodiment of the present invention, including:

[0069] A data acquisition module 301 is configured to acquire video sequence data and obtain a pedestrian detection target of a current frame and a pedestrian coordinate frame corresponding to the pedestrian detection target by identifying the pedestrian detection target from the video sequence data, wherein the pedestrian coordinate frame has a pedestrian feature vector;

[0070] A position prediction module 302 is configured to use a tracker to predict the position of a pedestrian target identified in a previous frame of the current frame to obtain a pedestrian prediction coordinate frame having a prediction feature vector and a tracking ID;

[0071] The tracking and matching module 303 is configured to calculate the intersection-and-union (IoU) ratio between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determine whether the IoU ratio is greater than a preset IoU ratio threshold, wherein:

[0072] If the intersection-over-union ratio is less than or equal to the preset intersection-over-union ratio threshold, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue;

[0073] If the IoU is greater than the preset IoU threshold, the cosine distance between the pedestrian feature vector and the predicted feature vector is further calculated, and the tracking matching result is calculated using the Hungarian algorithm based on the IoU and the cosine distance.

[0074] The multi-target tracking system 300 with optimized calculation can implement the steps in the multi-target tracking method with optimized calculation in the above embodiment, and can achieve the same technical effects. Please refer to the description in the above embodiment and will not be repeated here.

[0075] The embodiment of the present invention also provides a computer device, please refer to Figure 4 , Figure 4 4 is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. The computer device 400 includes: a memory 402, a processor 401, and a computer program stored in the memory 402 and executable on the processor 401.

[0076] The processor 401 calls the computer program stored in the memory 402 to execute the steps of the multi-target tracking method for optimization calculation provided by the embodiment of the present invention. Figure 1 , specifically including:

[0077] S101 , obtaining video sequence data, and obtaining a pedestrian detection target of a current frame and a pedestrian coordinate frame corresponding to the pedestrian detection target by identifying the pedestrian coordinate frame from the video sequence data, wherein the pedestrian coordinate frame has a pedestrian feature vector.

[0078] Furthermore, the pedestrian feature vector and the predicted feature vector are calculated from the pedestrian detection target and the past pedestrian target respectively using a trained deep learning model.

[0079] S102 : Use a tracker to predict the position of a pedestrian target identified in the previous frame of the current frame to obtain a pedestrian prediction coordinate frame, where the pedestrian prediction coordinate frame has a prediction feature vector and a tracking ID.

[0080] Furthermore, the tracker includes a plurality of trackers, each of which tracks one of the past pedestrian targets in a frame before the current frame.

[0081] Furthermore, for the pedestrian prediction coordinate frame, a matching counter is used to mark the distance between the pedestrian prediction coordinate frame and the last successfully matched video frame.

[0082] S103: Calculate the intersection-and-union (IoU) ratio between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determine whether the IoU ratio is greater than a preset IoU ratio threshold, wherein:

[0083] If the intersection-over-union ratio is less than or equal to the preset intersection-over-union ratio threshold, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue.

[0084] If the IoU is greater than the preset IoU threshold, the cosine distance between the pedestrian feature vector and the predicted feature vector is further calculated, and the tracking matching result is calculated using the Hungarian algorithm based on the IoU and the cosine distance.

[0085] Furthermore, the step of calculating the cosine distance between the pedestrian feature vector and the predicted feature vector, and obtaining the tracking matching result by using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance, includes the following sub-steps:

[0086] Calculating a cosine distance between the pedestrian feature vector and the predicted feature vector;

[0087] The tracking and matching result is calculated using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance, wherein:

[0088] If the Hungarian algorithm calculates a tracking matching result, outputting the tracking matching result;

[0089] If the Hungarian algorithm calculation does not obtain a tracking matching result, it is further determined whether the matching counter of the pedestrian prediction coordinate frame is greater than the maximum survival threshold, where:

[0090] If the matching counter is less than or equal to the maximum survival threshold, the pedestrian prediction coordinate frame is placed in the target tracking queue;

[0091] If the matching counter is greater than the maximum survival threshold, the pedestrian prediction coordinate frame is deleted, and the target tracking of the pedestrian prediction coordinate frame by the tracker is stopped.

[0092] Furthermore, the multi-target tracking method of the optimization calculation further includes the following steps:

[0093] S104: Assign the tracking ID of the pedestrian prediction coordinate frame as the tracking and matching result to the pedestrian coordinate frame.

[0094] S105 , using the tracker to assign a tracking ID to the unmatched pedestrian coordinate frame in the target tracking queue, and performing target tracking on the unmatched pedestrian coordinate frame and the pedestrian prediction coordinate frame.

[0095] Furthermore, the Hungarian algorithm is calculated based on a weighted average of the values ​​of the intersection-over-union ratio and the cosine distance.

[0096] The computer device 400 provided in the embodiment of the present invention can implement the steps in the multi-target tracking method of optimized calculation in the above embodiment and can achieve the same technical effects. Please refer to the description in the above embodiment and will not be repeated here.

[0097] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes and steps in the multi-target tracking method of optimized calculation provided by an embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0098] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0099] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0100] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0101] The embodiments of the present invention are described above in conjunction with the accompanying drawings. What is disclosed is only a preferred embodiment of the present invention. However, the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms and equivalent changes without departing from the scope of protection of the purpose of the present invention and the claims, which are all within the protection of the present invention.

Claims

1. A multi-target tracking method with optimized calculation, characterized in that: The method comprises: Acquire video sequence data, and obtain a pedestrian detection target of a current frame and a pedestrian coordinate frame corresponding to the pedestrian detection target by identifying the pedestrian detection target from the video sequence data, wherein the pedestrian coordinate frame has a pedestrian feature vector; Using a tracker to predict the position of a past pedestrian target identified in a frame before the current frame, to obtain a pedestrian prediction coordinate frame, wherein the pedestrian prediction coordinate frame has a prediction feature vector and a tracking ID; Calculate the intersection-and-union (IoU) ratio between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determine whether the IoU ratio is greater than a preset IoU ratio threshold, wherein: If the intersection-over-union ratio is less than or equal to the preset intersection-over-union ratio threshold, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue; If the IoU is greater than the preset IoU threshold, further calculating the cosine distance between the pedestrian feature vector and the predicted feature vector, and obtaining a tracking matching result using the Hungarian algorithm based on the IoU and the cosine distance; Wherein, for the pedestrian prediction coordinate frame, the distance between the pedestrian prediction coordinate frame and the last successful matching video frame is marked by a matching counter; The step of calculating the cosine distance between the pedestrian feature vector and the predicted feature vector, and obtaining a tracking matching result by using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance, comprises the following sub-steps: Calculating a cosine distance between the pedestrian feature vector and the predicted feature vector; The tracking and matching result is calculated using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance, wherein: If the Hungarian algorithm calculates a tracking matching result, outputting the tracking matching result; If the Hungarian algorithm calculation does not obtain a tracking matching result, it is further determined whether the matching counter of the pedestrian prediction coordinate frame is greater than the maximum survival threshold, where: If the matching counter is less than or equal to the maximum survival threshold, the pedestrian prediction coordinate frame is placed in the target tracking queue; If the matching counter is greater than the maximum survival threshold, the pedestrian prediction coordinate frame is deleted, and the target tracking of the pedestrian prediction coordinate frame by the tracker is stopped; The multi-target tracking method of the optimization calculation further comprises the following steps: Assigning the tracking ID of the pedestrian prediction coordinate frame as the tracking and matching result to the pedestrian coordinate frame; The tracker is used to assign a tracking ID to the unmatched pedestrian coordinate frame in the target tracking queue, and target tracking is performed on the unmatched pedestrian coordinate frame and the pedestrian prediction coordinate frame.

2. The multi-target tracking method of optimization calculation according to claim 1, characterized in that: The pedestrian feature vector and the predicted feature vector are respectively calculated from the pedestrian detection target and the past pedestrian target using a trained deep learning model.

3. The multi-target tracking method of optimization calculation according to claim 1, characterized in that: The trackers include a plurality of trackers, each of which tracks one of the past pedestrian targets in a frame previous to the current frame.

4. The multi-target tracking method of optimization calculation according to claim 1, characterized in that: The calculation basis of the Hungarian algorithm is the weighted average of the values ​​of the intersection-over-union ratio and the cosine distance.

5. A multi-target tracking system with optimized calculation, characterized in that: include: a data acquisition module, configured to acquire video sequence data, and obtain a pedestrian detection target of a current frame and a pedestrian coordinate frame corresponding to the pedestrian detection target by identifying the pedestrian detection target from the video sequence data, wherein the pedestrian coordinate frame has a pedestrian feature vector; a position prediction module, configured to use a tracker to predict the position of a pedestrian target identified in a previous frame of the current frame, and obtain a pedestrian prediction coordinate frame, wherein the pedestrian prediction coordinate frame has a prediction feature vector and a tracking ID; The tracking and matching module is used to calculate the intersection-and-union ratio between the pedestrian coordinate frame and the pedestrian prediction coordinate frame, and determine whether the intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold, wherein: If the intersection-over-union ratio is less than or equal to the preset intersection-over-union ratio threshold, the pedestrian coordinate frame and the pedestrian prediction coordinate frame are placed in a target tracking queue; If the IoU is greater than the preset IoU threshold, further calculating the cosine distance between the pedestrian feature vector and the predicted feature vector, and obtaining a tracking matching result using the Hungarian algorithm based on the IoU and the cosine distance; Wherein, for the pedestrian prediction coordinate frame, the distance between the pedestrian prediction coordinate frame and the last successful matching video frame is marked by a matching counter; The tracking and matching module is further configured to: Calculating a cosine distance between the pedestrian feature vector and the predicted feature vector; The tracking and matching result is calculated using the Hungarian algorithm according to the intersection-over-union ratio and the cosine distance, wherein: If the Hungarian algorithm calculates a tracking matching result, outputting the tracking matching result; If the Hungarian algorithm calculation does not obtain a tracking matching result, it is further determined whether the matching counter of the pedestrian prediction coordinate frame is greater than the maximum survival threshold, where: If the matching counter is less than or equal to the maximum survival threshold, the pedestrian prediction coordinate frame is placed in the target tracking queue; If the matching counter is greater than the maximum survival threshold, the pedestrian prediction coordinate frame is deleted, and the target tracking of the pedestrian prediction coordinate frame by the tracker is stopped; The tracking and matching module is further configured to: Assigning the tracking ID of the pedestrian prediction coordinate frame as the tracking and matching result to the pedestrian coordinate frame; The tracker is used to assign a tracking ID to the unmatched pedestrian coordinate frame in the target tracking queue, and target tracking is performed on the unmatched pedestrian coordinate frame and the pedestrian prediction coordinate frame.

6. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the multi-target tracking method with optimized calculation according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multi-target tracking method for optimization calculation according to any one of claims 1 to 4.

Images